* Last steps for pretesting @ 2006-04-17 0:56 Richard Stallman 2006-04-17 8:01 ` Ramprasad B ` (3 more replies) 0 siblings, 4 replies; 80+ messages in thread From: Richard Stallman @ 2006-04-17 0:56 UTC (permalink / raw) As far as I can see, all we need to do before we start pretesting is 1. fix the recently reported bugs 2. update the copyright notices (all files need 2006) 3. check a few manual files: anti.texi glossary.texi macos.texi maintaining.texi misc.texi msdog.texi trouble.texi and some of the tasks from the start of FOR-RELEASE. ** update facemenu-unlisted-faces. ** Regenerate the postscript files of the reference cards in etc. ** Ask maintainers of refcard translations to update them. ** Check what should be deleted or updated in MORE.STUFF. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-17 0:56 Last steps for pretesting Richard Stallman @ 2006-04-17 8:01 ` Ramprasad B 2006-04-17 8:37 ` Eli Zaretskii 2006-04-18 1:42 ` Richard Stallman 2006-04-18 17:31 ` Bill Wohler ` (2 subsequent siblings) 3 siblings, 2 replies; 80+ messages in thread From: Ramprasad B @ 2006-04-17 8:01 UTC (permalink / raw) Cc: emacs-devel [-- Attachment #1.1: Type: text/plain, Size: 509 bytes --] Richard Stallman <rms@gnu.org> wrote: > 2. update the copyright notices (all files need 2006) 1. Do we have to add `2006' also for a file which hasn't been edited in 2006 ? 2. If a file has 2001 copyright year, how is 2006 copyright year added ? Is it like 2001, 2006 or 2001-2006 or 2001, 2002, 2003, 2004, 2005, 2006 ? Thanks. -- Ramprasad B --------------------------------- Yahoo! Messenger with Voice. Make PC-to-Phone Calls to the US (and 30+ countries) for 2¢/min or less. [-- Attachment #1.2: Type: text/html, Size: 684 bytes --] [-- Attachment #2: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-17 8:01 ` Ramprasad B @ 2006-04-17 8:37 ` Eli Zaretskii 2006-04-18 1:42 ` Richard Stallman 1 sibling, 0 replies; 80+ messages in thread From: Eli Zaretskii @ 2006-04-17 8:37 UTC (permalink / raw) Cc: rms, emacs-devel > Date: Mon, 17 Apr 2006 01:01:58 -0700 (PDT) > From: Ramprasad B <ramprasad_i82@yahoo.com> > Cc: emacs-devel@gnu.org > > Richard Stallman <rms@gnu.org> wrote: > > > 2. update the copyright notices (all files need 2006) > > 1. Do we have to add `2006' also for a file which hasn't been edited in 2006 ? Yes. > 2. If a file has 2001 copyright year, how is 2006 copyright year added ? > Is it like 2001, 2006 or 2001-2006 or 2001, 2002, 2003, 2004, 2005, 2006 ? I think just add 2006, but please wait for Richard to give the definitive answer. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-17 8:01 ` Ramprasad B 2006-04-17 8:37 ` Eli Zaretskii @ 2006-04-18 1:42 ` Richard Stallman 2006-04-19 3:54 ` Ramprasad B 1 sibling, 1 reply; 80+ messages in thread From: Richard Stallman @ 2006-04-18 1:42 UTC (permalink / raw) Cc: emacs-devel 1. Do we have to add `2006' also for a file which hasn't been edited in 2006 ? Yes. All files should list 2006. 2. If a file has 2001 copyright year, how is 2006 copyright year added ? Is it like 2001, 2006 or 2001-2006 or 2001, 2002, 2003, 2004, 2005, 2006 ? All the files have 2005, now, don't they? Do you see any file which lacks 2005 in its copyright notice? So you just have to add 2006 at the end. There should be a function copyright-update which does this job. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-18 1:42 ` Richard Stallman @ 2006-04-19 3:54 ` Ramprasad B 2006-04-19 15:40 ` Richard Stallman 0 siblings, 1 reply; 80+ messages in thread From: Ramprasad B @ 2006-04-19 3:54 UTC (permalink / raw) Cc: emacs-devel --- Richard Stallman <rms@gnu.org> wrote: > Yes. All files should list 2006. Ok. > All the files have 2005, now, don't they? > Do you see any file which lacks 2005 in its copyright notice? Yes. Many, like in emacs/admin -> admin.el, alloc-colors.c, etc. > There should be a function copyright-update which does this job. Will do that. Thanks. -- Ramprasad B __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-19 3:54 ` Ramprasad B @ 2006-04-19 15:40 ` Richard Stallman 2006-04-20 5:23 ` Ramprasad B 0 siblings, 1 reply; 80+ messages in thread From: Richard Stallman @ 2006-04-19 15:40 UTC (permalink / raw) Cc: emacs-devel > All the files have 2005, now, don't they? > Do you see any file which lacks 2005 in its copyright notice? Yes. Many, like in emacs/admin -> admin.el, alloc-colors.c, etc. The admin directory is not terribly important, since it isn't part of the release. It also is not very big, so it would not be hard to fix the files in it. So how about if you fix them too? ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-19 15:40 ` Richard Stallman @ 2006-04-20 5:23 ` Ramprasad B 0 siblings, 0 replies; 80+ messages in thread From: Ramprasad B @ 2006-04-20 5:23 UTC (permalink / raw) Cc: emacs-devel [-- Attachment #1.1: Type: text/plain, Size: 389 bytes --] Richard Stallman <rms@gnu.org> wrote: > The admin directory is not terribly important, since it isn't part of > the release. Ok. > So how about if you fix them too? Yes fixed emacs/admin completely and fixing some more too. Thanks. -- Ramprasad B --------------------------------- New Yahoo! Messenger with Voice. Call regular phones from your PC and save big. [-- Attachment #1.2: Type: text/html, Size: 564 bytes --] [-- Attachment #2: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-17 0:56 Last steps for pretesting Richard Stallman 2006-04-17 8:01 ` Ramprasad B @ 2006-04-18 17:31 ` Bill Wohler 2006-04-19 17:02 ` Alan Mackenzie 2006-04-20 10:54 ` Reiner Steib 3 siblings, 0 replies; 80+ messages in thread From: Bill Wohler @ 2006-04-18 17:31 UTC (permalink / raw) Richard Stallman <rms@gnu.org> writes: > As far as I can see, all we need to do before we start pretesting is > ... > and some of the tasks from the start of FOR-RELEASE. ** Ensure MH-E 8.0 has been released. Assigned to Bill Wohler <wohler@newt.com>. Inches (centimeters) away. I'm down to two more tasks. > ** Check what should be deleted or updated in MORE.STUFF. Wasn't aware of this file. Added URL for MH-E. I'm glad to hear we're getting close! -- Bill Wohler <wohler@newt.com> http://www.newt.com/wohler/ GnuPG ID:610BD9AD Maintainer of comp.mail.mh FAQ and MH-E. Vote Libertarian! If you're passed on the right, you're in the wrong lane. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-17 0:56 Last steps for pretesting Richard Stallman 2006-04-17 8:01 ` Ramprasad B 2006-04-18 17:31 ` Bill Wohler @ 2006-04-19 17:02 ` Alan Mackenzie 2006-04-19 21:23 ` Stefan Monnier ` (2 more replies) 2006-04-20 10:54 ` Reiner Steib 3 siblings, 3 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-19 17:02 UTC (permalink / raw) Cc: emacs-devel Hi, Richard, Hi, all! On Sun, 16 Apr 2006, Richard Stallman wrote: >As far as I can see, all we need to do before we start pretesting is >1. fix the recently reported bugs >2. update the copyright notices (all files need 2006) >3. check a few manual files: I'm struggling somewhat to get CC Mode in a presentable state. I'm getting a steady trickle of syntax analysis bugs at bug-cc-mode (e.g., the "DEFUN" problem) which I'm fixing at about the same rate they're coming in at. Thanks to Michaël Cadilhac[*] for sending a patch in with "his" bug! I currently have two or three such bugs pending at the moment (I've lost count). [*] He tells me his copyright assignment papers are currently being processed. Besides these, there are some issues with connecting CC Mode up with Emacs 22: Foremost amongst them are: (i) Fixing the submode indicators (the "/la" of "C/la" on the mode line) so that their doc-string is displayed when the mouse hovers over them. (ii) Writing font-lock-expand-region functions, one for CC Mode as a whole (to replace (setq font-lock-lines-before 2)) and one for AWK Mode (to replace the advising of the Font Lock after-change functions). (iii) Tracking down the "CC Mode loads CL" problem. (I've made good progress on this). (iv) A problem in Emacs (See: Subject: Problem mit symlinks, locate-library and load-history. Date: Tue, 19 Mar 2006). This is caused by (a) load-history storing a promiscuous mix of true file names and file names containing symbolic links; (b) eval-after-load only recognising files as being already loaded if they're the ones currently showing on load-path. I've half-solved this problem by writing a function to search load-history properly, which I posted to emacs-devel. Richard tentatively approved it. Would it be possible to define load-history such that all file names in it are true names? Of these, I could use help for (i) and (iv); (ii) is fairly trivial, (iii) is something it is best for me to do. There are also several relatively minor issues, some of which I've reported to emacs-devel, some of which other people have reported to bug-cc-mode, which haven't been addressed yet. I feel a bit like the wet blanket at a party at the moment. I'm not going to get all of these things finished this week, or even next week. Maybe the week after. Sorry. Richard, when do you envisage going into pretest? -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-19 17:02 ` Alan Mackenzie @ 2006-04-19 21:23 ` Stefan Monnier 2006-04-19 21:30 ` David Kastrup ` (2 more replies) 2006-04-20 1:14 ` Last steps for pretesting Richard Stallman 2006-04-24 17:52 ` Richard Stallman 2 siblings, 3 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-19 21:23 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel > (ii) Writing font-lock-expand-region functions, one for CC Mode as a > whole (to replace (setq font-lock-lines-before 2)) and one for AWK Mode > (to replace the advising of the Font Lock after-change functions). BTW, I still intend to move font-lock-extend-region-function from after-change-functions to font-lock-fontify-region (or maybe even jit-lock-fontify). So if you really manage to use the current font-lock-extend-region-function (called from after-change-functions) in a way that's robust, efficient, and can't be done as efficiently/elegantly/robustly with a hook in font-lock-fontify-region (or with the font-lock-multiline property), please scream. > (iv) A problem in Emacs (See: Subject: Problem mit symlinks, > locate-library and load-history. Date: Tue, 19 Mar 2006). This is > caused by (a) load-history storing a promiscuous mix of true file names > and file names containing symbolic links; (b) eval-after-load only > recognising files as being already loaded if they're the ones currently > showing on load-path. I've half-solved this problem by writing a > function to search load-history properly, which I posted to emacs-devel. > Richard tentatively approved it. Would it be possible to define > load-history such that all file names in it are true names? As mentioned in that thread, I think it's better to "truify" names on-the-fly when you need it rather than directly inside load-history. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-19 21:23 ` Stefan Monnier @ 2006-04-19 21:30 ` David Kastrup 2006-04-20 21:35 ` Stefan Monnier 2006-04-19 22:43 ` Last steps for pretesting (font-lock-extend-region-function) Alan Mackenzie 2006-04-19 22:53 ` Last steps for pretesting (true file names in load-history?) Alan Mackenzie 2 siblings, 1 reply; 80+ messages in thread From: David Kastrup @ 2006-04-19 21:30 UTC (permalink / raw) Cc: Alan Mackenzie, Richard Stallman, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > As mentioned in that thread, I think it's better to "truify" names > on-the-fly when you need it rather than directly inside > load-history. Uh, but the truification has to happen when loading, anyway. One should not replicate the operation unnecessarily at load time, true, but since the info has to be generated anyway, one could think about how to make it accessible without additional overhead. If I am completely bonkers, do have a pointer to that thread? -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-19 21:30 ` David Kastrup @ 2006-04-20 21:35 ` Stefan Monnier 2006-04-20 21:37 ` David Kastrup 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-20 21:35 UTC (permalink / raw) Cc: Alan Mackenzie, Richard Stallman, emacs-devel > Uh, but the truification has to happen when loading, anyway. No, AFAIK "truification" doesn't currently happen at load time. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-20 21:35 ` Stefan Monnier @ 2006-04-20 21:37 ` David Kastrup 2006-04-20 21:44 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: David Kastrup @ 2006-04-20 21:37 UTC (permalink / raw) Cc: Alan Mackenzie, Richard Stallman, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> Uh, but the truification has to happen when loading, anyway. > > No, AFAIK "truification" doesn't currently happen at load time. I would consider it surprising if Emacs was able to load files without finding out where they are. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-20 21:37 ` David Kastrup @ 2006-04-20 21:44 ` Stefan Monnier 2006-04-20 21:46 ` David Kastrup 2006-04-20 22:50 ` Alan Mackenzie 0 siblings, 2 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-20 21:44 UTC (permalink / raw) Cc: Alan Mackenzie, Richard Stallman, emacs-devel >>> Uh, but the truification has to happen when loading, anyway. >> No, AFAIK "truification" doesn't currently happen at load time. > I would consider it surprising if Emacs was able to load files without > finding out where they are. I'd be surprised if you didn't know that if "/foo" is a symlink to "/bar", Emacs doesn't need to know it in order to load "/foo/toto/titi.elc", i.e. it doesn't need to first convert it to "/bar/toto/titi.elc". There must some kind of of misunderstanding. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-20 21:44 ` Stefan Monnier @ 2006-04-20 21:46 ` David Kastrup 2006-04-20 22:50 ` Alan Mackenzie 1 sibling, 0 replies; 80+ messages in thread From: David Kastrup @ 2006-04-20 21:46 UTC (permalink / raw) Cc: Alan Mackenzie, Richard Stallman, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >>>> Uh, but the truification has to happen when loading, anyway. >>> No, AFAIK "truification" doesn't currently happen at load time. >> I would consider it surprising if Emacs was able to load files without >> finding out where they are. > > I'd be surprised if you didn't know that if "/foo" is a symlink to > "/bar", Emacs doesn't need to know it in order to load > "/foo/toto/titi.elc", i.e. it doesn't need to first convert it to > "/bar/toto/titi.elc". There must some kind of of misunderstanding. I probably confused the issue with making file names absolute or resolving them in load-path. Sorry, -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-20 21:44 ` Stefan Monnier 2006-04-20 21:46 ` David Kastrup @ 2006-04-20 22:50 ` Alan Mackenzie 1 sibling, 0 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-20 22:50 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel On Thu, 20 Apr 2006, Stefan Monnier wrote: >>>> Uh, but the truification has to happen when loading, anyway. >>> No, AFAIK "truification" doesn't currently happen at load time. >> I would consider it surprising if Emacs was able to load files without >> finding out where they are. >I'd be surprised if you didn't know that if "/foo" is a symlink to "/bar", >Emacs doesn't need to know it in order to load "/foo/toto/titi.elc", i.e. it >doesn't need to first convert it to "/bar/toto/titi.elc". >There must some kind of of misunderstanding. Emacs DOES need to know, every time it evaluates an eval-after-load. For example, if this form is evaluated: (eval-after-load "/mnt/hda7/emacs/lisp/cc-fonts.elc" '(...)) and load-history has as one of its elements ("/home/acm/emacs/emacs/lisp/cc-fonts.elc" ....), ; a symlink then each stored file name must be truified at _every_ eval-after-load. -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-19 21:23 ` Stefan Monnier 2006-04-19 21:30 ` David Kastrup @ 2006-04-19 22:43 ` Alan Mackenzie 2006-04-20 9:13 ` Alan Mackenzie 2006-04-20 17:46 ` Stefan Monnier 2006-04-19 22:53 ` Last steps for pretesting (true file names in load-history?) Alan Mackenzie 2 siblings, 2 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-19 22:43 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Evening, Stefan! On Wed, 19 Apr 2006, Stefan Monnier wrote: >> (ii) Writing font-lock-expand-region functions, one for CC Mode as a >> whole (to replace (setq font-lock-lines-before 2)) and one for AWK Mode >> (to replace the advising of the Font Lock after-change functions). >BTW, I still intend to move font-lock-extend-region-function from >after-change-functions to font-lock-fontify-region (or maybe even >jit-lock-fontify). That would radically change the meaning of the function. Please keep Font Lock working equivalently with and without JIT switched on. Edebugging through font-lock-fontify-region (for debugging a mode's font lock settings) is a nightmare when JIT is enabled. How about the following compromise? font-lock-extend-region-function should get called BOTH in the two after-change functions AND in jit-lock-fontify-now and font-lock-default-fontify-region. The third parameter (OLD-LEN) gets bound to nil in the new case. In either case the extend-region function is entitled to return nil. My AWK extend region function would then look like this: (defun c-awk-font-lock-extend-region (beg end old-len) (cons (c-awk-beginning-of-logical-line beg) (if old-len ;; From an after-change-function (c-awk-end-of-change-region beg end old-len) ;; From a fontifying routine (c-awk-end-of-logical-line end)))) >So if you really manage to use the current >font-lock-extend-region-function (called from after-change-functions) in >a way that's robust, efficient, ..... So far, yes (except, perhaps, for the need of an extend-region function in j-l-f-n and f-l-d-f-r). >.... and can't be done as efficiently/elegantly/robustly with a hook in >font-lock-fontify-region (or with the font-lock-multiline property), >please scream. AAAAAAAAARRRRRRRRRRRRRRGGGGGGGGGGGGGGGGGGHHHHHHHHHHHHHHHHHHHHHH!!!!!!!!!!! I don't think we're going to agree on the relative elegance of font-lock-extend-region-function vs. font-lock-multiline. I think that the former is more elegant than the latter, and you think the latter is the more elegant. I'm convinced that either approach would be robust (hey, we're competent hackers ;-). However, I think that f-l-extend-region-function can be done more efficiently by the major mode maintainer, in the sense that it will require less time and effort reading manuals, reading fine source, and coding. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-19 22:43 ` Last steps for pretesting (font-lock-extend-region-function) Alan Mackenzie @ 2006-04-20 9:13 ` Alan Mackenzie 2006-04-20 17:46 ` Stefan Monnier 1 sibling, 0 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-20 9:13 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, again, Stefan! On Wed, 19 Apr 2006, Alan Mackenzie wrote: >On Wed, 19 Apr 2006, Stefan Monnier wrote: >>So if you really manage to use the current >>font-lock-extend-region-function (called from after-change-functions) >>in a way that's robust, efficient, ..... >So far, yes (except, perhaps, for the need of an extend-region function >in j-l-f-n and f-l-d-f-r). >>.... and can't be done as efficiently/elegantly/robustly with a hook in >>font-lock-fontify-region (or with the font-lock-multiline property), >>please scream. >AAAAAAAAARRRRRRRRRRRRRRGGGGGGGGGGGGGGGGGGHHHHHHHHHHHHHHHHHHHHHH!!!!!!!!!!! >I don't think we're going to agree on the relative elegance of >font-lock-extend-region-function vs. font-lock-multiline. I think that >the former is more elegant than the latter, and you think the latter is >the more elegant. >I'm convinced that either approach would be robust (hey, we're competent >hackers ;-). >However, I think that f-l-extend-region-function can be done more >efficiently by the major mode maintainer, in the sense that it will >require less time and effort reading manuals, reading fine source, and >coding. Also critically important: font-lock-extend-region-function is documented in the Elisp manual. The font-lock-multiline property isn't. Would you please write this documentation now. Then it would be much easier for Richard, me and everybody else to compare the two approaches. Thanks in anticipation! >> Stefan >Alan. -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-19 22:43 ` Last steps for pretesting (font-lock-extend-region-function) Alan Mackenzie 2006-04-20 9:13 ` Alan Mackenzie @ 2006-04-20 17:46 ` Stefan Monnier 2006-04-20 18:12 ` Alan Mackenzie 1 sibling, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-20 17:46 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >> BTW, I still intend to move font-lock-extend-region-function from >> after-change-functions to font-lock-fontify-region (or maybe even >> jit-lock-fontify). > That would radically change the meaning of the function. Of course. > Please keep Font Lock working equivalently with and without JIT switched > on. Edebugging through font-lock-fontify-region (for debugging a mode's > font lock settings) is a nightmare when JIT is enabled. Yes, OK, don't worry about the jit side-comment. > How about the following compromise? font-lock-extend-region-function > should get called BOTH in the two after-change functions AND in > jit-lock-fontify-now and font-lock-default-fontify-region. No, my problem is specifically with calling it from after-change-functions. >> So if you really manage to use the current >> font-lock-extend-region-function (called from after-change-functions) in >> a way that's robust, efficient, ..... > So far, yes (except, perhaps, for the need of an extend-region function > in j-l-f-n and f-l-d-f-r). So your code needs extend-region both in a-c-f and in f-l-d-f-r ? >> .... and can't be done as efficiently/elegantly/robustly with a hook in >> font-lock-fontify-region (or with the font-lock-multiline property), >> please scream. > AAAAAAAAARRRRRRRRRRRRRRGGGGGGGGGGGGGGGGGGHHHHHHHHHHHHHHHHHHHHHH!!!!!!!!!!! > I don't think we're going to agree on the relative elegance of > font-lock-extend-region-function vs. font-lock-multiline. I think that > the former is more elegant than the latter, and you think the latter is > the more elegant. Of course, we don't have to agree on every detail. > I'm convinced that either approach would be robust (hey, we're competent > hackers ;-). I'm convinced that only using a-c-f would *not* be robust, no matter how competent the hacker. > However, I think that f-l-extend-region-function can be done more > efficiently by the major mode maintainer, in the sense that it will > require less time and effort reading manuals, reading fine source, and > coding. more efficiently than what? Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-20 17:46 ` Stefan Monnier @ 2006-04-20 18:12 ` Alan Mackenzie 2006-04-20 20:58 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-20 18:12 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan! On Thu, 20 Apr 2006, Stefan Monnier wrote: >>> BTW, I still intend to move font-lock-extend-region-function from >>> after-change-functions to font-lock-fontify-region (or maybe even >>> jit-lock-fontify). >> That would radically change the meaning of the function. >Of course. >> How about the following compromise? font-lock-extend-region-function >> should get called BOTH in the two after-change functions AND in >> jit-lock-fontify-now and font-lock-default-fontify-region. >No, my problem is specifically with calling it from >after-change-functions. You mean in calling it ONLY from a-c-f? Or from calling it in after-change-functions at all? >>> So if you really manage to use the current >>> font-lock-extend-region-function (called from after-change-functions) >>> in a way that's robust, efficient, ..... >> So far, yes (except, perhaps, for the need of an extend-region >> function in j-l-f-n and f-l-d-f-r). >So your code needs extend-region both in a-c-f and in f-l-d-f-r ? Yes. The region needs extending in two distinct circumstances: (i) In after-change, to determine the entire region which needs refontifying; (ii) In jit-lock-fontify-now, to ensure that each chunk starts and stops at a "safe place". These are logically disjoint. But it seems that the major mode code to do each of them is so similar, that we might as well just have a single function, font-lock-extend-region-function, called in two different ways. >>> .... and can't be done as efficiently/elegantly/robustly with a hook >>> in font-lock-fontify-region (or with the font-lock-multiline >>> property), please scream. >> AAAAAAAAARRRRRRRRRRRRRRGGGGGGGGGGGGGGGGGGHHHHHHHHHHHHHHHHHHHHHH!!!!!!!!!!! >> I don't think we're going to agree on the relative elegance of >> font-lock-extend-region-function vs. font-lock-multiline. I think >> that the former is more elegant than the latter, and you think the >> latter is the more elegant. >Of course, we don't have to agree on every detail. :-) >> I'm convinced that either approach would be robust (hey, we're competent >> hackers ;-). >I'm convinced that only using a-c-f would *not* be robust, no matter how >competent the hacker. I agree with you here. >> However, I think that f-l-extend-region-function can be done more >> efficiently by the major mode maintainer, in the sense that it will >> require less time and effort reading manuals, reading fine source, and >> coding. >more efficiently than what? More efficiently than setting font-lock-multiline text properties. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-20 18:12 ` Alan Mackenzie @ 2006-04-20 20:58 ` Stefan Monnier 2006-04-20 22:40 ` Alan Mackenzie 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-20 20:58 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >> So your code needs extend-region both in a-c-f and in f-l-d-f-r ? > Yes. The region needs extending in two distinct circumstances: > (i) In after-change, to determine the entire region which needs > refontifying; > (ii) In jit-lock-fontify-now, to ensure that each chunk starts and stops > at a "safe place". > These are logically disjoint. But it seems that the major mode code to > do each of them is so similar, that we might as well just have a single > function, font-lock-extend-region-function, called in two different ways. IIRC part (i) needs more than an after-change-function: it also needs a before-change-function. And (again IIRC) it's the part that can already be solved by placing (from one of the font-lock-keywords rules) a font-lock-multiline property. Basically it's the part that makes sure that if a multiline element loses its multiline status, the whole (previously atomic) text gets properly refreshed. I find it very hard to believe that b-f-c + a-f-c ends up being as efficient/elegant/robust as placing a font-lock-multiline property. In all likely hood, adding the font-lock-multiline property is a matter of adding a single line of code at the right place (the place where you fontify the corresponding multiline entity). Here is the problems I see with your proposal: - an `extend-region' hook in font-lock-fontify-region is needed - the font-lock-multiline property should be enough in all cases to make it unnecessary to use an after-change-function hook. - adding an after-change-function hook additionally to a font-lock-fontify-region hook means making font-lock that much more complex to use and maintain, especially since that after-change-function basically can't be used without also adding a before-change-function. - an after-change-function is expensive, and even more when it comes bundled with a before-change-function, and even more so when compared to code that's only run during font-locking. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-20 20:58 ` Stefan Monnier @ 2006-04-20 22:40 ` Alan Mackenzie 2006-04-20 23:34 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-20 22:40 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan! On Thu, 20 Apr 2006, Stefan Monnier wrote: >>> So your code needs extend-region both in a-c-f and in f-l-d-f-r ? >> Yes. The region needs extending in two distinct circumstances: >> (i) In after-change, to determine the entire region which needs >> refontifying; >> (ii) In jit-lock-fontify-now, to ensure that each chunk starts and stops >> at a "safe place". >> These are logically disjoint. But it seems that the major mode code to >> do each of them is so similar, that we might as well just have a single >> function, font-lock-extend-region-function, called in two different ways. >IIRC part (i) needs more than an after-change-function: it also needs >a before-change-function. That would depend on the major mode, I think. AWK mode needs a before-change function. The CC Mode replacement for (setq font-lock-lines-before 2) will not need a b-c function. However, using the f-l-multiline property approach instead will also need a before-change function for exactly the same reasons - or else it will require the major mode to maintain the f-l-multiline property permanently. That would be a major overhead. >And (again IIRC) it's the part that can already be solved by placing >(from one of the font-lock-keywords rules) a font-lock-multiline >property. I'm not sure. Can a font-lock-keyword rule get access to after-change's OLD-LEN in an reasonable fashion? If a change consists of yanking 20k of text into a buffer, the Major Mode must not splat font-lock-multiline onto 20k + 20 bytes, since this would defeat the purpose of JIT lock (fontifying a bit at a time). The Major Mode would have to apply f-l-m separately to the head and tail of the (large) region separately, but in the normal case (small change region) would have to combine them as a single region. It would need logic do distinguish these two cases. This is inelegant. >Basically it's the part that makes sure that if a multiline >element loses its multiline status, the whole (previously atomic) text >gets properly refreshed. How is this to be done with the multiline property? >I find it very hard to believe that b-f-c + a-f-c ends up being as >efficient/elegant/robust as placing a font-lock-multiline property. In >all likely hood, adding the font-lock-multiline property is a matter of >adding a single line of code at the right place (the place where you >fontify the corresponding multiline entity). Stefan, the font-lock-multiline property is still undocumented. That makes it very frustrating, very difficult to evaluate the paragraph you've just written, very difficult to criticise it constructively (I think I've perhaps already done enough of the other sort of criticism ;-). Exactly where the "right place" is to add that single line of code is completely opaque to me at the moment. I could probably work it out with several hours study of the Font Lock source and documentation, but I don't want to do that at the moment - I've got enough on my plate. Could you please document the font-lock-multiline mechanism immediately, including details on how best to access the after-change-functions's OLD-LEN, and details of how and where to hook in the "(put-text-property a b 'font-lock-multiline t)" call. Either that, or tell me where to find an example of this text property being used, preferably one which uses OLD-LEN. Or, perhaps you could recode the following AWK Mode fragment into an equivalent which uses f-l-multiline instead of f-l-expand-region function. PLEASE! ######################################################################### (defvar c-awk-old-EOLL 0) (make-variable-buffer-local 'c-awk-old-EOLL) ;; End of logical line following the region which is about to be changed. Set ;; in c-awk-before-change and used in c-awk-font-lock-extend-region. (defun c-awk-before-change (beg end) ;; This function is called exclusively from the before-change-functions hook. ;; It does two things: Finds the end of the (logical) line on which END lies, ;; and clears c-awk-NL-prop text properties from this point onwards. (save-restriction (save-excursion (setq c-awk-old-EOLL (c-awk-end-of-logical-line end)) (c-save-buffer-state nil (c-awk-clear-NL-props end (point-max)))))) (add-hook 'before-change-functions c-awk-before-change nil t) (defun c-awk-end-of-change-region (beg end old-len) ;; Find the end of the region which needs to be font-locked after a change. ;; This is the end of the logical line on which the change happened, either ;; as it was before the change, or as it is now, whichever is later. ;; N.B. point is left undefined. (max (+ (- c-awk-old-EOLL old-len) (- end beg)) (c-awk-end-of-logical-line end))) (defun c-awk-font-lock-extend-region (beg end old-len) (cons (c-awk-beginning-of-logical-line beg) (if old-len ;; From an after-change-function (c-awk-end-of-change-region beg end old-len) ;; From a fontifying routine (c-awk-end-of-logical-line end)))) (setq font-lock-extend-region-function 'c-awk-font-lock-extend-region) ######################################################################### >Here is the problems I see with your proposal: >- an `extend-region' hook in font-lock-fontify-region is needed It is indeed. It is a simple patch, and I am ready and willing to write it, both for font-core.el and modes.texi. I do first ask that it's accepted in principle, though. >- the font-lock-multiline property should be enough in all cases to make > it unnecessary to use an after-change-function hook. f-l-extend-region-function also makes a (separate) after-change-f hook unnecessary. >- adding an after-change-function hook additionally to > a font-lock-fontify-region hook means making font-lock that much more > complex to use and maintain, especially since that after-change-function > basically can't be used without also adding a before-change-function. I disagree with everything in that paragraph. :-) >- an after-change-function is expensive, and even more when it comes bundled > with a before-change-function, and even more so when compared to > code that's only run during font-locking. Huh? Font Locking _is_ expensive. A before-change function need not be expensive (e.g. c-awk-before-change, see above), and calling it is not expensive. I suspect that using the f-l-multiline property will absolutely mandate an extra after-change function - how else can it access OLD-LEN? Stefan, please show me the code! (Or the documentation!). How does a hacker use the font-lock-multiline property? In detail. -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-20 22:40 ` Alan Mackenzie @ 2006-04-20 23:34 ` Stefan Monnier 2006-04-21 7:58 ` Alan Mackenzie 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-20 23:34 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >> IIRC part (i) needs more than an after-change-function: it also needs >> a before-change-function. > That would depend on the major mode, I think. AWK mode needs a > before-change function. The CC Mode replacement for (setq > font-lock-lines-before 2) will not need a b-c function. Simulating font-lock-lines-before only needs an `extend-region' hook in font-lock-default-fontify-region, a subject about which we already agree. > However, using the f-l-multiline property approach instead will also need > a before-change function for exactly the same reasons That's news to me. Could you explain the scenario you have in mind? > - or else it will require the major mode to maintain the f-l-multiline > property permanently. The font-lock-multiline property is removed by font-lock-unfontify-region and it is expected that it gets re-added (if needed) every time you fontify a piece of text. It's never seemed to be a big deal. No before-change-function in sight. >> And (again IIRC) it's the part that can already be solved by placing >> (from one of the font-lock-keywords rules) a font-lock-multiline >> property. > I'm not sure. Can a font-lock-keyword rule get access to after-change's > OLD-LEN in an reasonable fashion? What for? The font-lock-multiline it needs to add is for use by the next font-lock round. It says: "next time you fontify this area, make sure you refontify this piece atomically". > If a change consists of yanking 20k of text into a buffer, the Major Mode > must not splat font-lock-multiline onto 20k + 20 bytes, since this would > defeat the purpose of JIT lock (fontifying a bit at a time). The font-lock-multiline property is added not to the whole fontified text, but just to a particular region of text that matched a pattern (a "keyword"). I think you're absolutely right: I have to document it because you seem to completely misunderstand it. > Could you please document the font-lock-multiline mechanism immediately, > including details on how best to access the after-change-functions's > OLD-LEN, and details of how and where to hook in the "(put-text-property a > b 'font-lock-multiline t)" call. Either that, or tell me where to find > an example of this text property being used, preferably one which uses > OLD-LEN. AFAIK the OLD-LEN is never needed. See font-lock.el for an example of code that sets font-lock-multiline property: if a regexp matches several lines, it puts the font-lock-multiline property on the wholse matched text (this is only done if the font-lock-multiline variable is set). > Or, perhaps you could recode the following AWK Mode fragment into an > equivalent which uses f-l-multiline instead of f-l-expand-region > function. PLEASE! I'll try to, although I'd need to see the font-lock-keywords part (where I'll have to add the code). > ######################################################################### > (defvar c-awk-old-EOLL 0) > (make-variable-buffer-local 'c-awk-old-EOLL) > ;; End of logical line following the region which is about to be changed. Set > ;; in c-awk-before-change and used in c-awk-font-lock-extend-region. > (defun c-awk-before-change (beg end) > ;; This function is called exclusively from the before-change-functions hook. > ;; It does two things: Finds the end of the (logical) line on which END lies, > ;; and clears c-awk-NL-prop text properties from this point onwards. > (save-restriction > (save-excursion > (setq c-awk-old-EOLL (c-awk-end-of-logical-line end)) > (c-save-buffer-state nil > (c-awk-clear-NL-props end (point-max)))))) > (add-hook 'before-change-functions c-awk-before-change nil t) > (defun c-awk-end-of-change-region (beg end old-len) > ;; Find the end of the region which needs to be font-locked after a change. > ;; This is the end of the logical line on which the change happened, either > ;; as it was before the change, or as it is now, whichever is later. > ;; N.B. point is left undefined. > (max (+ (- c-awk-old-EOLL old-len) (- end beg)) > (c-awk-end-of-logical-line end))) > (defun c-awk-font-lock-extend-region (beg end old-len) > (cons (c-awk-beginning-of-logical-line beg) > (if old-len > ;; From an after-change-function > (c-awk-end-of-change-region beg end old-len) > ;; From a fontifying routine > (c-awk-end-of-logical-line end)))) > (setq font-lock-extend-region-function 'c-awk-font-lock-extend-region) > ######################################################################### IIUC, something like the following should do: (defun c-awk-font-lock-extend-region (beg end) (cons (c-awk-beginning-of-logical-line beg) (c-awk-end-of-logical-line end))) (setq font-lock-extend-region-function 'c-awk-font-lock-extend-region) (defconst c-awk-font-lock-keywords '(... (".*\\\\\n.*" (0 (progn (put-text-property (match-beginning 0) (match-end 0) 'font-lock-multiline t) nil))) ...)) although I suspect that you already have a font-lock-keyword that matches something akin to ".*\\\\\n.*" where you can add the put-text-property. >> Here is the problems I see with your proposal: >> - an `extend-region' hook in font-lock-fontify-region is needed > It is indeed. It is a simple patch, and I am ready and willing to write > it, both for font-core.el and modes.texi. I do first ask that it's > accepted in principle, though. It's completely accepted. I just hope we can call it "font-lock-extend-region-function". >> - the font-lock-multiline property should be enough in all cases to make >> it unnecessary to use an after-change-function hook. > f-l-extend-region-function also makes a (separate) after-change-f hook > unnecessary. Which f-l-extend-region-function? The one called from after-change-f? Then it's what I meant by "an after-change-function hook". The fact that it's not called directly from a-c-f but via f-t-a-c-f is not very important for this discussion. >> - an after-change-function is expensive, and even more when it comes bundled >> with a before-change-function, and even more so when compared to >> code that's only run during font-locking. > Huh? Font Locking _is_ expensive. Yup, but (thanks to jit-lock) it's not executed after each and every buffer modification. > A before-change function need not be expensive (e.g. c-awk-before-change, > see above), and calling it is not expensive. The code will have probably the same cost whether it's called from a-c-f or from font-lock, but in one case it'll be called (much) more often. > I suspect that using the f-l-multiline property will absolutely mandate an > extra after-change function - how else can it access OLD-LEN? Why would it need OLD-LEN? Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-20 23:34 ` Stefan Monnier @ 2006-04-21 7:58 ` Alan Mackenzie 2006-04-21 12:18 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-21 7:58 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Good morning, Stefan! On Thu, 20 Apr 2006, Stefan Monnier wrote: >>> IIRC part (i) needs more than an after-change-function: it also needs >>> a before-change-function. >> That would depend on the major mode, I think. AWK mode needs a >> before-change function. The CC Mode replacement for (setq >> font-lock-lines-before 2) will not need a b-c function. >Simulating font-lock-lines-before only needs an `extend-region' hook in >font-lock-default-fontify-region, a subject about which we already agree. We do. :-) >> - or else it will require the major mode to maintain the f-l-multiline >> property permanently. >The font-lock-multiline property is removed by font-lock-unfontify-region >and it is expected that it gets re-added (if needed) every time you fontify >a piece of text. It's never seemed to be a big deal. >No before-change-function in sight. That's why I think it won't work in AWK mode. For Font locking triggered by a change, you need the details of the _change_, not merely how the buffer looks after it. >>> And (again IIRC) it's the part that can already be solved by placing >>> (from one of the font-lock-keywords rules) a font-lock-multiline >>> property. >> I'm not sure. Can a font-lock-keyword rule get access to after-change's >> OLD-LEN in an reasonable fashion? >What for? The font-lock-multiline it needs to add is for use by the next >font-lock round. It says: "next time you fontify this area, make sure you >refontify this piece atomically". The font locking is triggered by the Change. Therefore it needs details of that change to determine what region of text needs refontifying. It is up to the major mode maintainer, whether she needs those details or not. AWK Mode, for example, does need them. >> If a change consists of yanking 20k of text into a buffer, the Major Mode >> must not splat font-lock-multiline onto 20k + 20 bytes, since this would >> defeat the purpose of JIT lock (fontifying a bit at a time). >The font-lock-multiline property is added not to the whole fontified text, >but just to a particular region of text that matched a pattern (a >"keyword"). I think you're absolutely right: I have to document it because >you seem to completely misunderstand it. :-) Thanks! >> Could you please document the font-lock-multiline mechanism >> immediately, including details on how best to access the >> after-change-functions's OLD-LEN, and details of how and where to hook >> in the "(put-text-property a b 'font-lock-multiline t)" call. Either >> that, or tell me where to find an example of this text property being >> used, preferably one which uses OLD-LEN. >AFAIK the OLD-LEN is never needed. It is most definitely needed. I don't think you have understood what I have been trying to say. :-( >See font-lock.el for an example of code that sets font-lock-multiline >property: if a regexp matches several lines, it puts the >font-lock-multiline property on the wholse matched text (this is only >done if the font-lock-multiline variable is set). Perhaps I have truly misunderstood you. I thought you were telling me that some code contained within the major mode would do the (put-text-property ... 'font-lock-multiline ...). Are you really saying that the major mode merely has to set up its font-lock-keywords so that font-lock.el finds the right places to apply the text property? >> Or, perhaps you could recode the following AWK Mode fragment into an >> equivalent which uses f-l-multiline instead of f-l-expand-region >> function. PLEASE! >I'll try to, although I'd need to see the font-lock-keywords part (where >I'll have to add the code). It's right at the end of cc-fonts.el. >> ######################################################################### >> (defvar c-awk-old-EOLL 0) >> (make-variable-buffer-local 'c-awk-old-EOLL) >> ;; End of logical line following the region which is about to be changed. Set >> ;; in c-awk-before-change and used in c-awk-font-lock-extend-region. >> (defun c-awk-before-change (beg end) >> ;; This function is called exclusively from the before-change-functions hook. >> ;; It does two things: Finds the end of the (logical) line on which END lies, >> ;; and clears c-awk-NL-prop text properties from this point onwards. >> (save-restriction >> (save-excursion >> (setq c-awk-old-EOLL (c-awk-end-of-logical-line end)) >> (c-save-buffer-state nil >> (c-awk-clear-NL-props end (point-max)))))) >> (add-hook 'before-change-functions c-awk-before-change nil t) >> (defun c-awk-end-of-change-region (beg end old-len) >> ;; Find the end of the region which needs to be font-locked after a change. >> ;; This is the end of the logical line on which the change happened, either >> ;; as it was before the change, or as it is now, whichever is later. >> ;; N.B. point is left undefined. >> (max (+ (- c-awk-old-EOLL old-len) (- end beg)) >> (c-awk-end-of-logical-line end))) >> (defun c-awk-font-lock-extend-region (beg end old-len) >> (cons (c-awk-beginning-of-logical-line beg) >> (if old-len >> ;; From an after-change-function >> (c-awk-end-of-change-region beg end old-len) >> ;; From a fontifying routine >> (c-awk-end-of-logical-line end)))) >> (setq font-lock-extend-region-function 'c-awk-font-lock-extend-region) >> ######################################################################### >IIUC, something like the following should do: >(defun c-awk-font-lock-extend-region (beg end) > (cons (c-awk-beginning-of-logical-line beg) > (c-awk-end-of-logical-line end))) <================================== >(setq font-lock-extend-region-function 'c-awk-font-lock-extend-region) That, quite demonstrably, WON'T do, since that will fail to extend the region to what was the end of the logical line before the change. >(defconst c-awk-font-lock-keywords > '(... > (".*\\\\\n.*" > (0 (progn (put-text-property (match-beginning 0) (match-end 0) > 'font-lock-multiline t) > nil))) > ...)) OK, I think I'm beginning to get the idea. >>> Here is the problems I see with your proposal: >>> - an `extend-region' hook in font-lock-fontify-region is needed >> It is indeed. It is a simple patch, and I am ready and willing to write >> it, both for font-core.el and modes.texi. I do first ask that it's >> accepted in principle, though. >It's completely accepted. I just hope we can call it >"font-lock-extend-region-function". It has been accepted (by Richard, no less), as part of the font-lock-after-change-function/jit-lock-after-change. I'm proposing to enhance it to do the necessary region extension from within f-l-default-fontify-region/j-l-fontify-now too. >>> - the font-lock-multiline property should be enough in all cases to make >>> it unnecessary to use an after-change-function hook. >> f-l-extend-region-function also makes a (separate) after-change-f hook >> unnecessary. >Which f-l-extend-region-function? The one called from after-change-f? Yes. >Then it's what I meant by "an after-change-function hook". The fact that >it's not called directly from a-c-f but via f-t-a-c-f is not very >important for this discussion. I think it's important. It saves the major mode maintainer from having to install his code as a separate a-c-function, and juggling them around to make sure they get called in the correct order. >>> - an after-change-function is expensive, and even more when it comes >>> bundled with a before-change-function, and even more so when compared >>> to code that's only run during font-locking. >> Huh? Font Locking _is_ expensive. >Yup, but (thanks to jit-lock) it's not executed after each and every >buffer modification. OK. I think your point is that the f-l-extend-region-function called from f-l-a-f-f/j-l-a-c is expensive because it is called after _every_ change. That is potentially true, and it is up to the major mode writer to ensure that this isn't a problem. In C Mode, for example, c-extend-region-function (as it will be in CC Mode 5.32/Emacs 23.1) will be an extensive thing, involving sophisticated caching, sneaky heuristics, and so on, so as it will run fast enough. In AWK Mode, the function is trivial enough that it's already not a problem. >> A before-change function need not be expensive (e.g. c-awk-before-change, >> see above), and calling it is not expensive. >The code will have probably the same cost whether it's called from a-c-f or >from font-lock, but in one case it'll be called (much) more often. Yes. But that calling from a-c-f is essential to correct font locking. >> I suspect that using the f-l-multiline property will absolutely >> mandate an extra after-change function - how else can it access >> OLD-LEN? >Why would it need OLD-LEN? Stefan, I've tried to answer this question several times already, but I've clearly not made a good job of it. I'm getting very frustrated with not being able to get my point across to you. PLEASE go through this canonical example in all its gory detail and tell me you've understood it. Please tell me that you understand what I'm doing in the c-awk-before-change, and what I'm doing with OLD-LEN. Please tell me you see why the way you suggested (89 lines higher up) wouldn't work. If you can then tell me a way of achieving the same effect WITHOUT the use of before-change and/or OLD-LEN, I'd be happy to hear about it. OK, here goes! ######################################################################### In AWK Mode: Point is at EOL 3, and we delete the backslash there. 1. "string \ 2. over \ 3. several \ <========= point is at EOL 3, about to delete the \ 4. #lines." 1. In c-awk-before-change (see above for the code), we calculate c-awk-old-EOLL ("end of old logical line"). This give 36, (EOL 4). 2. The \ gets deleted, and the text looks like this: 1. "string \ 2. over \ 3. several 4. #lines." 3. jit-lock-after-change gets called as (jit-lock-after-change 26 26 1), which in its turn calls (c-awk-font-lock-extend-region 26 26 1). This "1" is the OLD-LEN, of course. 4. c-awk-f-l-e-r calls (c-awk-end-of-change-region 26 26 1). This does the following: (i) It determines the current position of the PREVIOUS end of logical line: (+ (- c-awk-old-EOLL old-len) (- end beg)), => (+ (- 36 1) (- 26 26)) => 35 NOTE THE USE OF old-len. (ii) It determines the current end of logical line: (c-awk-end-of-logical 26) => 26. This is EOL 3 (iii) It selects the greater of these two positions: (max (+ (- c-awk-old-EOLL old-len) (- end beg)) (c-awk-end-of-logical-line end)) => (max 35 26) => 35. This 35 is the current EOL 4. (iv) 35 is returned to jit-lock-after-change in the cons (1 . 35), the extended fontification region. ######################################################################### > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-21 7:58 ` Alan Mackenzie @ 2006-04-21 12:18 ` Stefan Monnier 2006-04-21 19:51 ` Alan Mackenzie 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-21 12:18 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel > Good morning, Stefan! Good morning Alan, > The font locking is triggered by the Change. Therefore it needs details > of that change to determine what region of text needs refontifying. It No, it only needs to know what the current text looks like (obviously) and how the text was font-locked last time (to properly remove highlighting where it doesn't apply any more). This last part canbe obtained via the font-lock-multiline property. > Perhaps I have truly misunderstood you. I thought you were telling me > that some code contained within the major mode would do the > (put-text-property ... 'font-lock-multiline ...). Are you really saying > that the major mode merely has to set up its font-lock-keywords so that > font-lock.el finds the right places to apply the text property? I've generally been assuming that your code would explicitly do a put-text-property, but indeed font-lock.el includes some code which tries to do that for you. >> IIUC, something like the following should do: >> (defun c-awk-font-lock-extend-region (beg end) >> (cons (c-awk-beginning-of-logical-line beg) >> (c-awk-end-of-logical-line end))) <================================== >> (setq font-lock-extend-region-function 'c-awk-font-lock-extend-region) > That, quite demonstrably, WON'T do, since that will fail to extend the > region to what was the end of the logical line before the change. Of course it's not enough: it only takes care of making sure current atomic elements are properly fontified, but not that previously atomic elements are properly refontified. That's what the font-lock-multiline property is for. >>>> Here is the problems I see with your proposal: >>>> - an `extend-region' hook in font-lock-fontify-region is needed >>> It is indeed. It is a simple patch, and I am ready and willing to write >>> it, both for font-core.el and modes.texi. I do first ask that it's >>> accepted in principle, though. >> It's completely accepted. I just hope we can call it >> "font-lock-extend-region-function". > It has been accepted (by Richard, no less), as part of the > font-lock-after-change-function/jit-lock-after-change. I'm proposing to > enhance it to do the necessary region extension from within > f-l-default-fontify-region/j-l-fontify-now too. I consider the two as separate: the first is already installed (but I want it removed) and the second is not installed yet, but we agree that it should be added. I.e. I want it moved from a-c-f to f-l-d-f-r (it should not be added to j-l-fontify-now). >>>> - the font-lock-multiline property should be enough in all cases to make >>>> it unnecessary to use an after-change-function hook. >>> f-l-extend-region-function also makes a (separate) after-change-f hook >>> unnecessary. >> Which f-l-extend-region-function? The one called from after-change-f? > Yes. >> Then it's what I meant by "an after-change-function hook". The fact that >> it's not called directly from a-c-f but via f-t-a-c-f is not very >> important for this discussion. > I think it's important. It saves the major mode maintainer from having > to install his code as a separate a-c-function, and juggling them around > to make sure they get called in the correct order. Yes it is important in general. But this discussion is about the need to have *any* a-c-f hook. If we need one, its place is in f-l-a-c-f, indeed, but I argue that we don't need one, so the precise place is not really relevant to the discussion. >> The code will have probably the same cost whether it's called from a-c-f or >> from font-lock, but in one case it'll be called (much) more often. > Yes. But that calling from a-c-f is essential to correct font locking. That's what I claim is not true. > ######################################################################### > In AWK Mode: > Point is at EOL 3, and we delete the backslash there. > 1. "string \ > 2. over \ > 3. several \ <========= point is at EOL 3, about to delete the \ > 4. #lines." OK: if the text had never been font-locked before, an a hook in f-l-fontify-region is all we need, right? So the interesting case is when the text had already been fontified. In this case, the whole multiline-line has had a font-lock-multiline added, so at this point in your example, the multiline-line is 100% covered by a font-lock-multiline property. > 1. In c-awk-before-change (see above for the code), we calculate > c-awk-old-EOLL ("end of old logical line"). This give 36, (EOL 4). Here let's say instead we don't do anything. > 2. The \ gets deleted, and the text looks like this: > 1. "string \ > 2. over \ > 3. several > 4. #lines." OK, so the whole text from line 1 to line 4 still has a font-lock-multiline property. > 3. jit-lock-after-change gets called as (jit-lock-after-change 26 26 1), > which in its turn calls (c-awk-font-lock-extend-region 26 26 1). This > "1" is the OLD-LEN, of course. Here let's also say that instead we don't do anything special. > 4. c-awk-f-l-e-r calls (c-awk-end-of-change-region 26 26 1). This does > the following: > (i) It determines the current position of the PREVIOUS end of logical > line: (+ (- c-awk-old-EOLL old-len) (- end beg)), > => (+ (- 36 1) (- 26 26)) > => 35 > NOTE THE USE OF old-len. > (ii) It determines the current end of logical line: > (c-awk-end-of-logical 26) > => 26. This is EOL 3 > (iii) It selects the greater of these two positions: > (max (+ (- c-awk-old-EOLL old-len) (- end beg)) > (c-awk-end-of-logical-line end)) > => (max 35 26) > => 35. > This 35 is the current EOL 4. > (iv) 35 is returned to jit-lock-after-change in the cons (1 . 35), the > extended fontification region. > ######################################################################### Here let's also say that instead we don't do anything special. But when we later get to f-l-fontify-region, the code looks at the font-lock-multiline property at the boundary (i.e. at the beginning and end of line 3) and noticies that it's set, so it extends the region to be fontified to span all 4 lines. Just what you wanted all along. Look ma! No old-len! Tadaaa! If you insert text (instead of removing it), it gets a bit more interesting (because the inserted text doesn't have a font-lock-multiline property) but it basically works along the same lines. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-21 12:18 ` Stefan Monnier @ 2006-04-21 19:51 ` Alan Mackenzie 2006-04-21 22:28 ` Stefan Monnier 2006-04-21 23:14 ` Drew Adams 0 siblings, 2 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-21 19:51 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Good evening, Stefan! On Fri, 21 Apr 2006, Stefan Monnier wrote: >> Good morning, Stefan! >Good morning Alan, >> The font locking is triggered by the Change. Therefore it needs details >> of that change to determine what region of text needs refontifying. It >No, it only needs to know what the current text looks like (obviously) and >how the text was font-locked last time (to properly remove highlighting >where it doesn't apply any more). This last part can be obtained via the >font-lock-multiline property. OK. I think you're right here, but I'm not 100% sure. >> Perhaps I have truly misunderstood you. I thought you were telling me >> that some code contained within the major mode would do the >> (put-text-property ... 'font-lock-multiline ...). Are you really >> saying that the major mode merely has to set up its font-lock-keywords >> so that font-lock.el finds the right places to apply the text >> property? >I've generally been assuming that your code would explicitly do >a put-text-property, but indeed font-lock.el includes some code which tries >to do that for you. It would do this, presumably, in an after-change (or before-change) function, either directly on one of these hooks or called from font-lock-after-change. >>> IIUC, something like the following should do: >>> (defun c-awk-font-lock-extend-region (beg end) >>> (cons (c-awk-beginning-of-logical-line beg) >>> (c-awk-end-of-logical-line end))) <================================== >>> (setq font-lock-extend-region-function 'c-awk-font-lock-extend-region) >> That, quite demonstrably, WON'T do, since that will fail to extend the >> region to what was the end of the logical line before the change. >Of course it's not enough: it only takes care of making sure current >atomic elements are properly fontified, but not that previously atomic >elements are properly refontified. That's what the font-lock-multiline >property is for. OK. So the font-lock-multiline property is put on exactly a region of text which needs fontifying atomically, yet straddles a line break. An after-change function (and/or a before-change function) is what will do this putting and erasing of f-l-multiline. >>>>> Here is the problems I see with your proposal: >>>>> - an `extend-region' hook in font-lock-fontify-region is needed >>>> It is indeed. It is a simple patch, and I am ready and willing to write >>>> it, both for font-core.el and modes.texi. I do first ask that it's >>>> accepted in principle, though. >>> It's completely accepted. I just hope we can call it >>> "font-lock-extend-region-function". >> It has been accepted (by Richard, no less), as part of the >> font-lock-after-change-function/jit-lock-after-change. I'm proposing to >> enhance it to do the necessary region extension from within >> f-l-default-fontify-region/j-l-fontify-now too. >I consider the two as separate: the first is already installed (but I want >it removed) and the second is not installed yet, but we agree that it should >be added. I.e. I want it moved from a-c-f to f-l-d-f-r (it should not be >added to j-l-fontify-now). I think it's more accurate just to say you want to remove the extend-region stuff from f-l-after-change. I want it to stay. :-) Why don't we call the thing we need in f-l-d-f-r "font-lock-extend-CHUNK-function", since it's more likely to be a jit-lock chunk than a region supplied by the user or major mode? What is your objection to f-l-extend-region-f in a-c-f? Is it because it might gobble too much processor time? It needs to be in j-l-fontify-now, so that that function knows what bytes to apply the 'fontified property to. Some refactoring of the code might be helpful - say, extracting `font-lock-core-fontify-region' from font-lock-default-fontify-region, this new function fontifying exactly the region it is given, and doing it immediately. >>>>> - the font-lock-multiline property should be enough in all cases to >>>>> make it unnecessary to use an after-change-function hook. >>>> f-l-extend-region-function also makes a (separate) after-change-f hook >>>> unnecessary. >>> Which f-l-extend-region-function? The one called from after-change-f? >> Yes. >>> Then it's what I meant by "an after-change-function hook". The fact that >>> it's not called directly from a-c-f but via f-t-a-c-f is not very >>> important for this discussion. >> I think it's important. It saves the major mode maintainer from having >> to install his code as a separate a-c-function, and juggling them around >> to make sure they get called in the correct order. >Yes it is important in general. But this discussion is about the need to >have *any* a-c-f hook. If we need one, its place is in f-l-a-c-f, indeed, >but I argue that we don't need one, so the precise place is not really >relevant to the discussion. I can't see how a major mode can manipulate f-l-multiline properties anywhere BUT an a-c-f (or a before-c-f). >>> The code will have probably the same cost whether it's called from >>> a-c-f or from font-lock, but in one case it'll be called (much) more >>> often. >> Yes. But that calling from a-c-f is essential to correct font >> locking. >That's what I claim is not true. I can't see that yet, but I'm trying to. I think you're saying that instead of AWK Mode recording information in c-awk-old-EOLL ("c-awk old end of logical line") in the before-change function, it would make some setting of font-lock-multiline over the pertinent part of the buffer. >> ######################################################################### >> In AWK Mode: >> Point is at EOL 3, and we delete the backslash there. >> 1. "string \ >> 2. over \ >> 3. several \ <========= point is at EOL 3, about to delete the \ >> 4. #lines." >OK: if the text had never been font-locked before, a hook in >f-l-fontify-region is all we need, right? So the interesting case is >when the text had already been fontified. In this case, the whole >multiline-line has had a font-lock-multiline added, so at this point in >your example, the multiline-line is 100% covered by a >font-lock-multiline property. OK. >> 1. In c-awk-before-change (see above for the code), we calculate >> c-awk-old-EOLL ("end of old logical line"). This give 36, (EOL 4). >Here let's say instead we don't do anything. OK. >> 2. The \ gets deleted, and the text looks like this: >> 1. "string \ >> 2. over \ >> 3. several >> 4. #lines." >OK, so the whole text from line 1 to line 4 still has a font-lock-multiline >property. OK. >> 3. jit-lock-after-change gets called as (jit-lock-after-change 26 26 1), >> which in its turn calls (c-awk-font-lock-extend-region 26 26 1). This >> "1" is the OLD-LEN, of course. >Here let's also say that instead we don't do anything special. OK. >> 4. c-awk-f-l-e-r calls (c-awk-end-of-change-region 26 26 1). This does >> the following: >> (i) It determines the current position of the PREVIOUS end of logical >> line: (+ (- c-awk-old-EOLL old-len) (- end beg)), >> => (+ (- 36 1) (- 26 26)) >> => 35 >> NOTE THE USE OF old-len. >> (ii) It determines the current end of logical line: >> (c-awk-end-of-logical 26) >> => 26. This is EOL 3 >> (iii) It selects the greater of these two positions: >> (max (+ (- c-awk-old-EOLL old-len) (- end beg)) >> (c-awk-end-of-logical-line end)) >> => (max 35 26) >> => 35. >> This 35 is the current EOL 4. >> (iv) 35 is returned to jit-lock-after-change in the cons (1 . 35), the >> extended fontification region. >> ######################################################################### >Here let's also say that instead we don't do anything special. OK. >But when we later get to f-l-fontify-region, the code looks at the >font-lock-multiline property at the boundary (i.e. at the beginning and end >of line 3) and noticies that it's set, so it extends the region to be >fontified to span all 4 lines. Just what you wanted all along. OK, but ..... >Look ma! No old-len! Tadaaa! .... but some code, somewhere, sometime needs to adjust the f-l-m property, removing it from L4. In general, it will also need to _split_ the text property into L1,2,3 and L4,5,... Otherwise, a f-l-m region could gradually engulf an entire buffer, making a nonsense of jit-lock. (OK, this would be an extreme situation. :-) >If you insert text (instead of removing it), it gets a bit more >interesting (because the inserted text doesn't have a >font-lock-multiline property) but it basically works along the same >lines. Something, somewhere, sometime has to analyse the newly inserted code, putting f-l-m properties throughout it, I think, and adjusting those at the boundaries of the new region. Is this to be done at the time of the change or at the time it gets font locked? I think it has to be done at the change, otherwise the f-l-m settings will be wrong when it comes time to fontify a chunk. ######################################################################### SUMMARY: 1. I think font-lock-extend-region-function, called as it is from j-l-after-change, is a good thing. It is a correct way of giving a major mode an opportunity to massage a fontification region. I think it should stay. You think it should be removed. 2. f-l-d-f-r / j-l-f-now absolutely need a "font-lock-extend-chunk-function" functionality (regardless of its precise name). We agree on this. 3. You have convinced me that it is possible to use the f-l-m properties. 4. I think that f-l-extend-region-f, called from f-l-after-change, is likely to be more convenient and less error prone than the direct manipulation of the f-l-multiline property by major mode code. 5. I suspect you think that f-l-extend-region-f is likely to consume more processor time than direct manipulation of the f-l-m property. Is there any reason why we can't leave both methods side by side in the code? > Stefan BTW: I'll be away from the net over the next couple of days. ;-) -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-21 19:51 ` Alan Mackenzie @ 2006-04-21 22:28 ` Stefan Monnier 2006-04-24 19:28 ` Alan Mackenzie 2006-04-21 23:14 ` Drew Adams 1 sibling, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-21 22:28 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >> I've generally been assuming that your code would explicitly do >> a put-text-property, but indeed font-lock.el includes some code which tries >> to do that for you. > It would do this, presumably, in an after-change (or before-change) > function, either directly on one of these hooks or called from > font-lock-after-change. No, it's done in font-lock-default-fontify-region. > OK. So the font-lock-multiline property is put on exactly a region of > text which needs fontifying atomically, yet straddles a line break. An > after-change function (and/or a before-change function) is what will do > this putting and erasing of f-l-multiline. No, it's done in font-lock-default-fontify-region. > I think it's more accurate just to say you want to remove the > extend-region stuff from f-l-after-change. I want it to stay. :-) > Why don't we call the thing we need in f-l-d-f-r > "font-lock-extend-CHUNK-function", since it's more likely to be a > jit-lock chunk than a region supplied by the user or major mode? Since it's a hook run from font-lock-default-fontify-region, it makes sense to call it "extend-region", don't you think? Naming (and documenting) something based on how it is used rather than what it does is usually a bad idea. > What is your objection to f-l-extend-region-f in a-c-f? Is it because it > might gobble too much processor time? That and the fact that it's not necessary, and that to use it (as seen in your example) you need a good bit more work/code than doing it "the other way" (with code solely run from font-lock-default-fontify-region). > It needs to be in j-l-fontify-now, so that that function knows what bytes > to apply the 'fontified property to. This is not needed for correctness. It's only a matter of avoiding redundant work. This need wouldn't be new (it has existed since Emacs-21 becaue of font-lock-multiline) and doesn't need to be addressed right away. >> Yes it is important in general. But this discussion is about the need to >> have *any* a-c-f hook. If we need one, its place is in f-l-a-c-f, indeed, >> but I argue that we don't need one, so the precise place is not really >> relevant to the discussion. > I can't see how a major mode can manipulate f-l-multiline properties > anywhere BUT an a-c-f (or a before-c-f). What about the sample solution I provided: it's all manipulated from font-lock-keywords (i.e. from font-lock-default-fontify-region)? >> Look ma! No old-len! Tadaaa! > .... but some code, somewhere, sometime needs to adjust the f-l-m > property, removing it from L4. In general, it will also need to _split_ > the text property into L1,2,3 and L4,5,... Otherwise, a f-l-m region > could gradually engulf an entire buffer, making a nonsense of jit-lock. > (OK, this would be an extreme situation. :-) Yes, of course. It's done this way: font-lock-unfontify-region (called from font-lock-default-fontify-region) removes the font-lock-mutliline property on the whole fontified region jsut before proceeding with the actual fontification, so the font-lock-keywords need to re-add it at those places where it's still needed. >> If you insert text (instead of removing it), it gets a bit more >> interesting (because the inserted text doesn't have a >> font-lock-multiline property) but it basically works along the same >> lines. > Something, somewhere, sometime has to analyse the newly inserted code, > putting f-l-m properties throughout it, I think, and adjusting those at > the boundaries of the new region. Is this to be done at the time of the > change or at the time it gets font locked? I think it has to be done at > the change, otherwise the f-l-m settings will be wrong when it comes time > to fontify a chunk. Let me extend my example. In the last episode, we stopped just before refontification was to take place and had seen that the f-l-m property was going to make the refontification apply to all 4 lines, as needed. The fontification will work as follows: 5. font-lock-default-fontify-region gets asked to refontify line 3. 6. it extends the refontified region at beg and at end because of the presence of f-l-m property. So the refontified region now spans all 4 lines. 7. Prior to applying the new fontification, font-lock-unfontify-region is called which removes all `face' properties and `font-lock-multiline' properties on all 4 lines. 8. the actual refontification takes place, using font-lock-keywords. During this step, the rule (".*\\\\\n.*" (0 (progn (put-text-property (match-beginning 0) (match-end 0) 'font-lock-multiline t) nil))) is used to re-add a font-lock-multiline property to the first 3 lines (the 4th is not part of the multiline-line any more). If we had remoed the \ on line 2 instead, the refontification would have added the font-lock-multiline to lines 1&2 and then 2&3 (the two separated by a \n which is does not have this property). See, it does analyse the new text to adjust the font-lock-multiline property. > 1. I think font-lock-extend-region-function, called as it is from > j-l-after-change, is a good thing. It is a correct way of giving a major > mode an opportunity to massage a fontification region. I think it should > stay. You think it should be removed. Yup. > 2. f-l-d-f-r / j-l-f-now absolutely need a > "font-lock-extend-chunk-function" functionality (regardless of its > precise name). We agree on this. Yup. > 3. You have convinced me that it is possible to use the f-l-m properties. Good. > 4. I think that f-l-extend-region-f, called from f-l-after-change, is > likely to be more convenient and less error prone than the direct > manipulation of the f-l-multiline property by major mode code. You mean adding a single put-text-property call in your font-lock-keywords rule is more error prone than cooking up a b-c-f plus an a-c-f plus some way to store info between the two so that the a-c-f can figure out what happened, ...? > 5. I suspect you think that f-l-extend-region-f is likely to consume more > processor time than direct manipulation of the f-l-m property. The problemis not what is done but when. after-change-fucntions get run more often than font-lock-fontify-region. > Is there any reason why we can't leave both methods side by side in > the code? Unnecessary complexity. And it will mislead people into thinking that using a a-c-f is a good way to solve their problem. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-21 22:28 ` Stefan Monnier @ 2006-04-24 19:28 ` Alan Mackenzie 2006-04-24 21:06 ` Stefan Monnier ` (4 more replies) 0 siblings, 5 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-24 19:28 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan. On Fri, 21 Apr 2006, Stefan Monnier wrote: [ .... ] >> What is your objection to f-l-extend-region-f in a-c-f? Is it because it >> might gobble too much processor time? >That and the fact that it's not necessary, and that to use it (as seen >in your example) you need a good bit more work/code than doing it "the >other way" (with code solely run from font-lock-default-fontify-region). It will need an order of magnitude less work for f-l-extend-region-f than for f-l-multiline properties. At least, it will for those hackers who aren't totally familiar with the internals of font-lock-keywords, etc. f-l-e-r-f only needs to be applied at the extremes of the region. f-l-m needs to be applied throughout the entire region. I doubt very much whether f-l-m would use less processing power than f-l-e-r-f. Also, f-l-extend-region-function (called from f-l-a-c-f/j-l-a-c) will work EVERY time, because of its directness and simplicity. Using only f-l-m will fail in some circumstances. In the most general situation, a regexp in f-l-keywords will be inadequate - a defun will be needed. This will be at least as complicated as the one for f-l-extend-region-f. Writing a f-l-extend-region-f does not require any detailed knowledge of font-lock-keywords - thus the two problems of determining the region to fontify and of specifying how to fontify it are kept separate, hence simplified. Abusing f-l-keywords to set exotic text properties (as against its primary purpose, to set faces) requires contorted thinking. To write a f-l-e-r-f, one need only read a single page of the elisp manual. It's me that's going to have to implement this mechanism for CC Mode (all modes other than AWK Mode). It's going to be difficult enough as it is, without the added complications of the difficult to understand structure font-lock-keywords. >> It needs to be in j-l-fontify-now, so that that function knows what >> bytes to apply the 'fontified property to. >This is not needed for correctness. It's only a matter of avoiding >redundant work. This need wouldn't be new (it has existed since >Emacs-21 becaue of font-lock-multiline) and doesn't need to be addressed >right away. It might cause errors, and it will certainly increase run time. It isn't much effort to be correct, here. Why not do it right? (Yes, I'm prepared to do this patch, too.) [ .... ] >> I can't see how a major mode can manipulate f-l-multiline properties >> anywhere BUT an a-c-f (or a before-c-f). >What about the sample solution I provided: it's all manipulated from >font-lock-keywords (i.e. from font-lock-default-fontify-region)? It won't work in every case. An after-change function (at the very least) will need to be added to your solution to make it robust. >>> Look ma! No old-len! Tadaaa! >> .... but some code, somewhere, sometime needs to adjust the f-l-m >> property, removing it from L4. In general, it will also need to >> _split_ the text property into L1,2,3 and L4,5,... Otherwise, a f-l-m >> region could gradually engulf an entire buffer, making a nonsense of >> jit-lock. (OK, this would be an extreme situation. :-) >Yes, of course. It's done this way: font-lock-unfontify-region (called >from font-lock-default-fontify-region) removes the font-lock-mutliline >property on the whole fontified region just before proceeding with the >actual fontification, so the font-lock-keywords need to re-add it at >those places where it's still needed. OK. [ .... ] >> Something, somewhere, sometime has to analyse the newly inserted code, >> putting f-l-m properties throughout it, I think, and adjusting those >> at the boundaries of the new region. Is this to be done at the time >> of the change or at the time it gets font locked? I think it has to >> be done at the change, otherwise the f-l-m settings will be wrong when >> it comes time to fontify a chunk. >Let me extend my example. In the last episode, we stopped just before >refontification was to take place and had seen that the f-l-m property was >going to make the refontification apply to all 4 lines, as needed. >The fontification will work as follows: >5. font-lock-default-fontify-region gets asked to refontify line 3. >6. it extends the refontified region at beg and at end because of the > presence of f-l-m property. So the refontified region now spans all > 4 lines. >7. Prior to applying the new fontification, font-lock-unfontify-region is > called which removes all `face' properties and `font-lock-multiline' > properties on all 4 lines. >8. the actual refontification takes place, using font-lock-keywords. > During this step, the rule > (".*\\\\\n.*" > (0 (progn (put-text-property (match-beginning 0) (match-end 0) > 'font-lock-multiline t) > nil))) > is used to re-add a font-lock-multiline property to the first 3 lines > (the 4th is not part of the multiline-line any more). > If we had remoed the \ on line 2 instead, the refontification would have > added the font-lock-multiline to lines 1&2 and then 2&3 (the two > separated by a \n which is does not have this property). >See, it does analyse the new text to adjust the font-lock-multiline property. It sounds good in theory, but hasn't yet been tried in practice, AFAIK. Or has it? For which major mode? I really don't want to have to be the first person to try your approach. I don't like it and don't think it will work very well. >> 1. I think font-lock-extend-region-function, called as it is from >> j-l-after-change, is a good thing. It is a correct way of giving a >> major mode an opportunity to massage a fontification region. I think >> it should stay. You think it should be removed. >Yup. >> 2. f-l-d-f-r / j-l-f-now absolutely need a >> "font-lock-extend-chunk-function" functionality (regardless of its >> precise name). We agree on this. >Yup. >> 3. You have convinced me that it is possible to use the f-l-m >> properties. >Good. >> 4. I think that f-l-extend-region-f, called from f-l-after-change, is >> likely to be more convenient and less error prone than the direct >> manipulation of the f-l-multiline property by major mode code. >You mean adding a single put-text-property call in your >font-lock-keywords rule is more error prone than cooking up a b-c-f plus >an a-c-f plus some way to store info between the two so that the a-c-f >can figure out what happened, ...? Yes, absolutely. There are so many special cases, font-lock-keywords has a complicated structure, it would be very easy to put the property on ".*\\\\$" rather than ".*\\\\\n", for example. >> 5. I suspect you think that f-l-extend-region-f is likely to consume >> more processor time than direct manipulation of the f-l-m property. >The problem is not what is done but when. after-change-functions get run >more often than font-lock-fontify-region. No it doesn't. font-lock-fontify-region gets run at every change (unless jit-lock-defer-time is non-nil - it's nil by default). f-l-f-r gets run additionally for background fontification. >> Is there any reason why we can't leave both methods side by side in >> the code? >Unnecessary complexity. And it will mislead people into thinking that >using a a-c-f is a good way to solve their problem. after-change-functions is the canonical way of solving problems related to buffer changes. This includes Font Locking. I say to you again - your solution is not robust. I don't think it's been tried at all (correct me if I'm wrong). A variant form of f-l-extend-region-f (the advice on the f-l a-c-functions, as implemented for AWK Mode) has been in use since June 2003, and so far no bugs related to it have been reported by users. We are both aware of a refinement which is needed, namely calling some sort of f-l-extend-region-f from f-l-d-f-r and j-l-f-n. So - please leave the better tested mechanism in place. > Stefan -- Alan. P.S. We're boring everybody else as well as ourselves. :-( ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 19:28 ` Alan Mackenzie @ 2006-04-24 21:06 ` Stefan Monnier 2006-04-25 6:21 ` Ralf Angeli 2006-04-25 11:33 ` Alan Mackenzie 2006-04-24 21:20 ` Stefan Monnier ` (3 subsequent siblings) 4 siblings, 2 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-24 21:06 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >>> What is your objection to f-l-extend-region-f in a-c-f? Is it because it >>> might gobble too much processor time? >> That and the fact that it's not necessary, and that to use it (as seen >> in your example) you need a good bit more work/code than doing it "the >> other way" (with code solely run from font-lock-default-fontify-region). > It will need an order of magnitude less work for f-l-extend-region-f than > for f-l-multiline properties. Are you talking about CPU-work or programmer work? I was talking about programmer work, and I've shown you the code for font-lock-multiline: it's quite a bit shorter and more straightforward than your hack using b-c-f and a-c-f: just match the multiline element, place a font-lock-multiline property on it, and you're set. > At least, it will for those hackers who aren't totally familiar with the > internals of font-lock-keywords, etc. You don't have to be familiar with the internals. At least not more than necessary to write the proper foo-before-change-function plus foo-font-lock-lock-extend-region-after-change. > f-l-e-r-f only needs to be applied at the extremes of the region. f-l-m > needs to be applied throughout the entire region. I doubt very much > whether f-l-m would use less processing power than f-l-e-r-f. As I said, the important aspect is not how much time it takes each time it's executed, but how many times it's executed. In an after-change-function, it risks being executed enough times to beomce visible to the user. > Also, f-l-extend-region-function (called from f-l-a-c-f/j-l-a-c) will work > EVERY time, because of its directness and simplicity. Reality check? Here's your code: (defvar c-awk-old-EOLL 0) (make-variable-buffer-local 'c-awk-old-EOLL) ;; End of logical line following the region which is about to be changed. ;; Set in c-awk-before-change and used in c-awk-font-lock-extend-region. (defun c-awk-before-change (beg end) ;; This function is called exclusively from the before-change-functions hook. ;; It does two things: Finds the end of the (logical) line on which END lies, ;; and clears c-awk-NL-prop text properties from this point onwards. (save-restriction (save-excursion (setq c-awk-old-EOLL (c-awk-end-of-logical-line end)) (c-save-buffer-state nil (c-awk-clear-NL-props end (point-max)))))) (add-hook 'before-change-functions c-awk-before-change nil t) (defun c-awk-end-of-change-region (beg end old-len) ;; Find the end of the region which needs to be font-locked after a change. ;; This is the end of the logical line on which the change happened, either ;; as it was before the change, or as it is now, whichever is later. ;; N.B. point is left undefined. (max (+ (- c-awk-old-EOLL old-len) (- end beg)) (c-awk-end-of-logical-line end))) Here's my alternative implementation: (defconst c-awk-font-lock-keywords '(... (".*\\\\\n.*" (0 (progn (put-text-property (match-beginning 0) (match-end 0) 'font-lock-multiline t) nil))) ...)) Now which one's more direct and simple, really? > Using only f-l-m will fail in some circumstances. Please backup such claims. > In the most general situation, a regexp in f-l-keywords will be > inadequate - a defun will be needed. This will be at least as > complicated as the one for f-l-extend-region-f. So you mean in the most general case, it'll be no worse than your code. Here, we agree ;-) > Writing a f-l-extend-region-f does not require any detailed knowledge of > font-lock-keywords - thus the two problems of determining the region to > fontify and of specifying how to fontify it are kept separate, hence > simplified. Abusing f-l-keywords to set exotic text properties (as > against its primary purpose, to set faces) requires contorted thinking. > To write a f-l-e-r-f, one need only read a single page of the elisp > manual. And then one needs to think hard about how to actually write the code, what are the relevant odd cases one has to worry about, where to store the info you need from b-c-f to a-c-f, ... so much fun, indeed. Before/after-change-functions are great hammers, but you know what they say about hammers. > It's me that's going to have to implement this mechanism for CC Mode (all > modes other than AWK Mode). It's going to be difficult enough as it is, > without the added complications of the difficult to understand the structure > font-lock-keywords. Just take a deep breath, a couple of steps back, and try to look at the example use of font-lock-multiline above as if it were very simple. >>> It needs to be in j-l-fontify-now, so that that function knows what >>> bytes to apply the 'fontified property to. >> This is not needed for correctness. It's only a matter of avoiding >> redundant work. This need wouldn't be new (it has existed since >> Emacs-21 becaue of font-lock-multiline) and doesn't need to be addressed >> right away. > It might cause errors, Again, please backup such claims. > and it will certainly increase run time. In some cases it will, yes. Those cases tend to suffer from other performance problems anyway, tho (i.e. they typically involve large regions that need to be rehighlighted atomically, so even if you fix this jit-lock-fontify-now performance misfeature the underlying performance issue will still bite you and you'll end up having to use a different solution for which the jit-lock-fontify-now problem won't apply either). At least this was my experience with smerge-mode and it didn't seem particularly specific to my situation. > It isn't much effort to be correct, here. Why not do it right? (Yes, I'm > prepared to do this patch, too.) It's enough changes that I don't think it should go into Emacs-22. I prefer keeping the same performance misfeature that we already had in Emacs-21 and release Emacs-22 a few days earlier. But if you want to fix it, here's how a good fix should work IMNSHO: - every function that get added to jit-lock-functions (such as font-lock-fontify-region) should be changed to return the region it actually jit'd. - jit-lock-fontify-now should then take the intersection of the returned regions to determine where to set the `fontified' property. >> What about the sample solution I provided: it's all manipulated from >> font-lock-keywords (i.e. from font-lock-default-fontify-region)? > It won't work in every case. An after-change function (at the very > least) will need to be added to your solution to make it robust. Again, I see no evidence of this. Back up your claim. > Yes, absolutely. There are so many special cases, font-lock-keywords has > a complicated structure, it would be very easy to put the property on > ".*\\\\$" rather than ".*\\\\\n", for example. The only really important part where the font-lock-multiline property should be added is the text that covers all the "backslash newlines" of a multiline line. Whether it includes the last ".*$" or the leading "^.*" doesn't really matter. It's not nearly as brittle as you think. >>> 5. I suspect you think that f-l-extend-region-f is likely to consume >>> more processor time than direct manipulation of the f-l-m property. >> The problem is not what is done but when. after-change-functions get run >> more often than font-lock-fontify-region. > No it doesn't. font-lock-fontify-region gets run at every change (unless > jit-lock-defer-time is non-nil - it's nil by default). No, it's run at every redisplay. For self-insert-command, it's usually the same as "every change", but for many other commands it can be quite different (or even for self-insert-command, if you trigger an abbrev expansion or if you trigger auto-fill, ...). "at every redisplay" basically means "at human pace", whereas "after every change" means "at CPU pace". > I say to you again - your solution is not robust. I don't think it's > been tried at all (correct me if I'm wrong). A variant form of > f-l-extend-region-f (the advice on the f-l a-c-functions, as implemented > for AWK Mode) has been in use since June 2003, and so far no bugs related > to it have been reported by users. Without using a hook in font-lock-default-fontify-region you can't claim it's robust: I can easily cook up a case where it fails. > We are both aware of a refinement which is needed, namely calling some > sort of f-l-extend-region-f from f-l-d-f-r. It's not a refinement. It's a requirement for correctness. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 21:06 ` Stefan Monnier @ 2006-04-25 6:21 ` Ralf Angeli 2006-04-25 10:53 ` Alan Mackenzie ` (2 more replies) 2006-04-25 11:33 ` Alan Mackenzie 1 sibling, 3 replies; 80+ messages in thread From: Ralf Angeli @ 2006-04-25 6:21 UTC (permalink / raw) * Stefan Monnier (2006-04-24) writes: > I was talking about programmer work, and I've shown you the code for > font-lock-multiline: it's quite a bit shorter and more straightforward than > your hack using b-c-f and a-c-f: just match the multiline element, place > a font-lock-multiline property on it, and you're set. Do you even have to bother with placing font-lock-multiline properties manually? In AUCTeX we simply set the font-lock-multiline variable to t and font-lock added the property automatically. -- Ralf ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 6:21 ` Ralf Angeli @ 2006-04-25 10:53 ` Alan Mackenzie 2006-04-25 18:37 ` Ralf Angeli ` (2 more replies) 2006-04-25 19:23 ` Last steps for pretesting (font-lock-extend-region-function) Stefan Monnier 2006-04-25 20:18 ` Stefan Monnier 2 siblings, 3 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 10:53 UTC (permalink / raw) Cc: emacs-devel Hi, Ralf! On Tue, 25 Apr 2006, Ralf Angeli wrote: >* Stefan Monnier (2006-04-24) writes: >> I was talking about programmer work, and I've shown you the code for >> font-lock-multiline: it's quite a bit shorter and more straightforward than >> your hack using b-c-f and a-c-f: just match the multiline element, place >> a font-lock-multiline property on it, and you're set. >Do you even have to bother with placing font-lock-multiline properties >manually? In AUCTeX we simply set the font-lock-multiline variable to >t and font-lock added the property automatically. Yes, absolutely! There are two separate things here which Stefan is failing to distinguish: (i) The matching of keywords which span line breaks; (ii) Extending a font-lock region to include all text needing refontification. If I've understood font-lock-multiline properly, only manually setting f-l-m can achieve purpose (ii). >Ralf -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 10:53 ` Alan Mackenzie @ 2006-04-25 18:37 ` Ralf Angeli 2006-04-25 22:00 ` Alan Mackenzie 2006-04-25 19:26 ` Stefan Monnier 2006-04-25 20:10 ` font-lock-multiline for cc-awk Stefan Monnier 2 siblings, 1 reply; 80+ messages in thread From: Ralf Angeli @ 2006-04-25 18:37 UTC (permalink / raw) * Alan Mackenzie (2006-04-25) writes: > On Tue, 25 Apr 2006, Ralf Angeli wrote: > >>Do you even have to bother with placing font-lock-multiline properties >>manually? In AUCTeX we simply set the font-lock-multiline variable to >>t and font-lock added the property automatically. > > Yes, absolutely! There are two separate things here which Stefan is > failing to distinguish: > > (i) The matching of keywords which span line breaks; > > (ii) Extending a font-lock region to include all text needing > refontification. > > If I've understood font-lock-multiline properly, only manually setting > f-l-m can achieve purpose (ii). As far as I can see font-lock cares about that itself in case of keyword fontification if the value of the variable `font-lock-multiline' is t. There are `(put-text-property ... 'font-lock-multiline t)' calls in `font-lock-fontify-achored-keywords' and `font-lock-fontify-keywords-region'. As I wrote before, AUCTeX uses this feature without manually adding `font-lock-multiline' properties. -- Ralf ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 18:37 ` Ralf Angeli @ 2006-04-25 22:00 ` Alan Mackenzie 2006-04-26 6:14 ` Ralf Angeli 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 22:00 UTC (permalink / raw) Cc: emacs-devel Hi, Ralf! On Tue, 25 Apr 2006, Ralf Angeli wrote: >* Alan Mackenzie (2006-04-25) writes: >> On Tue, 25 Apr 2006, Ralf Angeli wrote: >>>Do you even have to bother with placing font-lock-multiline properties >>>manually? In AUCTeX we simply set the font-lock-multiline variable to >>>t and font-lock added the property automatically. >> Yes, absolutely! There are two separate things here which Stefan is >> failing to distinguish: >> (i) The matching of keywords which span line breaks; >> (ii) Extending a font-lock region to include all text needing >> refontification. >> If I've understood font-lock-multiline properly, only manually setting >> f-l-m can achieve purpose (ii). >As far as I can see font-lock cares about that itself in case of >keyword fontification if the value of the variable >`font-lock-multiline' is t. There are `(put-text-property >... 'font-lock-multiline t)' calls in >`font-lock-fontify-achored-keywords' and >`font-lock-fontify-keywords-region'. Yes, but as far as I can see, these function are capable of putting the text property only on bytes between BEG and END. It cannot modify BEG by applying the property to bytes before BEG. >As I wrote before, AUCTeX uses this feature without manually adding >`font-lock-multiline' properties. >Ralf -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 22:00 ` Alan Mackenzie @ 2006-04-26 6:14 ` Ralf Angeli 0 siblings, 0 replies; 80+ messages in thread From: Ralf Angeli @ 2006-04-26 6:14 UTC (permalink / raw) * Alan Mackenzie (2006-04-26) writes: > On Tue, 25 Apr 2006, Ralf Angeli wrote: > >>As far as I can see font-lock cares about that itself in case of >>keyword fontification if the value of the variable >>`font-lock-multiline' is t. There are `(put-text-property >>... 'font-lock-multiline t)' calls in >>`font-lock-fontify-achored-keywords' and >>`font-lock-fontify-keywords-region'. > > Yes, but as far as I can see, these function are capable of putting the > text property only on bytes between BEG and END. It cannot modify BEG by > applying the property to bytes before BEG. Sure, but that was not was I was after anyway. I just wanted to mention a way for the f-l-m property to be applied automatically. As this only works for keyword fontification and not for syntactic keyword fontification (which your code is using at some point as Stefan pointed out) this point is moot now, however. -- Ralf ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 10:53 ` Alan Mackenzie 2006-04-25 18:37 ` Ralf Angeli @ 2006-04-25 19:26 ` Stefan Monnier 2006-04-25 20:10 ` font-lock-multiline for cc-awk Stefan Monnier 2 siblings, 0 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 19:26 UTC (permalink / raw) Cc: Ralf Angeli, emacs-devel >>> I was talking about programmer work, and I've shown you the code for >>> font-lock-multiline: it's quite a bit shorter and more straightforward than >>> your hack using b-c-f and a-c-f: just match the multiline element, place >>> a font-lock-multiline property on it, and you're set. >> Do you even have to bother with placing font-lock-multiline properties >> manually? In AUCTeX we simply set the font-lock-multiline variable to >> t and font-lock added the property automatically. > Yes, absolutely! There are two separate things here which Stefan is > failing to distinguish: > (i) The matching of keywords which span line breaks; This can only reliably be handled by a hook in font-lock-default-fontify-region. > (ii) Extending a font-lock region to include all text needing > refontification. > If I've understood font-lock-multiline properly, only manually setting > f-l-m can achieve purpose (ii). No, setting the font-lock-multiline variable will (if all goes well) take care of (ii), but won't reliably take care of (i). Setting the font-lock-multiline property by hand will take care of (ii) but not of (i). Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* font-lock-multiline for cc-awk. 2006-04-25 10:53 ` Alan Mackenzie 2006-04-25 18:37 ` Ralf Angeli 2006-04-25 19:26 ` Stefan Monnier @ 2006-04-25 20:10 ` Stefan Monnier 2006-04-26 7:37 ` Alan Mackenzie 2 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 20:10 UTC (permalink / raw) Cc: emacs-devel AFAICT the patch below converts cc-awk to use font-lock-multiline instead of an after-change-function. Stefan --- cc-awk.el 06 Mar 2006 14:26:48 -0500 1.9 +++ cc-awk.el 25 Apr 2006 16:08:37 -0400 @@ -677,7 +677,13 @@ (goto-char (1+ beg)) (or (eobp) (while (search-forward "\"" end t) - (c-put-char-property (1- (point)) 'syntax-table '(1)))))) + (c-put-char-property (1- (point)) 'syntax-table '(1))))) + ;; Make sure that this element will be properly refontified next time + ;; around even if it spans several lines. + (save-excursion + (goto-char beg) + (if (search-forward "\n" end t) + (put-text-property beg end 'font-lock-multiline t)))) (defun c-awk-syntax-tablify-string () ;; Point is at the opening " or _" of a string. Set the syntax-table @@ -759,10 +765,12 @@ ;; (iv) Inside a comment, all syntax-table properties are cleared. ;; ;; This function does hidden buffer changes. - (let (anchor - (anchor-state-/div nil)) ; t means a following / would be a div sign. (c-awk-beginning-of-logical-line) ; ACM 2002/7/21. This is probably redundant. + ;; Redundant if called from font-lock. --Stef (c-clear-char-properties (point) lim 'syntax-table) + (let (anchor + (anchor-state-/div nil) + (linebeg (point))) ; t means a following / would be a div sign. ;; Once round the next loop for each string, regexp, or div sign (while (progn ;; Skip any "harmless" lines before the next tricky one. @@ -776,83 +784,19 @@ (setq anchor-state-/div (if (looking-at "_?\"") (c-awk-syntax-tablify-string) + (if (save-excursion (re-search-backward "\n" linebeg t)) + ;; Whether / is interpreted as a div or not depends on all + ;; the text between linebeg and point, so if something is + ;; changed there, even if it's on another line, we need to + ;; refontify this piece of text. + ;; Note that jit-lock-contextually also takes care of + ;; that, so this is only necessary if we want it to work + ;; without jit-lock-contextually or if we want to avoid + ;; the jit-lock-context-time delay. + (put-text-property linebeg (point) 'font-lock-multiline t)) (c-awk-syntax-tablify-/ anchor anchor-state-/div)))) nil)) - -;; ACM, 2002/07/21: Thoughts: We need an AWK Mode after-change function to set -;; the syntax-table properties even when font-lock isn't enabled, for the -;; subsequent use of movement functions, etc. However, it seems that if font -;; lock _is_ enabled, we can always leave it to do the job. -(defvar c-awk-old-EOLL 0) -(make-variable-buffer-local 'c-awk-old-EOLL) -;; End of logical line following the region which is about to be changed. Set -;; in c-awk-before-change and used in c-awk-after-change. - -(defun c-awk-before-change (beg end) -;; This function is called exclusively from the before-change-functions hook. -;; It does two things: Finds the end of the (logical) line on which END lies, -;; and clears c-awk-NL-prop text properties from this point onwards. -;; -;; This function might do hidden buffer changes. - (save-restriction - (save-excursion - (setq c-awk-old-EOLL (c-awk-end-of-logical-line end)) - (c-save-buffer-state nil - (c-awk-clear-NL-props end (point-max)))))) - -(defun c-awk-end-of-change-region (beg end old-len) - ;; Find the end of the region which needs to be font-locked after a change. - ;; This is the end of the logical line on which the change happened, either - ;; as it was before the change, or as it is now, whichever is later. - ;; N.B. point is left undefined. - ;; - ;; This function might do hidden buffer changes. - (max (+ (- c-awk-old-EOLL old-len) (- end beg)) - (c-awk-end-of-logical-line end))) - -(defun c-awk-after-change (beg end old-len) -;; This function is called exclusively as an after-change function in -;; AWK Mode. It ensures that the syntax-table properties get set in the -;; changed region. However, if font-lock is enabled, this function does -;; nothing, since an enabled font-lock after-change function will always do -;; this. -;; -;; This function might do hidden buffer changes. - (unless (and (boundp 'font-lock-mode) font-lock-mode) - (save-restriction - (save-excursion - (save-match-data - (setq end (c-awk-end-of-change-region beg end old-len)) - (c-awk-beginning-of-logical-line beg) - (c-save-buffer-state nil ; So that read-only status isn't affected. - ; (e.g. when first loading the buffer) - (c-awk-set-syntax-table-properties end))))))) - -;; ACM 2002/5/25. When font-locking is invoked by a buffer change, the region -;; specified by the font-lock after-change function must be expanded to -;; include ALL of any string or regexp within the region. The simplest way to -;; do this in practice is to use the beginning/end-of-logical-line functions. -;; Don't overlook the possibility of the buffer change being the "recapturing" -;; of a previously escaped newline. -(defmacro c-awk-advise-fl-for-awk-region (function) - `(defadvice ,function (before get-awk-region activate) -;; When font-locking an AWK Mode buffer, make sure that any string/regexp is -;; completely font-locked. - (when (eq major-mode 'awk-mode) - (save-excursion - (ad-set-arg 1 (c-awk-end-of-change-region - (ad-get-arg 0) ; beg - (ad-get-arg 1) ; end - (ad-get-arg 2))) ; old-len - (ad-set-arg 0 (c-awk-beginning-of-logical-line (ad-get-arg 0))))))) - -(c-awk-advise-fl-for-awk-region font-lock-after-change-function) -(c-awk-advise-fl-for-awk-region jit-lock-after-change) -(c-awk-advise-fl-for-awk-region lazy-lock-defer-rest-after-change) -(c-awk-advise-fl-for-awk-region lazy-lock-defer-line-after-change) - -\f ;; ACM 2002/9/29. Movement functions, e.g. for C-M-a and C-M-e ;; The following three regexps differ from those earlier on in cc-awk.el in @@ -1011,5 +955,5 @@ \f (cc-provide 'cc-awk) ; Changed from 'awk-mode, ACM 2002/5/21 -;;; arch-tag: c4836289-3aa4-4a59-9934-9ccc2bacccf3 +;; arch-tag: c4836289-3aa4-4a59-9934-9ccc2bacccf3 ;;; awk-mode.el ends here ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: font-lock-multiline for cc-awk. 2006-04-25 20:10 ` font-lock-multiline for cc-awk Stefan Monnier @ 2006-04-26 7:37 ` Alan Mackenzie 2006-04-26 13:55 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-26 7:37 UTC (permalink / raw) Cc: emacs-devel Hi, Stefan! On Tue, 25 Apr 2006, Stefan Monnier wrote: >AFAICT the patch below converts cc-awk to use font-lock-multiline >instead of an after-change-function. You've rubbed out my after-change function! Heeeellllllllllpppppppp!! The after-change function is needed to set syntax-table text properties, since this must be done regardless of whether font lock is enabled. In fact, it's a massive pain in the posterior that font lock erases these text properties without getting permission from the major mode. This is the only reason that c-awk-set-syntax-table-properties is hooked into font-lock-syntactic keywords - essentially, the syntax-table properties are an after-change thing. I don't want to mix up the syntax-table and f-l-multiline properties, because they're very different things. Doing so would weaken the existing functions, which are strong single-purpose functions. To do things this sort of way, I'd put in a new function c-awk-set-fl-multiline-properties (a sister function to c-awk-set-syntax-table-properties, with the same calling conventions). The existing function loops round string/regexp/division sign constructs, whereas the new one would be looping round lines. They don't mix very happily. But mainly, I don't want to be applying f-l-m properties throughout the region[*]. They should be applied only at the region boundaries when these need extending, and only when font lock is enabled, since they serve no purpose otherwise. [*] As you're aware, I don't want to use f-l-m properties at all. But if I must use them, I only want them where absolutely needed. And you've rubbed out my before-change function! Ruuuuuunnnnnnnn awwwwwaaaayyyyyyy!!!!!!! This is needed for other things besides calculating font lock boundaries. Why not leave things alone until Richard has settled the whole matter? > Stefan [ Patch tentatively applied and examined. ] -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: font-lock-multiline for cc-awk. 2006-04-26 7:37 ` Alan Mackenzie @ 2006-04-26 13:55 ` Stefan Monnier 0 siblings, 0 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-26 13:55 UTC (permalink / raw) Cc: emacs-devel >> AFAICT the patch below converts cc-awk to use font-lock-multiline >> instead of an after-change-function. > You've rubbed out my after-change function! Heeeellllllllllpppppppp!! Well, it was just a sample patch. > In fact, it's a massive pain in the posterior that font lock erases these > text properties without getting permission from the major mode. Huh? font-lock only clears the syntax-table properties if you specifically ask it to manage that property (by setting font-lock-syntactic-keywords). > I don't want to mix up the syntax-table and f-l-multiline properties, > because they're very different things. On the contrary, they belong together: the font-lock-multiline accompagnies the syntax-table property to say "this syntax-table property depends on this chunk of text", so that if a part of this chunk of text is changed, the syntax-table property can be recomputed. > Doing so would weaken the > existing functions, which are strong single-purpose functions. To do > things this sort of way, I'd put in a new function > c-awk-set-fl-multiline-properties (a sister function to > c-awk-set-syntax-table-properties, with the same calling conventions). > The existing function loops round string/regexp/division sign constructs, > whereas the new one would be looping round lines. A multiline-line without strings or regexps doesn't need to be refontified atomically because none of the other patterns you use can span or depend on multiple lines of text. > They don't mix very happily. On the contrary: they belong together. > But mainly, I don't want to be applying f-l-m properties throughout the > region[*]. They should be applied only at the region boundaries when > these need extending, and only when font lock is enabled, since they serve > no purpose otherwise. > [*] As you're aware, I don't want to use f-l-m properties at all. But if > I must use them, I only want them where absolutely needed. Look, I'm just showing this code to try and get you to understand how font-lock-multiline properties work. It seems like you've been thinking much too hard about your a-c-f code and can't think outside of that setting. [ Reminds me of "a calculus of capabilities" and "alias types": they present a new way to guarantee the safety of "manual" memory management primitives. Contrary to previous appoaches (which tended to be very complex and incomplete or much too restrictive), they allow dangling pointers. Many people found it terribly difficult to believe to understand that a system that allows dangling pointers can be safe w.r.t manual memory management: all previous approaches cared about checking that "free" is safe, whereas those new approaches only care about checking that the memory access is safe: i.e. they simply disallow access after a "free" rather than disallow "free" if the object is still "live" (which they typically define as "reachable"). ] > Why not leave things alone until Richard has settled the whole matter? Of course, I didn't intend to apply the patch. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 6:21 ` Ralf Angeli 2006-04-25 10:53 ` Alan Mackenzie @ 2006-04-25 19:23 ` Stefan Monnier 2006-04-25 20:18 ` Stefan Monnier 2 siblings, 0 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 19:23 UTC (permalink / raw) Cc: emacs-devel >> I was talking about programmer work, and I've shown you the code for >> font-lock-multiline: it's quite a bit shorter and more straightforward >> than your hack using b-c-f and a-c-f: just match the multiline element, >> place a font-lock-multiline property on it, and you're set. > Do you even have to bother with placing font-lock-multiline properties > manually? In AUCTeX we simply set the font-lock-multiline variable to > t and font-lock added the property automatically. Of course, you can indeed just set the variable, but this has more impact on performance (e.g. every time it matches a keyword, font-lock will then check whether it spans several lines, even if the keyword pattern is something silly like "foo" which clearly can't span multiple lines), and will not always work correctly if the MATCHER was a function rather than a regexp. I suggested to do it manually because it is more explicit, it gives you more control, and most importantly I haven't seen the part of c-awk-font-lock-keywords which cares about multiple lines, so I haven't convinced myself that it would work for his example. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 6:21 ` Ralf Angeli 2006-04-25 10:53 ` Alan Mackenzie 2006-04-25 19:23 ` Last steps for pretesting (font-lock-extend-region-function) Stefan Monnier @ 2006-04-25 20:18 ` Stefan Monnier 2 siblings, 0 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 20:18 UTC (permalink / raw) Cc: emacs-devel >> I was talking about programmer work, and I've shown you the code for >> font-lock-multiline: it's quite a bit shorter and more straightforward than >> your hack using b-c-f and a-c-f: just match the multiline element, place >> a font-lock-multiline property on it, and you're set. > Do you even have to bother with placing font-lock-multiline properties > manually? In AUCTeX we simply set the font-lock-multiline variable to > t and font-lock added the property automatically. [ Now that I've actually looked at his code: ] Actually, yes he does because the font-lock-multiline variable is only applied to font-lock-keywords, whereas his multiline needs come from font-lock-syntactic-keywords, and also because he doesn't actually use font-lock-syntactic-keywords, he just puts there a single MATCHER which does the work by hand. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 21:06 ` Stefan Monnier 2006-04-25 6:21 ` Ralf Angeli @ 2006-04-25 11:33 ` Alan Mackenzie 2006-04-25 11:59 ` David Kastrup 2006-04-25 12:33 ` Stefan Monnier 1 sibling, 2 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 11:33 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan! On Mon, 24 Apr 2006, Stefan Monnier wrote: >>>> What is your objection to f-l-extend-region-f in a-c-f? Is it because it >>>> might gobble too much processor time? >>> That and the fact that it's not necessary, and that to use it (as seen >>> in your example) you need a good bit more work/code than doing it "the >>> other way" (with code solely run from font-lock-default-fontify-region). >> It will need an order of magnitude less work for f-l-extend-region-f than >> for f-l-multiline properties. >Are you talking about CPU-work or programmer work? Programmer work. I can state that with 100% certainty when that programmer is me. ;-( >I was talking about programmer work, and I've shown you the code for >font-lock-multiline: it's quite a bit shorter and more straightforward >than your hack using b-c-f and a-c-f: just match the multiline element, >place a font-lock-multiline property on it, and you're set. "just", you write. ;-) The complexity of font-lock-keywords is an order of magnitude higher than that of a defun, or a short sequence of related defuns. There's the complexity of where to quote, where to omit quotes, wondering whether to put the "region" regexps before or after the face setting regexps, whether to make f-l-m start-sticky or stop-sticky.... I find the elisp manual page which describes font-lock-keywords ("Search-based Fontification") all but inpenetrable. It's reduced me to tears on several occasions. I've just spent five minutes looking through it, trying to find the description of the form you suggested to me: (".*\\\\\n.*" (0 (progn (put-text-property (match-beginning 0) (match-end 0) 'font-lock-multiline t) nil))) . I didn't find it. Consider the difficulty of testing. With the approach I've used, it's a simple matter of C-u C-M-x, and edebugging through the function with M-:. With your f-l-m approach, you've got to delve into the bowels of font-lock, find the appropriate function (far from trivial: it's f-l-default-f-r, I think), instrument it, f and blind as recursive levels of jit lock build up on your screen, turn off jit-lock (a far from trivial operation when you've not done it before - it's not documented), start the whole process again. >> At least, it will for those hackers who aren't totally familiar with >> the internals of font-lock-keywords, etc. >You don't have to be familiar with the internals. At least not more >than necessary to write the proper foo-before-change-function plus >foo-font-lock-lock-extend-region-after-change. With respect, Stefan, that's rubbish. You need to know where and how to write the above Lisp form (the one starting (".*\\\\n.*" ...)) into f-l-keywords. For me, that's perhaps half an hour to an hour of frustration trying to find the right place in "Search-based Fontification". >> f-l-e-r-f only needs to be applied at the extremes of the region. >> f-l-m needs to be applied throughout the entire region. I doubt very >> much whether f-l-m would use less processing power than f-l-e-r-f. >As I said, the important aspect is not how much time it takes each time >it's executed, but how many times it's executed. In an >after-change-function, it risks being executed enough times to become >visible to the user. [Hey, "beomce" was a difficult typo to correct ;-] What about the run-time overhead of having an extra text property over the entire buffer? The f-l-e-r-f, as you envisage it (being called only from f-l-default-fontify-r) will also be called at virtually every buffer change, since almost every change causes a redisplay. >> Also, f-l-extend-region-function (called from f-l-a-c-f/j-l-a-c) will >> work EVERY time, because of its directness and simplicity. >Reality check? Here's your code: > (defvar c-awk-old-EOLL 0) > (make-variable-buffer-local 'c-awk-old-EOLL) > ;; End of logical line following the region which is about to be changed. > ;; Set in c-awk-before-change and used in c-awk-font-lock-extend-region. > (defun c-awk-before-change (beg end) > ;; This function is called exclusively from the before-change-functions hook. > ;; It does two things: Finds the end of the (logical) line on which END lies, > ;; and clears c-awk-NL-prop text properties from this point onwards. > (save-restriction > (save-excursion > (setq c-awk-old-EOLL (c-awk-end-of-logical-line end)) > (c-save-buffer-state nil > (c-awk-clear-NL-props end (point-max)))))) > (add-hook 'before-change-functions c-awk-before-change nil t) > (defun c-awk-end-of-change-region (beg end old-len) > ;; Find the end of the region which needs to be font-locked after a change. > ;; This is the end of the logical line on which the change happened, either > ;; as it was before the change, or as it is now, whichever is later. > ;; N.B. point is left undefined. > (max (+ (- c-awk-old-EOLL old-len) (- end beg)) > (c-awk-end-of-logical-line end))) >Here's my alternative implementation: > (defconst c-awk-font-lock-keywords > '(... > (".*\\\\\n.*" > (0 (progn (put-text-property (match-beginning 0) (match-end 0) > 'font-lock-multiline t) > nil))) > ...)) >Now which one's more direct and simple, really? The one which is adequately commented. The one which doesn't use an obscure list structure that you need to look up in a manual. The one with thin, obvious interfaces with the rest of Emacs. The one which is trivial to debug with edebug. The one which doesn't conflate determining the font-lock region with actually fontifying it. The one which runs faster on a large region. [ .... ] >> It's me that's going to have to implement this mechanism for CC Mode >> (all modes other than AWK Mode). It's going to be difficult enough as >> it is, without the added complications of the difficult to understand >> the structure font-lock-keywords. >Just take a deep breath, a couple of steps back, and try to look at the >example use of font-lock-multiline above as if it were very simple. I've tried, but I just don't have the intellectual capability to cope with such complexity without the greatest of difficulty. You obviously do. Complicated nested structures make my eyes glaze over. There's a good chance I'm not the only Emacs hacker who has this sort of trouble. There's an assymetry between our positions: by threatening to remove the f-l-e-r-f hook (after-change version), you're trying to force me to use the f-l-m mechanism. You're trying to impose your taste and judgement on me, and I resent this quite a lot. I think the f-l-m text property is a revolting kludge, but I'm not trying to do away with it. I'm happy enough for other hackers to use it, though I'd rather not have to maintain their code afterwards. Is there any chance we could bring this discussion to an end, now? The things we've been talking about are important, but this discussion has taken up an enormous amount of my time, and probably yours too. We've both got other urgent things to do. [ .... ] >> It isn't much effort to be correct, here. Why not do it right? (Yes, >> I'm prepared to do this patch, too.) >It's enough changes that I don't think it should go into Emacs-22. I >prefer keeping the same performance misfeature that we already had in >Emacs-21 and release Emacs-22 a few days earlier. >But if you want to fix it, here's how a good fix should work IMNSHO: >- every function that get added to jit-lock-functions (such as > font-lock-fontify-region) should be changed to return the region > it actually jit'd. >- jit-lock-fontify-now should then take the intersection of the returned > regions to determine where to set the `fontified' property. My fix is simpler: to recognise that f-l-default-fontify-region performs two disparate tasks: (i) Extending the region to be fontified; (ii) Fontifying this region. By extracting (ii) into a separate function, that function could be called directly from jit-lock-fontify-now, bypassing the redundant second massaging of the region. [ .... ] >> We are both aware of a refinement which is needed, namely calling some >> sort of f-l-extend-region-f from f-l-d-f-r. >It's not a refinement. It's a requirement for correctness. It is, and we need it. It needs a name - here are the possibilities: (i) It uses the same hook, font-lock-extend-region-function, as f-l-after-change-functions, setting the third parameter, OLD-LEN, to nil; (ii) It uses a distinct hook with a new name, say, font-lock-extend-chunk-function; (iii) It usurps the name f-l-e-r-f from its current use, abolishing the region extension mechanism in the f-l-after-change-f/j-l-after-change. I'd prefer (i), but would have no trouble with (ii). I absolutely do NOT want (iii). > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 11:33 ` Alan Mackenzie @ 2006-04-25 11:59 ` David Kastrup 2006-04-25 12:33 ` Stefan Monnier 1 sibling, 0 replies; 80+ messages in thread From: David Kastrup @ 2006-04-25 11:59 UTC (permalink / raw) Cc: emacs-devel, Stefan Monnier, Richard Stallman Alan Mackenzie <acm@muc.de> writes: > On Mon, 24 Apr 2006, Stefan Monnier wrote: [...] > I find the elisp manual page which describes font-lock-keywords > ("Search-based Fontification") all but inpenetrable. It's reduced me to > tears on several occasions. I've just spent five minutes looking through > it, trying to find the description of the form you suggested to me: > > (".*\\\\\n.*" > (0 (progn (put-text-property (match-beginning 0) (match-end 0) > 'font-lock-multiline t) > nil))) > > . I didn't find it. > [...] >>You don't have to be familiar with the internals. At least not more >>than necessary to write the proper foo-before-change-function plus >>foo-font-lock-lock-extend-region-after-change. > > With respect, Stefan, that's rubbish. You need to know where and how to > write the above Lisp form (the one starting (".*\\\\n.*" ...)) into > f-l-keywords. For me, that's perhaps half an hour to an hour of > frustration trying to find the right place in "Search-based > Fontification". [...] >>Here's my alternative implementation: > >> (defconst c-awk-font-lock-keywords >> '(... >> (".*\\\\\n.*" >> (0 (progn (put-text-property (match-beginning 0) (match-end 0) >> 'font-lock-multiline t) >> nil))) >> ...)) > >>Now which one's more direct and simple, really? > > The one which is adequately commented. Not relevant. > The one which doesn't use an obscure list structure that you need to > look up in a manual. The one with thin, obvious interfaces with the > rest of Emacs. It appears to me like you are uncomfortable around font lock expressions. And I doubt that anybody having worked with font lock entries and syntax table entries has not teared out his hairs in frustration several times round. But I don't think that the solution is trying to move everything somewhere else. It would rather be sensible to create appropriate accessor functions that save the programmer from having to deal with the internal list structures. We do similar things with the "event" data structure by having posn-* accessor functions. font-lock should be made more programmer-friendly instead of delegating stuff to various ad-hoc hooks. > I've tried, but I just don't have the intellectual capability to > cope with such complexity without the greatest of difficulty. You > obviously do. Complicated nested structures make my eyes glaze > over. There's a good chance I'm not the only Emacs hacker who has > this sort of trouble. Then we need to create functions or macros that build and access the data structures. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 11:33 ` Alan Mackenzie 2006-04-25 11:59 ` David Kastrup @ 2006-04-25 12:33 ` Stefan Monnier 2006-04-25 14:07 ` Alan Mackenzie 1 sibling, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 12:33 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel > I find the elisp manual page which describes font-lock-keywords > ("Search-based Fontification") all but inpenetrable. It's reduced me to > tears on several occasions. I've just spent five minutes looking through > it, trying to find the description of the form you suggested to me: > (".*\\\\\n.*" > (0 (progn (put-text-property (match-beginning 0) (match-end 0) > 'font-lock-multiline t) > nil))) > . I didn't find it. Then the problem isn't font-lock-multiline. Please don't mix up matters. This form is the standard form (except you haven't noticed yet that the FACENAME part of the font-lock-keywords is not a face symbol but an Elisp expression whose value after exavluation should be a face). It's: (MATCHER HIGHLIGHT ...) where MATCHER is ".*\\\\\n.*" and HIGHLIGHT is of the form MATCH-HIGHLIGHT which is of the form (SUBEXP FACENAME [OVERRIDE [LAXMATCH]]) where SUBEXP is 0 and FACENAME is (progn ...). > With respect, Stefan, that's rubbish. You need to know where and how to > write the above Lisp form (the one starting (".*\\\\n.*" ...)) into > f-l-keywords. For me, that's perhaps half an hour to an hour of > frustration trying to find the right place in "Search-based > Fontification". If you don't know how font-lock-keywords work, then learn that first before trying to work on the font-lock support for cc-mode. That'll save you a lot more time. > What about the run-time overhead of having an extra text property over > the entire buffer? It's not over the entire buffer. And I haven't seen any evidence that it's noticeable (especially compared the number of `face' properties or of the `fontified' property added to the whole buffer). > The f-l-e-r-f, as you envisage it (being called only from > f-l-default-fontify-r) will also be called at virtually every buffer > change, since almost every change causes a redisplay. AFAIK, the case where a single change takes place between redisplay is the case where there is no performance issue: it takes place at the user's pace anyway. The important case is when Emacs has more work to do in response to a user's action, i.e. there's more work to be done until the next redisplay. >> Just take a deep breath, a couple of steps back, and try to look at the >> example use of font-lock-multiline above as if it were very simple. > I've tried, but I just don't have the intellectual capability to cope > with such complexity without the greatest of difficulty. The whole point is that you shouldn't have to. It's like recursion (tho not nearly as elegant, I must admit): have faith, don't try to simulate it all in your head, and then it becomes simple. > You obviously do. Complicated nested structures make my eyes glaze over. > There's a good chance I'm not the only Emacs hacker who has this sort > of trouble. Any Emacs hacker that needs to write complex syntax highlighting rules will have to learn it anyway unless he prefers to circumvent font-lock and roll his own instead. Sounds like NIH to me. > There's an assymetry between our positions: by threatening to remove the > f-l-e-r-f hook (after-change version), you're trying to force me to use > the f-l-m mechanism. Yes. > You're trying to impose your taste and judgement on me, and I resent this > quite a lot. I'm as close as it gets to a font-lock maintainer. So from where I stand, you're trying to impose your taste and judgment on me. > I think the f-l-m text property is a revolting kludge, but > I'm not trying to do away with it. I'm happy enough for other hackers to > use it, though I'd rather not have to maintain their code afterwards. > Is there any chance we could bring this discussion to an end, now? The > things we've been talking about are important, but this discussion has > taken up an enormous amount of my time, and probably yours too. We've > both got other urgent things to do. 3 ways this can end: - Richard says that you're right. - You convince me that the a-c-f code is better for some realistic case (e.g. yours). - I remove the hook in a-c-f. > My fix is simpler: to recognise that f-l-default-fontify-region performs > two disparate tasks: (i) Extending the region to be fontified; (ii) > Fontifying this region. By extracting (ii) into a separate function, > that function could be called directly from jit-lock-fontify-now, > bypassing the redundant second massaging of the region. It has to be in font-lock-default-fontify-buffer since ther'es no guarantee this is only called from jit-lock-fontify-now. So you're suggesting to add a redundant call to the extend-region hook. Maybe it's simpler but it's ugly (because of the redundance, and because it adds a dependency between jit-lock and font-lock). > (i) It uses the same hook, font-lock-extend-region-function, as > f-l-after-change-functions, setting the third parameter, OLD-LEN, to nil; > (ii) It uses a distinct hook with a new name, say, > font-lock-extend-chunk-function; > (iii) It usurps the name f-l-e-r-f from its current use, abolishing the > region extension mechanism in the f-l-after-change-f/j-l-after-change. (iv) it reuses the name font-lock-extend-region-function and renames the a-c-f hook to font-lock-after-change-extend-region-function. This will happen hopefully today. At the same time font-lock-after-change-extend-region-function will be moved from font-core to font-lock (where it belongs) and its make-variable-buffer-local call will be removed as well (use make-local-variable manually if you need it). Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 12:33 ` Stefan Monnier @ 2006-04-25 14:07 ` Alan Mackenzie 2006-04-25 16:05 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 14:07 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan! On Tue, 25 Apr 2006, Stefan Monnier wrote: >> I find the elisp manual page which describes font-lock-keywords >> ("Search-based Fontification") all but inpenetrable. It's reduced me to >> tears on several occasions. I've just spent five minutes looking through >> it, trying to find the description of the form you suggested to me: >> (".*\\\\\n.*" >> (0 (progn (put-text-property (match-beginning 0) (match-end 0) >> 'font-lock-multiline t) >> nil))) >> . I didn't find it. >This form is the standard form (except you haven't noticed yet that the >FACENAME part of the font-lock-keywords is not a face symbol but an Elisp >expression whose value after exavluation should be a face). It's: > (MATCHER HIGHLIGHT ...) >where MATCHER is ".*\\\\\n.*" and HIGHLIGHT is of the form MATCH-HIGHLIGHT >which is of the form (SUBEXP FACENAME [OVERRIDE [LAXMATCH]]) where SUBEXP is >0 and FACENAME is (progn ...). OK. So FACENAME comes back as nil (which presumably means "don't apply a facename property") and the side effect is applying f-l-multiline if the buffer line matches the regexp. >> With respect, Stefan, that's rubbish. You need to know where and how >> to write the above Lisp form (the one starting (".*\\\\n.*" ...)) into >> f-l-keywords. For me, that's perhaps half an hour to an hour of >> frustration trying to find the right place in "Search-based >> Fontification". >If you don't know how font-lock-keywords work, then learn that first >before trying to work on the font-lock support for cc-mode. That'll >save you a lot more time. I cannot learn it, any more than I could learn logarithm tables when such were still in use. I need to go through the elisp page every single time. The same applies to things like c-offsets-alist too, by the way. >> What about the run-time overhead of having an extra text property over >> the entire buffer? >It's not over the entire buffer. Load a buffer and wait for it to be jit-locked. The entire buffer will have been scanned and this text property, in effect, set to t or nil on every single byte. >And I haven't seen any evidence that it's noticeable (especially >compared the number of `face' properties or of the `fontified' property >added to the whole buffer). I'm sure it's not. >> The f-l-e-r-f, as you envisage it (being called only from >> f-l-default-fontify-r) will also be called at virtually every buffer >> change, since almost every change causes a redisplay. >AFAIK, the case where a single change takes place between redisplay is the >case where there is no performance issue: it takes place at the user's >pace anyway. The important case is when Emacs has more work to do in >response to a user's action, i.e. there's more work to be done until the >next redisplay. Is it really that important to responsiveness whether the delay happens at the redisplay or immediately after the change? With the after-change f-l-e-r-f, only the two boundaries of the region get checked, plus any chunk boundaries when this is bigger than 500 bytes. With the extra pattern in f-l-keywords, then entire region gets checked. An after-change f-l-e-r-f needs to be fast enough, and if it's not, that might be a good reason for using f-l-m instead. However, f-l-m is more expensive in aggregate - it marks _every_ sequence which must be atomically fontified, just in case that sequence might straddle a chunk boundary in the future. Not only that, it erases and recalculates the property every time it fontifies a region, even when there has been no buffer change in that region. After-change f-l-e-r-f only checks for those region boundaries it needs to know immediately. [ .... ] >I'm as close as it gets to a font-lock maintainer. So from where I >stand, you're trying to impose your taste and judgment on me. I'm as close as it gets to a CC Mode maintainer at the moment. What's constraining my freedom in that department now? ;-) [ .... ] >> My fix is simpler: to recognise that f-l-default-fontify-region >> performs two disparate tasks: (i) Extending the region to be >> fontified; (ii) Fontifying this region. By extracting (ii) into a >> separate function, that function could be called directly from >> jit-lock-fontify-now, bypassing the redundant second massaging of the >> region. >It has to be in font-lock-default-fontify-buffer since there's no >guarantee this is only called from jit-lock-fontify-now. No. It has to be _called_ from font-lock-default-fontify-buffer. >So you're suggesting to add a redundant call to the extend-region hook. >Maybe it's simpler but it's ugly (because of the redundance, and because >it adds a dependency between jit-lock and font-lock). No. I'm suggesting refactoring the font lock code a little so that each path through it calls the extend-region hook exactly once. Once in the jit-lock "branch" and once in the plain font lock "branch". >> (i) It uses the same hook, font-lock-extend-region-function, as >> f-l-after-change-functions, setting the third parameter, OLD-LEN, to nil; >> (ii) It uses a distinct hook with a new name, say, >> font-lock-extend-chunk-function; >> (iii) It usurps the name f-l-e-r-f from its current use, abolishing the >> region extension mechanism in the f-l-after-change-f/j-l-after-change. >(iv) it reuses the name font-lock-extend-region-function and renames the > a-c-f hook to font-lock-after-change-extend-region-function. Or that. It's a good job our terminals are wider than 80 columns nowadays. :-) >This will happen hopefully today. Hopefully yes! >At the same time font-lock-after-change-extend-region-function will be >moved from font-core to font-lock (where it belongs) and its >make-variable-buffer-local call will be removed as well (use >make-local-variable manually if you need it). Surely not - that variable is essentially buffer local, and it makes no sense for it not to be so. To remove the make-variable-buffer-local call could lead to the same problems which have bedevilled paragraph-start and paragraph-separate. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 14:07 ` Alan Mackenzie @ 2006-04-25 16:05 ` Stefan Monnier 2006-04-25 16:09 ` Stefan Monnier 2006-04-25 21:49 ` Alan Mackenzie 0 siblings, 2 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 16:05 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel > OK. So FACENAME comes back as nil (which presumably means "don't apply a > facename property") and the side effect is applying f-l-multiline if the > buffer line matches the regexp. Right. >>> What about the run-time overhead of having an extra text property over >>> the entire buffer? >> It's not over the entire buffer. > Load a buffer and wait for it to be jit-locked. The entire buffer will > have been scanned and this text property, in effect, set to t or nil on > every single byte. Actually the `nil' value is not added, so the property is only added where it's non-nil. Hence it's not added to the entire buffer. >> And I haven't seen any evidence that it's noticeable (especially >> compared the number of `face' properties or of the `fontified' property >> added to the whole buffer). > I'm sure it's not. Good. > Is it really that important to responsiveness whether the delay happens > at the redisplay or immediately after the change? With the after-change > f-l-e-r-f, only the two boundaries of the region get checked, plus any > chunk boundaries when this is bigger than 500 bytes. With the extra > pattern in f-l-keywords, then entire region gets checked. But a single command may do thousands of buffer-modifications. Look at it this way. Let's define things as follows: - a single buffer-modification, without your hook, takes time T1 - font-locking, without my font-lock-multiline thingy, takes time T2 - your hook takes times N1*T1 - my font-lock-multiline takes time N2*T2 So your approach can slow down user-visible execution by a fator N1, whereas mine can slow it down by a factor N2. So the important question (to me anyway) is whether N1 is expected to be bigger or smaller than N2. Note that in all likelyhood T2 is much larger than T1, so it should be expected that N1 is significantly larger than N2. > expensive in aggregate - it marks _every_ sequence which must be > atomically fontified, just in case that sequence might straddle a chunk > boundary in the future. No: it's only applied to those atomic elements which do straddle a line boundary. >> I'm as close as it gets to a font-lock maintainer. So from where I >> stand, you're trying to impose your taste and judgment on me. > I'm as close as it gets to a CC Mode maintainer at the moment. What's > constraining my freedom in that department now? ;-) We're talking about font-lock code here. You're free to do whatever you like in cc-mode, including adding an advice to font-lock-after-change-function. >> It has to be in font-lock-default-fontify-buffer since there's no >> guarantee this is only called from jit-lock-fontify-now. > No. It has to be _called_ from font-lock-default-fontify-buffer. RIght, I meant to say that there's no guarantee font-lock-default-fontify-buffer will be called from jit-lock-fontify-now, which is why a call to extend-region-function needs to be in font-lock-default-fontify-buffer. >> So you're suggesting to add a redundant call to the extend-region hook. >> Maybe it's simpler but it's ugly (because of the redundance, and because >> it adds a dependency between jit-lock and font-lock). > No. I'm suggesting refactoring the font lock code a little so that each > path through it calls the extend-region hook exactly once. Once in the > jit-lock "branch" and once in the plain font lock "branch". That's still redundancy (admittedly less so because it's only static redundancy). In my book, if you can get away with putting it it at only 1 spot, it's better from a software engineering point of view. >> At the same time font-lock-after-change-extend-region-function will be >> moved from font-core to font-lock (where it belongs) and its >> make-variable-buffer-local call will be removed as well (use >> make-local-variable manually if you need it). > Surely not - that variable is essentially buffer local, and it makes no > sense for it not to be so. To remove the make-variable-buffer-local call > could lead to the same problems which have bedevilled paragraph-start and > paragraph-separate. It's a variable which should be modified via `add-hook' (which has a parameter to specify whether it should be applied buffer-locally or not), not via setq. So there is need for make-variable-buffer-local. Especially since it does make sense to add stuff to it globally as well: Think about someone who wants to re-add the font-lock-extra-lines feature. Similarly the font-lock-multiline property could be handled by moving the code from font-lock-default-fontify-region to this hook. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 16:05 ` Stefan Monnier @ 2006-04-25 16:09 ` Stefan Monnier 2006-04-25 21:52 ` Alan Mackenzie 2006-04-25 21:49 ` Alan Mackenzie 1 sibling, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 16:09 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >> Surely not - that variable is essentially buffer local, and it makes no >> sense for it not to be so. To remove the make-variable-buffer-local call >> could lead to the same problems which have bedevilled paragraph-start and >> paragraph-separate. > It's a variable which should be modified via `add-hook' (which has > a parameter to specify whether it should be applied buffer-locally or not), > not via setq. So there is need for make-variable-buffer-local. > Especially since it does make sense to add stuff to it globally as well: > Think about someone who wants to re-add the font-lock-extra-lines feature. > Similarly the font-lock-multiline property could be handled by moving the > code from font-lock-default-fontify-region to this hook. Hmmm... sorry 'bout that: I was assuming changing it a list of functions, but indeed it's not necessarily a good idea. Still: even if it's a simple variable, you shouldn't set it via `setq' but via font-lock-defaults (which will use make-local-variable for you), so there's still no justification for make-variable-buffer-local. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 16:09 ` Stefan Monnier @ 2006-04-25 21:52 ` Alan Mackenzie 0 siblings, 0 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 21:52 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan! On Tue, 25 Apr 2006, Stefan Monnier wrote: >>> Surely not - that variable is essentially buffer local, and it makes no >>> sense for it not to be so. To remove the make-variable-buffer-local call >>> could lead to the same problems which have bedevilled paragraph-start and >>> paragraph-separate. >> It's a variable which should be modified via `add-hook' (which has >> a parameter to specify whether it should be applied buffer-locally or not), >> not via setq. So there is need for make-variable-buffer-local. >> Especially since it does make sense to add stuff to it globally as well: >> Think about someone who wants to re-add the font-lock-extra-lines feature. >> Similarly the font-lock-multiline property could be handled by moving the >> code from font-lock-default-fontify-region to this hook. >Hmmm... sorry 'bout that: I was assuming changing it a list of functions, >but indeed it's not necessarily a good idea. >Still: even if it's a simple variable, you shouldn't set it via `setq' >but via font-lock-defaults (which will use make-local-variable for you), >so there's still no justification for make-variable-buffer-local. I don't think XEmacs has got this mechanism yet, so portable code will be carrying on using `setq' for some time. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 16:05 ` Stefan Monnier 2006-04-25 16:09 ` Stefan Monnier @ 2006-04-25 21:49 ` Alan Mackenzie 2006-04-26 4:50 ` Stefan Monnier 1 sibling, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 21:49 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Evening, Stefan! On Tue, 25 Apr 2006, Stefan Monnier wrote: >> OK. So FACENAME comes back as nil (which presumably means "don't apply a >> facename property") and the side effect is applying f-l-multiline if the >> buffer line matches the regexp. >Right. >>>> What about the run-time overhead of having an extra text property over >>>> the entire buffer? >>> It's not over the entire buffer. >> Load a buffer and wait for it to be jit-locked. The entire buffer will >> have been scanned and this text property, in effect, set to t or nil on >> every single byte. >Actually the `nil' value is not added, so the property is only added where >it's non-nil. Hence it's not added to the entire buffer. Maybe the 'nil' isn't explicitly added, but the entire font-lock region (or buffer) has to be scanned. Every time a region gets fontified, the f-l-m properties are erased and every multiline sequence in that region is recalculated. In the approach I favour, only the bits at the region boundaries need checking, and then only at the time they are needed. This continual erasure and recalculation of f-l-m properties could have a serious impact on the speed of C Mode, when that finally acquires a suitable region calcualtion function - because this function will essentially have to be c-beginning-of-statement-1, with various strategies, such as cacheing, to speed it up. Calculating this function for _every_ statement within the region would be prohibitively expensive. >>> And I haven't seen any evidence that it's noticeable (especially >>> compared the number of `face' properties or of the `fontified' >>> property added to the whole buffer). >> I'm sure it's not. >Good. >> Is it really that important to responsiveness whether the delay >> happens at the redisplay or immediately after the change? With the >> after-change f-l-e-r-f, only the two boundaries of the region get >> checked, plus any chunk boundaries when this is bigger than 500 bytes. >> With the extra pattern in f-l-keywords, then entire region gets >> checked. >But a single command may do thousands of buffer-modifications. What is a typical number for an "advanced" command? >Look at it this way. Let's define things as follows: >- a single buffer-modification, without your hook, takes time T1 >- font-locking, without my font-lock-multiline thingy, takes time T2 >- your hook takes times N1*T1 >- my font-lock-multiline takes time N2*T2 >So your approach can slow down user-visible execution by a fator N1, >whereas mine can slow it down by a factor N2. So the important question >(to me anyway) is whether N1 is expected to be bigger or smaller than >N2. Equally important is T1 and T2. >Note that in all likelyhood T2 is much larger than T1, so it should be >expected that N1 is significantly larger than N2. You've given a good argument for keeping both approaches available. >> expensive in aggregate - it marks _every_ sequence which must be >> atomically fontified, just in case that sequence might straddle a chunk >> boundary in the future. >No: it's only applied to those atomic elements which do straddle >a line boundary. My understanding of the form you were suggesting for AWK Mode's f-l-keywords, ("^.*\\\\\n" . (progn ......)), was that this regexp would be matched against ALL lines in the region, marking all multi-lines, in case they might in the future come to straddle chunk boundaries. >>> I'm as close as it gets to a font-lock maintainer. So from where I >>> stand, you're trying to impose your taste and judgment on me. >> I'm as close as it gets to a CC Mode maintainer at the moment. What's >> constraining my freedom in that department now? ;-) >We're talking about font-lock code here. You're free to do whatever you >like in cc-mode, including adding an advice to >font-lock-after-change-function. Richard has specifically required this advice to be eradicated, on the grounds that if such advice is necessary there's some deficiency in the underlying functionality, and that deficiency should be fixed. [ .... ] > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 21:49 ` Alan Mackenzie @ 2006-04-26 4:50 ` Stefan Monnier 0 siblings, 0 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-26 4:50 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel > This continual erasure and recalculation of f-l-m properties could have a > serious impact on the speed of C Mode, when that finally acquires a > suitable region calcualtion function - because this function will > essentially have to be c-beginning-of-statement-1, with various > strategies, such as cacheing, to speed it up. Calculating this function > for _every_ statement within the region would be prohibitively expensive. Look at my suggested patch for cc-awk and you'll see that you never need to compute anything extra: you just have to place the font-lock-multiline property at the moment where you highlight the multiline element (at which point you already have all the necessary info). My experience has been that this has always been the case. And it's not a coincidence. >> Look at it this way. Let's define things as follows: >> - a single buffer-modification, without your hook, takes time T1 >> - font-locking, without my font-lock-multiline thingy, takes time T2 >> - your hook takes times N1*T1 >> - my font-lock-multiline takes time N2*T2 >> So your approach can slow down user-visible execution by a fator N1, >> whereas mine can slow it down by a factor N2. So the important question >> (to me anyway) is whether N1 is expected to be bigger or smaller than >> N2. > Equally important is T1 and T2. Both T1 and T2 are outside the scope of this discussion, so I'm not sure why you find them important. >> Note that in all likelyhood T2 is much larger than T1, so it should be >> expected that N1 is significantly larger than N2. > You've given a good argument for keeping both approaches available. Why? Because you think that in some potential hypothetical case, T2 might be smaller than T1? In my experience not only is T2 always much larger than T1, but N2 is also tiny (because it's really nothing more than adding a few put-text-property calls). So the reverse case sounds hopelessly unlikely to me. Get back to me when you've found one. >>> expensive in aggregate - it marks _every_ sequence which must be >>> atomically fontified, just in case that sequence might straddle a chunk >>> boundary in the future. >> No: it's only applied to those atomic elements which do straddle >> a line boundary. > My understanding of the form you were suggesting for AWK Mode's > f-l-keywords, ("^.*\\\\\n" . (progn ......)), was that this regexp would > be matched against ALL lines in the region, marking all multi-lines, > in case they might in the future come to straddle chunk boundaries. Oh yes, that's the case, sorry I misunderstood your use of "chunk" (even though it was indeed quite clear, my fault). So the only correction I'd make to your claim is that it's not "_every_ sequence which must be atomically fontified", but "_every_ multiline sequence which must be atomically fontified" (that was probably meant implicitly). >>>> I'm as close as it gets to a font-lock maintainer. So from where I >>>> stand, you're trying to impose your taste and judgment on me. >>> I'm as close as it gets to a CC Mode maintainer at the moment. What's >>> constraining my freedom in that department now? ;-) >> We're talking about font-lock code here. You're free to do whatever you >> like in cc-mode, including adding an advice to >> font-lock-after-change-function. > Richard has specifically required this advice to be eradicated, on the > grounds that if such advice is necessary there's some deficiency in the > underlying functionality, and that deficiency should be fixed. Yes, and there is no such deficiency, as shown with my sample patch, using font-lock-multiline. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 19:28 ` Alan Mackenzie 2006-04-24 21:06 ` Stefan Monnier @ 2006-04-24 21:20 ` Stefan Monnier 2006-04-25 7:45 ` Alan Mackenzie 2006-04-24 21:33 ` Stefan Monnier ` (2 subsequent siblings) 4 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-24 21:20 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel > I say to you again - your solution is not robust. I don't think it's > been tried at all (correct me if I'm wrong). Of course it has, in smerge-mode (since Emacs-21.1) which I use daily, in cvs-status (since Emacs-21.1) in perl-mode (since some time in late 2004 IIRC), in sh-script.el (a bit before perl-mode IIRC), in message.el (where it's been buggy at times because I didn't understand that I needed a font-lock-default-fontify-region hook). They do all suffer from the missing font-lock-default-fontify-region hook, just like your code, of course. > So - please leave the better tested mechanism in place. I don't claim it's buggy or even more buggy than font-lock-multiline. It's just an ugly kludge, much more heavyweight codewise (this is objectively the case in terms of lines of code, and I hope my comparison above convinces a few more people that it's also subjectively the case). Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 21:20 ` Stefan Monnier @ 2006-04-25 7:45 ` Alan Mackenzie 2006-04-25 12:12 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 7:45 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan! On Mon, 24 Apr 2006, Stefan Monnier wrote: >> I say to you again - your solution is not robust. I don't think it's >> been tried at all (correct me if I'm wrong). >Of course it has, in smerge-mode (since Emacs-21.1) which I use daily, in >cvs-status (since Emacs-21.1) in perl-mode (since some time in late 2004 >IIRC), in sh-script.el (a bit before perl-mode IIRC), in message.el (where >it's been buggy at times because I didn't understand that I needed >a font-lock-default-fontify-region hook). >They do all suffer from the missing font-lock-default-fontify-region >hook, just like your code, of course. >> So - please leave the better tested mechanism in place. >I don't claim it's buggy or even more buggy than font-lock-multiline. >It's just an ugly kludge, ..... I can't understand at all why you feel that. I really can't. To me, f-l-e-r-f seems natural, simple and obvious. Optionally setting a hook variable to a function is done countless times in Emacs. I can't see why font-lock-extend-region-function is any more ugly than, say, adaptive-fill-function. >....much more heavyweight codewise (this is objectively the case in >terms of lines of code, and I hope my comparison above convinces a few >more people that it's also subjectively the case). This is not the case. We're agreed that, for the f-l-multiline mechanism to work, it needs supplementing by a f-l-extend-region-function hook. If we need this hook anyway, why complicate it with f-l-multiline when the hook works just fine on its own? > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 7:45 ` Alan Mackenzie @ 2006-04-25 12:12 ` Stefan Monnier 2006-04-25 21:15 ` Alan Mackenzie 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 12:12 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >> I don't claim it's buggy or even more buggy than font-lock-multiline. >> It's just an ugly kludge, ..... > I can't understand at all why you feel that. I really can't. To me, > f-l-e-r-f seems natural, simple and obvious. Optionally setting a hook > variable to a function is done countless times in Emacs. I can't see why > font-lock-extend-region-function is any more ugly than, say, > adaptive-fill-function. I'm not saying the hook is a kludge: I'm saying the code that uses it is a kludge, compared to the code that uses font-lock-multiline. >> ....much more heavyweight codewise (this is objectively the case in >> terms of lines of code, and I hope my comparison above convinces a few >> more people that it's also subjectively the case). > This is not the case. We're agreed that, for the f-l-multiline mechanism > to work, it needs supplementing by a f-l-extend-region-function hook. Yes, it needs such a hook called from font-lock-default-fontify-region. Just like your current hook that's called from font-lock-after-change-function. This is orthogonal. > If we need this hook anyway, why complicate it with f-l-multiline when the > hook works just fine on its own? There are two alternatives when writing your code in cc-mode: - use a hook in f-l-d-f-r + font-lock-multiline. The font-lock-multiline part of the code is a couple lines added to font-lock-keywords. - use a hook in f-l-d-f-r + a hook in f-l-a-c-f + a hook in b-c-f. The *-change-function part of the code is a good bit more than a couple lines. In which way does font-lock-multiline complicate "it"? Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 12:12 ` Stefan Monnier @ 2006-04-25 21:15 ` Alan Mackenzie 2006-04-26 4:33 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 21:15 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Good evening, Stefan! On Tue, 25 Apr 2006, Stefan Monnier wrote: >> I can't understand at all why you feel that. I really can't. To me, >> f-l-e-r-f seems natural, simple and obvious. Optionally setting a >> hook variable to a function is done countless times in Emacs. I can't >> see why font-lock-extend-region-function is any more ugly than, say, >> adaptive-fill-function. >I'm not saying the hook is a kludge: I'm saying the code that uses it is >a kludge, compared to the code that uses font-lock-multiline. OK. I feel that existing code in AWK Mode is natural. (OK, I wrote it, so I'm not in a position to judge it dispassionately, but ...). It notes the logical EOL before a change. It notes the logical EOL after a change. The later one is the end position for fontification. By contrast, the f-l-multiline method (ab)uses the structure for setting face properties to set region boundaries for possible future changes. >>> ....much more heavyweight codewise (this is objectively the case in >>> terms of lines of code, and I hope my comparison above convinces a >>> few more people that it's also subjectively the case). >> This is not the case. We're agreed that, for the f-l-multiline >> mechanism to work, it needs supplementing by a >> f-l-extend-region-function hook. >Yes, it needs such a hook called from font-lock-default-fontify-region. >Just like your current hook that's called from >font-lock-after-change-function. This is orthogonal. Agreed. [ .... ] > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 21:15 ` Alan Mackenzie @ 2006-04-26 4:33 ` Stefan Monnier 2006-04-26 8:30 ` Alan Mackenzie 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-26 4:33 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >> I'm not saying the hook is a kludge: I'm saying the code that uses it is >> a kludge, compared to the code that uses font-lock-multiline. > OK. I feel that existing code in AWK Mode is natural. (OK, I wrote it, > so I'm not in a position to judge it dispassionately, but ...). It notes > the logical EOL before a change. It notes the logical EOL after a > change. The later one is the end position for fontification. Why doesn't the BOL before change matter? Why does the EOL before change matter? > By contrast, the f-l-multiline method (ab)uses the structure for setting > face properties to set region boundaries for possible future changes. Abuses the structure of what? There two issues about multiline elements: - properly notice them when they appear - properly *re*fontify them when they change The first is taken care of by font-lock-fontify-region-function. The second talks about *re*fontification, so it's natural that it would involves saving some info during fontification for the next time around. Doing it in a-c-f is fundamentally not the right place since it doesn't know whether and how the changed text was fontified: it works, of course, but it's not particularly more natural. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-26 4:33 ` Stefan Monnier @ 2006-04-26 8:30 ` Alan Mackenzie 2006-04-26 13:36 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-26 8:30 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan! On Wed, 26 Apr 2006, Stefan Monnier wrote: >>> I'm not saying the hook is a kludge: I'm saying the code that uses it is >>> a kludge, compared to the code that uses font-lock-multiline. >> OK. I feel that existing code in AWK Mode is natural. (OK, I wrote it, >> so I'm not in a position to judge it dispassionately, but ...). It notes >> the logical EOL before a change. It notes the logical EOL after a >> change. The later one is the end position for fontification. >Why doesn't the BOL before change matter? Maybe it does/would do in some major modes. It happens not to have any effect in AWK. >Why does the EOL before change matter? Because a buffer change can alter this EOL's syntactic context. >> By contrast, the f-l-multiline method (ab)uses the structure for setting >> face properties to set region boundaries for possible future changes. >Abuses the structure of what? font-lock-keywords. The purpose of this structure is to fontify strings matching specific regular expressions. Using it to expand the region to be font-locked at a future fontification seems utterly disconnected with this purpose. >There two issues about multiline elements: >- properly notice them when they appear >- properly *re*fontify them when they change I cannot bring myself to consider the extension of a font lock region as "the manipulation of a multiline element". It just feels too contorted. I have always understood the term "multiline element" to mean an element within the font-lock region which crosses a line boundary. [Such things occur a lot in Texinfo Mode, for example, and turning on the f-l-multiline variable does indeed make these elements fontify properly.] >The first is taken care of by font-lock-fontify-region-function. The AWK Mode regular expressions, even those that can straddle line breaks, match just fine without having to turn on the f-l-multiline variable. >The second talks about *re*fontification, so it's natural that it would >involves saving some info during fontification for the next time around. I think I would agree, except that font-lock brutally erases these properties at every fontification, only to have to recalculate them. This wastes processor time, probably quite a lot of it. What is the point of doing this, unless there has been a buffer change? The same applies to the syntax-table property. >Doing it in a-c-f is fundamentally not the right place since it doesn't >know whether and how the changed text was fontified: it works, of >course, but it's not particularly more natural. To know how text is to be refontified, you need to know how it has been changed. before/after-change-functions is a perfectly natural place to find this out. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-26 8:30 ` Alan Mackenzie @ 2006-04-26 13:36 ` Stefan Monnier 0 siblings, 0 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-26 13:36 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >>>> I'm not saying the hook is a kludge: I'm saying the code that uses it is >>>> a kludge, compared to the code that uses font-lock-multiline. >>> OK. I feel that existing code in AWK Mode is natural. (OK, I wrote it, >>> so I'm not in a position to judge it dispassionately, but ...). It notes >>> the logical EOL before a change. It notes the logical EOL after a >>> change. The later one is the end position for fontification. >> Why doesn't the BOL before change matter? > Maybe it does/would do in some major modes. It happens not to have any > effect in AWK. >> Why does the EOL before change matter? > Because a buffer change can alter this EOL's syntactic context. But why does it matter? Honestly, neither explanation seems very clear to me. I don't know what others think, but this is an example of the complexity in your code. You end up having to discuss various scenarios to justify your code, which leaves the reader with the lingering question "but are all the relevant scenarios covered?". Clearly, we 're biased, but I find that the code using font-lock-multiline doesn't suffer from such problems. It does suffer from the fact that you have to trust font-lock's use of the font-lock-multiline property, but if you assume it works, then all you have to do is "place the property on the chunk of text that needs to be refontified if any part gets changed". > font-lock-keywords. The purpose of this structure is to fontify strings > matching specific regular expressions. Using it to expand the region to > be font-locked at a future fontification seems utterly disconnected with > this purpose. My sample code using the ".*\\(\\\\\n.*\\)*" regexp was just a quick way to show you what it could look like without having to delve into your code. Check my cc-awk patch: it shows what typically happens in practice, which is that you place the put-text-property right where you add the face (or syntax-table property), so it's a lot more natural. >> There two issues about multiline elements: >> - properly notice them when they appear >> - properly *re*fontify them when they change > I cannot bring myself to consider the extension of a font lock region as > "the manipulation of a multiline element". I don't understand what you're saying here. > It just feels too contorted. I have always understood the term "multiline > element" to mean an element within the font-lock region which crosses > a line boundary. IIUC, that's how I understand it as well, except I wouldn't restrict it to the "font-lock region". >> The first is taken care of by font-lock-fontify-region-function. > The AWK Mode regular expressions, even those that can straddle line > breaks, match just fine without having to turn on the f-l-multiline > variable. I don't see how my text and yours are related here. >> The second talks about *re*fontification, so it's natural that it would >> involves saving some info during fontification for the next time around. > I think I would agree, except that font-lock brutally erases these > properties at every fontification, only to have to recalculate them. > This wastes processor time, probably quite a lot of it. No it doesn't *particularly* "waste processor time quite a lot": it's just what font-lock does for a living: erase text-properties and the re-add them (usually it only does it for `face', sometimes it does it for `syntax-table' and/or `font-lock-multiline', and it can do that for other properties (see font-lock-extra-managed-props)). > What is the point of doing this, unless there has been a buffer change? If font-lock really happens to remove a font-lock-multiline property, it means that somebody placed it there before. Since this property is normally placed during font-locking, it implies that we're actually *re*fontifying. *Re*fontification normally takes place in response to a buffer change. So, the "removal + re-addition" of the font-lock-multiline property (just like any other property managed by font-lock) indeed only happens if there has been a buffer change. >> Doing it in a-c-f is fundamentally not the right place since it doesn't >> know whether and how the changed text was fontified: it works, of >> course, but it's not particularly more natural. > To know how text is to be refontified, you need to know how it has been > changed. The font-lock-multiline property is a "living" proof that this is not true. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 19:28 ` Alan Mackenzie 2006-04-24 21:06 ` Stefan Monnier 2006-04-24 21:20 ` Stefan Monnier @ 2006-04-24 21:33 ` Stefan Monnier 2006-04-25 7:27 ` Alan Mackenzie 2006-04-25 4:39 ` Tomas Zerolo 2006-04-25 19:02 ` Ralf Angeli 4 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-24 21:33 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel > I say to you again - your solution is not robust. I don't think it's > been tried at all (correct me if I'm wrong). A variant form of > f-l-extend-region-f (the advice on the f-l a-c-functions, as implemented > for AWK Mode) has been in use since June 2003, and so far no bugs related > to it have been reported by users. We are both aware of a refinement > which is needed, namely calling some sort of f-l-extend-region-f from > f-l-d-f-r and j-l-f-n. > So - please leave the better tested mechanism in place. I feel like Alice in wonderland: you talk as if I wanted to remove a well-established hook in favor of some new fangled feature, whereas the font-lock-multiline property is the one that's been in Emacs since 21.1, whereas your hook is the one that's new and that you want to add to Emacs-22. I think in order to justify this new hook you need to show that the pre-existing font-lock-multiline solution is not good enough in some cases. I know you have a distaste for it, but I find it to be a far cry from showing that it's not good enough. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 21:33 ` Stefan Monnier @ 2006-04-25 7:27 ` Alan Mackenzie 2006-04-25 12:03 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 7:27 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Hi, Stefan! On Mon, 24 Apr 2006, Stefan Monnier wrote: >> I say to you again - your solution is not robust. I don't think it's >> been tried at all (correct me if I'm wrong). A variant form of >> f-l-extend-region-f (the advice on the f-l a-c-functions, as implemented >> for AWK Mode) has been in use since June 2003, and so far no bugs related >> to it have been reported by users. We are both aware of a refinement >> which is needed, namely calling some sort of f-l-extend-region-f from >> f-l-d-f-r and j-l-f-n. >> So - please leave the better tested mechanism in place. >I feel like Alice in wonderland: So do I. :-) I'm still not 100% sure that we're arguing about exactly the same thing. >you talk as if I wanted to remove a well-established hook in favor of >some new fangled feature, whereas the font-lock-multiline property is >the one that's been in Emacs since 21.1, whereas your hook is the one >that's new and that you want to add to Emacs-22. It has already been added. I'm desperately trying to stop you from taking it away again, because that would make my (Emacs) life much more difficult. >I think in order to justify this new hook you need to show that the >pre-existing font-lock-multiline solution is not good enough in some >cases. I don't. I merely need to point out that Richard specifically asked for this hook to be installed. When he did so, he was aware that it was called from an after-change function. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 7:27 ` Alan Mackenzie @ 2006-04-25 12:03 ` Stefan Monnier 2006-04-25 13:14 ` Alan Mackenzie 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 12:03 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel >> you talk as if I wanted to remove a well-established hook in favor of >> some new fangled feature, whereas the font-lock-multiline property is >> the one that's been in Emacs since 21.1, whereas your hook is the one >> that's new and that you want to add to Emacs-22. > It has already been added. It hasn't been added in released code yet. > I'm desperately trying to stop you from taking it away again, because that > would make my (Emacs) life much more difficult. Actually no: as I've shown with my sample alternative solution, it would only force you to make you life simpler. >> I think in order to justify this new hook you need to show that the >> pre-existing font-lock-multiline solution is not good enough in some >> cases. > I don't. I merely need to point out that Richard specifically asked for > this hook to be installed. No: he agreed to install it after you claimed it was needed. As a matter of fact, it was needed in font-lock-default-fontify-region (where you didn't add it) but not in font-lock-after-change-function. > When he did so, he was aware that it was called from an > after-change function. It seemed clear to him that an "extend-region" hook was a good idea and we all agree that it is. But I highly doubt it that he took the time to understand enough to be able to judge whether placing it in font-lock-after-change-function was a good choice or not. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 12:03 ` Stefan Monnier @ 2006-04-25 13:14 ` Alan Mackenzie 2006-04-26 0:22 ` Miles Bader 0 siblings, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 13:14 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Good afternoon, Stefan! On Tue, 25 Apr 2006, Stefan Monnier wrote: >>> you talk as if I wanted to remove a well-established hook in favor of >>> some new fangled feature, whereas the font-lock-multiline property is >>> the one that's been in Emacs since 21.1, whereas your hook is the one >>> that's new and that you want to add to Emacs-22. >> It has already been added. >It hasn't been added in released code yet. >> I'm desperately trying to stop you from taking it away again, because >> that would make my (Emacs) life much more difficult. >Actually no: as I've shown with my sample alternative solution, it would >only force you to make you life simpler. You have shown nothing of the kind. You're in no better position to decide what is objectively simpler than I am, since neither of is is unbiassed in this debate. I'm in a better position than you to know what makes MY life simpler. Please respect that. >>> I think in order to justify this new hook you need to show that the >>> pre-existing font-lock-multiline solution is not good enough in some >>> cases. >> I don't. I merely need to point out that Richard specifically asked for >> this hook to be installed. >No: he agreed to install it after you claimed it was needed. As a >matter of fact, it was needed in font-lock-default-fontify-region (where >you didn't add it) but not in font-lock-after-change-function. He requested it. Subject: Re: [sigra@home.se: C++-mode: Syntax highlighting: wrong color for function identifier depending on the kind of whitespace that follows]. RMS, 2005-02-12: >Here's the change I was talking about. Would someone please adapt this >_now_ to the current sources, and install it? Then please rename >before-font-lock-after-change-function to >font-lock-extend-region-function, and rename >font-lock-run-before-after-change-hook to font-lock-extend-region? That was a positive request, not an acquiescence. In fact, a year earlier in the same thread: RMS, 2005-03-11: >I think we should add that hook now. >It is simple and easy to understand. >Even if in some cases it is not a good solution, >it is a good solution part of the time. >That is enough reason. It seems clear that Richard totally understood my patch. However, he is the best person to say what actually meant. At the time, you saw its purpose as being subtlely different from my intention, and we spent some time at loggerheads because of this. As a result of that discussion, it became clear to me that we need a similar hook to adjust the 500-byte chunks produced by jit. >> When he did so, he was aware that it was called from an after-change >> function. >It seemed clear to him that an "extend-region" hook was a good idea and >we all agree that it is. But I highly doubt it that he took the time to >understand enough to be able to judge whether placing it in >font-lock-after-change-function was a good choice or not. RMS: "It is simple and easy to understand." I think he can speak for himself on this matter, and we should let him. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 13:14 ` Alan Mackenzie @ 2006-04-26 0:22 ` Miles Bader 0 siblings, 0 replies; 80+ messages in thread From: Miles Bader @ 2006-04-26 0:22 UTC (permalink / raw) Cc: emacs-devel, Stefan Monnier, Richard Stallman Alan Mackenzie <acm@muc.de> writes: >>Actually no: as I've shown with my sample alternative solution, it would >>only force you to make you life simpler. > > You have shown nothing of the kind. You're in no better position to > decide what is objectively simpler than I am, since neither of is is > unbiassed in this debate. I'm in a better position than you to know what > makes MY life simpler. Please respect that. I gotta say -- I know nothing about the subject, but as far as the code posted, Stefan's looks a _lot_ simpler. More specifically, I get the sense I could figure out what Stefan's code is doing by reading the docs a bit, whereas the corresponding code from Alan seems impenetrable. Maybe that doesn't matter to you Alan, because you're immersed in your current solution, but I think it would be good to think about future maintainers too. -Miles -- We are all lying in the gutter, but some of us are looking at the stars. -Oscar Wilde ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 19:28 ` Alan Mackenzie ` (2 preceding siblings ...) 2006-04-24 21:33 ` Stefan Monnier @ 2006-04-25 4:39 ` Tomas Zerolo 2006-04-25 19:02 ` Ralf Angeli 4 siblings, 0 replies; 80+ messages in thread From: Tomas Zerolo @ 2006-04-25 4:39 UTC (permalink / raw) Cc: emacs-devel, Stefan Monnier, Richard Stallman [-- Attachment #1.1: Type: text/plain, Size: 328 bytes --] On Mon, Apr 24, 2006 at 07:28:44PM +0000, Alan Mackenzie wrote: [on font lock] > P.S. We're boring everybody else as well as ourselves. :-( No way. I don't know about yourselves, but I'm not bored. I'm following and learning a lot from the discusion, even if I'm not understanding everything. Thanks -- tomás [-- Attachment #1.2: Digital signature --] [-- Type: application/pgp-signature, Size: 189 bytes --] [-- Attachment #2: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-24 19:28 ` Alan Mackenzie ` (3 preceding siblings ...) 2006-04-25 4:39 ` Tomas Zerolo @ 2006-04-25 19:02 ` Ralf Angeli 2006-04-25 19:30 ` Stefan Monnier 2006-04-25 22:16 ` Alan Mackenzie 4 siblings, 2 replies; 80+ messages in thread From: Ralf Angeli @ 2006-04-25 19:02 UTC (permalink / raw) * Alan Mackenzie (2006-04-24) writes: > after-change-functions is the canonical way of solving problems related > to buffer changes. This includes Font Locking. But extending the region in the after-change case does not help with "normal" fontification by region as e.g. done by jit-lock. > I say to you again - your solution is not robust. I don't think it's > been tried at all (correct me if I'm wrong). Hopefully I understand "your solution" correctly here if I am assuming this relates to putting the hook into `font-lock-default-fontify-region'. I've implemented a variant of this approach for AUCTeX by using a custom-made function for `font-lock-fontify-region-function' which calls other functions for extending the region and then calls `font-lock-default-fontify-region' with the extended region as arguments. There is no special code in `after-change-functions' and the approach actually works quite well. > We are both aware of a refinement > which is needed, namely calling some sort of f-l-extend-region-f from > f-l-d-f-r and j-l-f-n. If putting the hook into `font-lock-default-fontify-region' works reliably, why do you need it in `after-change-functions' and `jit-lock-fontify-now' as well? -- Ralf ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 19:02 ` Ralf Angeli @ 2006-04-25 19:30 ` Stefan Monnier 2006-04-25 20:12 ` Ralf Angeli 2006-04-25 22:30 ` Alan Mackenzie 2006-04-25 22:16 ` Alan Mackenzie 1 sibling, 2 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 19:30 UTC (permalink / raw) Cc: emacs-devel > Hopefully I understand "your solution" correctly here if I am assuming > this relates to putting the hook into > `font-lock-default-fontify-region'. I've implemented a variant of > this approach for AUCTeX by using a custom-made function for > `font-lock-fontify-region-function' which calls other functions for > extending the region and then calls `font-lock-default-fontify-region' > with the extended region as arguments. There is no special code in > `after-change-functions' and the approach actually works quite well. Hey, that's a very good point. There isn't any real need for a new "extend-region" hook in font-lock-default-fontify-region since we can simply use the font-lock-fontify-region-function hook for that purpose. Thanks for pointing it out, Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 19:30 ` Stefan Monnier @ 2006-04-25 20:12 ` Ralf Angeli 2006-04-25 20:26 ` Stefan Monnier 2006-04-25 22:30 ` Alan Mackenzie 1 sibling, 1 reply; 80+ messages in thread From: Ralf Angeli @ 2006-04-25 20:12 UTC (permalink / raw) * Stefan Monnier (2006-04-25) writes: >> Hopefully I understand "your solution" correctly here if I am assuming >> this relates to putting the hook into >> `font-lock-default-fontify-region'. I've implemented a variant of >> this approach for AUCTeX by using a custom-made function for >> `font-lock-fontify-region-function' which calls other functions for >> extending the region and then calls `font-lock-default-fontify-region' >> with the extended region as arguments. There is no special code in >> `after-change-functions' and the approach actually works quite well. > > Hey, that's a very good point. There isn't any real need for a new > "extend-region" hook in font-lock-default-fontify-region since we can simply > use the font-lock-fontify-region-function hook for that purpose. > > Thanks for pointing it out, Now I feel a bit guilty that I didn't mention this earlier. Anyway, I thought that a hook in f-l-d-f-r might be useful despite the possibility mentioned above as a "documentable" and common way of extending the region. But you are right, it's not really necessary. But then, IMO, a hook in `after-change-functions' is not strictly necessary either. Its only benefit might be that refontification of the extended region happens faster. -- Ralf ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 20:12 ` Ralf Angeli @ 2006-04-25 20:26 ` Stefan Monnier 2006-04-25 20:58 ` Ralf Angeli 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 20:26 UTC (permalink / raw) Cc: emacs-devel > Now I feel a bit guilty that I didn't mention this earlier. Anyway, I > thought that a hook in f-l-d-f-r might be useful despite the > possibility mentioned above as a "documentable" and common way of > extending the region. But you are right, it's not really necessary. I was thinking of documenting this use of font-lock-fontify-region-function in the new "multi line font lock elements" node of the elisp manual. Do you think a separate hook is still necessary? The way I see it, a separate hook would only really make sense if it were a "normal" hook where you can place several functions, so that minor modes can also use it reliably and easily. > But then, IMO, a hook in `after-change-functions' is not strictly > necessary either. Its only benefit might be that refontification of > the extended region happens faster. I don't understand what you're saying here. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 20:26 ` Stefan Monnier @ 2006-04-25 20:58 ` Ralf Angeli 2006-04-25 21:11 ` Stefan Monnier 0 siblings, 1 reply; 80+ messages in thread From: Ralf Angeli @ 2006-04-25 20:58 UTC (permalink / raw) * Stefan Monnier (2006-04-25) writes: >> Now I feel a bit guilty that I didn't mention this earlier. Anyway, I >> thought that a hook in f-l-d-f-r might be useful despite the >> possibility mentioned above as a "documentable" and common way of >> extending the region. But you are right, it's not really necessary. > > I was thinking of documenting this use of font-lock-fontify-region-function > in the new "multi line font lock elements" node of the elisp manual. > Do you think a separate hook is still necessary? If the above mentioned use of f-l-f-r-f will be documented, the need for a hook diminishes even further. It would only serve as a more convenient way of achieving the same thing. So, no, I don't think the hook is still necessary. > The way I see it, a separate hook would only really make sense if it were > a "normal" hook where you can place several functions, so that minor modes > can also use it reliably and easily. Yes. In AUCTeX I used a regular list, not a hook, for this purpose. >> But then, IMO, a hook in `after-change-functions' is not strictly >> necessary either. Its only benefit might be that refontification of >> the extended region happens faster. > > I don't understand what you're saying here. When experimenting with extension of the region to be fontified in AUCTeX I advised `jit-lock-after-change' to do this extension and with that piece of advice refontification of a multiline construct after a buffer change happened immediately while without the advice it took a few moments. Possibly as long as `jit-lock-context-time' IIUC. -- Ralf ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 20:58 ` Ralf Angeli @ 2006-04-25 21:11 ` Stefan Monnier 0 siblings, 0 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-25 21:11 UTC (permalink / raw) Cc: emacs-devel > When experimenting with extension of the region to be fontified in > AUCTeX I advised `jit-lock-after-change' to do this extension and with > that piece of advice refontification of a multiline construct after a > buffer change happened immediately while without the advice it took a > few moments. Possibly as long as `jit-lock-context-time' IIUC. That seems to be unrelated to any kind of hook in font-lock-fontify-region. It seems what happens is that without the advice the text doesn't get a font-lock-multiline property but happens to still be correctly refreshed thanks to jit-lock-contextually. Or something like that. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 19:30 ` Stefan Monnier 2006-04-25 20:12 ` Ralf Angeli @ 2006-04-25 22:30 ` Alan Mackenzie 2006-04-26 4:25 ` Stefan Monnier 1 sibling, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 22:30 UTC (permalink / raw) Cc: Ralf Angeli, emacs-devel Hi, Stefan! On Tue, 25 Apr 2006, Stefan Monnier wrote: >> Hopefully I understand "your solution" correctly here if I am assuming >> this relates to putting the hook into >> `font-lock-default-fontify-region'. I've implemented a variant of >> this approach for AUCTeX by using a custom-made function for >> `font-lock-fontify-region-function' which calls other functions for >> extending the region and then calls `font-lock-default-fontify-region' >> with the extended region as arguments. There is no special code in >> `after-change-functions' and the approach actually works quite well. >Hey, that's a very good point. There isn't any real need for a new >"extend-region" hook in font-lock-default-fontify-region since we can >simply use the font-lock-fontify-region-function hook for that purpose. Would this not be very inconvenient? Somebody needing the extend-region functionality in f-l-d-f-r would then have to copy the entire source of the default function, amend the source, make the f-l-fontify-region-funtion buffer local, then set it. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 22:30 ` Alan Mackenzie @ 2006-04-26 4:25 ` Stefan Monnier 2006-04-26 7:44 ` Alan Mackenzie 0 siblings, 1 reply; 80+ messages in thread From: Stefan Monnier @ 2006-04-26 4:25 UTC (permalink / raw) Cc: Ralf Angeli, emacs-devel > Would this not be very inconvenient? Somebody needing the extend-region > functionality in f-l-d-f-r would then have to copy the entire source of > the default function, amend the source, make the > f-l-fontify-region-funtion buffer local, then set it. Of course not, just use a function like (defun c-awk-font-lock-fontify-region (beg end &rest args) (apply 'font-lock-default-fontify-region (c-awk-beginning-of-logical-line beg) (c-awk-end-of-logical-line end) args)) -- Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-26 4:25 ` Stefan Monnier @ 2006-04-26 7:44 ` Alan Mackenzie 0 siblings, 0 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-26 7:44 UTC (permalink / raw) Cc: Ralf Angeli, emacs-devel Hi, Stefan~ On Wed, 26 Apr 2006, Stefan Monnier wrote: >> Would this not be very inconvenient? Somebody needing the extend-region >> functionality in f-l-d-f-r would then have to copy the entire source of >> the default function, amend the source, make the >> f-l-fontify-region-funtion buffer local, then set it. >Of course not, just use a function like > (defun c-awk-font-lock-fontify-region (beg end &rest args) > (apply 'font-lock-default-fontify-region > (c-awk-beginning-of-logical-line beg) > (c-awk-end-of-logical-line end) > args)) Heh! Thanks, I should have spotted that myself! That wouldn't work in the general case, though, because the current implementation of f-l-d-f-r would foul up the BEG and END parameters unless they happen to be at line boundaries. This is another reason for the refactoring I've been suggesting, to extract a core function from f-l-d-f-r which fontifies exactly the region it's given. >-- Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 19:02 ` Ralf Angeli 2006-04-25 19:30 ` Stefan Monnier @ 2006-04-25 22:16 ` Alan Mackenzie 2006-04-26 4:36 ` Stefan Monnier 1 sibling, 1 reply; 80+ messages in thread From: Alan Mackenzie @ 2006-04-25 22:16 UTC (permalink / raw) Cc: emacs-devel Hi, Ralf! On Tue, 25 Apr 2006, Ralf Angeli wrote: >* Alan Mackenzie (2006-04-24) writes: >> after-change-functions is the canonical way of solving problems related >> to buffer changes. This includes Font Locking. >But extending the region in the after-change case does not help with >"normal" fontification by region as e.g. done by jit-lock. >> I say to you again - your solution is not robust. I don't think it's >> been tried at all (correct me if I'm wrong). >Hopefully I understand "your solution" correctly here if I am assuming >this relates to putting the hook into >`font-lock-default-fontify-region'. Yes - more precisely, putting the hook ONLY into f-l-d-f-r. >I've implemented a variant of this approach for AUCTeX by using a >custom-made function for `font-lock-fontify-region-function' which calls >other functions for extending the region and then calls >`font-lock-default-fontify-region' with the extended region as >arguments. There is no special code in `after-change-functions' and the >approach actually works quite well. >> We are both aware of a refinement which is needed, namely calling some >> sort of f-l-extend-region-f from f-l-d-f-r and j-l-f-n. >If putting the hook into `font-lock-default-fontify-region' works >reliably, why do you need it in `after-change-functions' and >`jit-lock-fontify-now' as well? It doesn't work reliably. The region which needs fontification after a buffer change depends on that _change_, not merely on what the buffer looks like after the change. If the f-l-extend-region-f isn't called from after-change-functions, necessary information gets lost. Take, once more, the standard canonical example: 1: "string \ 2: over \ 3: several <======= point is here, having just deleted a character. 4: lines." Point is at EOL 3, having just deleted a character. A hook in f-l-after-change-functions cannot determine whether L4 belongs to the region, without knowing whether the deleted character was a \. A hook is thus necessary in font-lock-after-change-functions too, to record this information in some fashion. >Ralf -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (font-lock-extend-region-function) 2006-04-25 22:16 ` Alan Mackenzie @ 2006-04-26 4:36 ` Stefan Monnier 0 siblings, 0 replies; 80+ messages in thread From: Stefan Monnier @ 2006-04-26 4:36 UTC (permalink / raw) Cc: Ralf Angeli, emacs-devel >> If putting the hook into `font-lock-default-fontify-region' works >> reliably, why do you need it in `after-change-functions' and >> `jit-lock-fontify-now' as well? > It doesn't work reliably. On its own, it doesn't. Just like an after-change-function on its own is not reliable, and a font-lock-multiline on its own is not reliable. But he uses font-lock-multiline as well, and that coupled with font-lock-default-fontify-region *is* reliable. > 1: "string \ > 2: over \ > 3: several <======= point is here, having just deleted a character. > 4: lines." Come on' Alan, we've been through this: font-lock-multiline handles it just fine. Stefan ^ permalink raw reply [flat|nested] 80+ messages in thread
* RE: Last steps for pretesting (font-lock-extend-region-function) 2006-04-21 19:51 ` Alan Mackenzie 2006-04-21 22:28 ` Stefan Monnier @ 2006-04-21 23:14 ` Drew Adams 1 sibling, 0 replies; 80+ messages in thread From: Drew Adams @ 2006-04-21 23:14 UTC (permalink / raw) Do we get a summary, once you have reached fixed point? Thx. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting (true file names in load-history?) 2006-04-19 21:23 ` Stefan Monnier 2006-04-19 21:30 ` David Kastrup 2006-04-19 22:43 ` Last steps for pretesting (font-lock-extend-region-function) Alan Mackenzie @ 2006-04-19 22:53 ` Alan Mackenzie 2 siblings, 0 replies; 80+ messages in thread From: Alan Mackenzie @ 2006-04-19 22:53 UTC (permalink / raw) Cc: Richard Stallman, emacs-devel Evening again, Stefan! On Wed, 19 Apr 2006, Stefan Monnier wrote: >> (iv) A problem in Emacs (See: Subject: Problem mit symlinks, >> locate-library and load-history. Date: Tue, 19 Mar 2006). This is >> caused by (a) load-history storing a promiscuous mix of true file names >> and file names containing symbolic links; (b) eval-after-load only >> recognising files as being already loaded if they're the ones currently >> showing on load-path. I've half-solved this problem by writing a >> function to search load-history properly, which I posted to emacs-devel. >> Richard tentatively approved it. Would it be possible to define >> load-history such that all file names in it are true names? >As mentioned in that thread, I think it's better to "truify" names >on-the-fly when you need it rather than directly inside load-history. I think that's wrong. Every time you evaluate an eval-after-load form, you've got to scan the list of loaded files in load-history. The way you suggest, the names in load-history would need to be "truified" (Hey, that's a lovely word!) many times, once for each eval-after-load. If you "truify" the names as you enter them into load-history, you only need to do it once. However, I think it is urgently necessary to define rigorously what file name goes into load-history. The lack of this rigour is what triggered a bug in CC Mode "File mode specification error: (void-variable c-font-lock-keywords-3)". I think the absolute true file name would be the best. > Stefan -- Alan. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-19 17:02 ` Alan Mackenzie 2006-04-19 21:23 ` Stefan Monnier @ 2006-04-20 1:14 ` Richard Stallman 2006-04-24 17:52 ` Richard Stallman 2 siblings, 0 replies; 80+ messages in thread From: Richard Stallman @ 2006-04-20 1:14 UTC (permalink / raw) Cc: emacs-devel (iii) Tracking down the "CC Mode loads CL" problem. (I've made good progress on this). I sent you some info about it yesterday. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-19 17:02 ` Alan Mackenzie 2006-04-19 21:23 ` Stefan Monnier 2006-04-20 1:14 ` Last steps for pretesting Richard Stallman @ 2006-04-24 17:52 ` Richard Stallman 2 siblings, 0 replies; 80+ messages in thread From: Richard Stallman @ 2006-04-24 17:52 UTC (permalink / raw) Cc: emacs-devel Richard, when do you envisage going into pretest? It's a matter of fixing the pending bugs. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-17 0:56 Last steps for pretesting Richard Stallman ` (2 preceding siblings ...) 2006-04-19 17:02 ` Alan Mackenzie @ 2006-04-20 10:54 ` Reiner Steib 2006-04-21 0:10 ` Richard Stallman 3 siblings, 1 reply; 80+ messages in thread From: Reiner Steib @ 2006-04-20 10:54 UTC (permalink / raw) On Mon, Apr 17 2006, Richard Stallman wrote: > 3. check a few manual files: > > anti.texi glossary.texi macos.texi > maintaining.texi misc.texi msdog.texi trouble.texi xresources.texi is missing here. Bye, Reiner. -- ,,, (o o) ---ooO-(_)-Ooo--- | PGP key available | http://rsteib.home.pages.de/ ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-20 10:54 ` Reiner Steib @ 2006-04-21 0:10 ` Richard Stallman 2006-04-21 5:46 ` David Kastrup 0 siblings, 1 reply; 80+ messages in thread From: Richard Stallman @ 2006-04-21 0:10 UTC (permalink / raw) Cc: emacs-devel xresources.texi is missing here. I think we will move most or all of that out of the Emacs manual. It is not necessary for editing with Emacs. ^ permalink raw reply [flat|nested] 80+ messages in thread
* Re: Last steps for pretesting 2006-04-21 0:10 ` Richard Stallman @ 2006-04-21 5:46 ` David Kastrup 0 siblings, 0 replies; 80+ messages in thread From: David Kastrup @ 2006-04-21 5:46 UTC (permalink / raw) Cc: Reiner Steib, emacs-devel Richard Stallman <rms@gnu.org> writes: > xresources.texi is missing here. > > I think we will move most or all of that out of the Emacs manual. > It is not necessary for editing with Emacs. I strongly disagree. The X resources (and equivalents) are the only way to specify startup fonts and geometry in a way that makes them work on Emacs startup such that the window manager has a chance to accommodate them. X resources and startup options are so important that they are one of the few things mentioned in man pages of programs even when separate and more elaborate info pages exist. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 80+ messages in thread
end of thread, other threads:[~2006-04-26 13:55 UTC | newest] Thread overview: 80+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2006-04-17 0:56 Last steps for pretesting Richard Stallman 2006-04-17 8:01 ` Ramprasad B 2006-04-17 8:37 ` Eli Zaretskii 2006-04-18 1:42 ` Richard Stallman 2006-04-19 3:54 ` Ramprasad B 2006-04-19 15:40 ` Richard Stallman 2006-04-20 5:23 ` Ramprasad B 2006-04-18 17:31 ` Bill Wohler 2006-04-19 17:02 ` Alan Mackenzie 2006-04-19 21:23 ` Stefan Monnier 2006-04-19 21:30 ` David Kastrup 2006-04-20 21:35 ` Stefan Monnier 2006-04-20 21:37 ` David Kastrup 2006-04-20 21:44 ` Stefan Monnier 2006-04-20 21:46 ` David Kastrup 2006-04-20 22:50 ` Alan Mackenzie 2006-04-19 22:43 ` Last steps for pretesting (font-lock-extend-region-function) Alan Mackenzie 2006-04-20 9:13 ` Alan Mackenzie 2006-04-20 17:46 ` Stefan Monnier 2006-04-20 18:12 ` Alan Mackenzie 2006-04-20 20:58 ` Stefan Monnier 2006-04-20 22:40 ` Alan Mackenzie 2006-04-20 23:34 ` Stefan Monnier 2006-04-21 7:58 ` Alan Mackenzie 2006-04-21 12:18 ` Stefan Monnier 2006-04-21 19:51 ` Alan Mackenzie 2006-04-21 22:28 ` Stefan Monnier 2006-04-24 19:28 ` Alan Mackenzie 2006-04-24 21:06 ` Stefan Monnier 2006-04-25 6:21 ` Ralf Angeli 2006-04-25 10:53 ` Alan Mackenzie 2006-04-25 18:37 ` Ralf Angeli 2006-04-25 22:00 ` Alan Mackenzie 2006-04-26 6:14 ` Ralf Angeli 2006-04-25 19:26 ` Stefan Monnier 2006-04-25 20:10 ` font-lock-multiline for cc-awk Stefan Monnier 2006-04-26 7:37 ` Alan Mackenzie 2006-04-26 13:55 ` Stefan Monnier 2006-04-25 19:23 ` Last steps for pretesting (font-lock-extend-region-function) Stefan Monnier 2006-04-25 20:18 ` Stefan Monnier 2006-04-25 11:33 ` Alan Mackenzie 2006-04-25 11:59 ` David Kastrup 2006-04-25 12:33 ` Stefan Monnier 2006-04-25 14:07 ` Alan Mackenzie 2006-04-25 16:05 ` Stefan Monnier 2006-04-25 16:09 ` Stefan Monnier 2006-04-25 21:52 ` Alan Mackenzie 2006-04-25 21:49 ` Alan Mackenzie 2006-04-26 4:50 ` Stefan Monnier 2006-04-24 21:20 ` Stefan Monnier 2006-04-25 7:45 ` Alan Mackenzie 2006-04-25 12:12 ` Stefan Monnier 2006-04-25 21:15 ` Alan Mackenzie 2006-04-26 4:33 ` Stefan Monnier 2006-04-26 8:30 ` Alan Mackenzie 2006-04-26 13:36 ` Stefan Monnier 2006-04-24 21:33 ` Stefan Monnier 2006-04-25 7:27 ` Alan Mackenzie 2006-04-25 12:03 ` Stefan Monnier 2006-04-25 13:14 ` Alan Mackenzie 2006-04-26 0:22 ` Miles Bader 2006-04-25 4:39 ` Tomas Zerolo 2006-04-25 19:02 ` Ralf Angeli 2006-04-25 19:30 ` Stefan Monnier 2006-04-25 20:12 ` Ralf Angeli 2006-04-25 20:26 ` Stefan Monnier 2006-04-25 20:58 ` Ralf Angeli 2006-04-25 21:11 ` Stefan Monnier 2006-04-25 22:30 ` Alan Mackenzie 2006-04-26 4:25 ` Stefan Monnier 2006-04-26 7:44 ` Alan Mackenzie 2006-04-25 22:16 ` Alan Mackenzie 2006-04-26 4:36 ` Stefan Monnier 2006-04-21 23:14 ` Drew Adams 2006-04-19 22:53 ` Last steps for pretesting (true file names in load-history?) Alan Mackenzie 2006-04-20 1:14 ` Last steps for pretesting Richard Stallman 2006-04-24 17:52 ` Richard Stallman 2006-04-20 10:54 ` Reiner Steib 2006-04-21 0:10 ` Richard Stallman 2006-04-21 5:46 ` David Kastrup
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.