Re: Is there something like `on-display-functions'?

Re: Is there something like `on-display-functions'?

[Lennart Borgman]

On Wed, Jan 27, 2010 at 2:57 PM, Alan Mackenzie <acm@muc.de> wrote:
> Hi, Emacs,
>
> Is there some hook called each time something's about to be displayed on
> the screen (regardless of whether or not font-lock is enabled)?
>
> If there is, I could use it to apply the appropriate text properties to
> C++ template delimiters as they're about to be displayed, thus
> potentially speeding up startup for C++ (and like languages).
>
> Yes, I've tried searching for it and not found it.


Just curious, but why don't you want to use font-lock for it?

Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, Emacs,

Is there some hook called each time something's about to be displayed on
the screen (regardless of whether or not font-lock is enabled)?

If there is, I could use it to apply the appropriate text properties to
C++ template delimiters as they're about to be displayed, thus
potentially speeding up startup for C++ (and like languages).

Yes, I've tried searching for it and not found it.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[alin.s]

The answer is surely no, because the only place where such a hook should be
called is REDISPLAY_INTERNAL or its backtrace, and on its trace there is no
call of hook functions.

I am just curious why do you need it.


Alin.




Alan Mackenzie wrote:
> 
> Is there some hook called each time something's about to be displayed on
> the screen (regardless of whether or not font-lock is enabled)?
> 
> 

-- 
View this message in context: http://old.nabble.com/Is-there-something-like-%60on-display-functions%27--tp27339387p27340016.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: Is there something like `on-display-functions'?

[David Kastrup]

Alan Mackenzie <acm@muc.de> writes:

> Hi, Emacs,
>
> Is there some hook called each time something's about to be displayed on
> the screen (regardless of whether or not font-lock is enabled)?
>
> If there is, I could use it to apply the appropriate text properties to
> C++ template delimiters as they're about to be displayed, thus
> potentially speeding up startup for C++ (and like languages).
>
> Yes, I've tried searching for it and not found it.

(info "(elisp) Other display specs")

       You can make any display specification conditional.  To do that,
    package it in another list of the form `(when CONDITION . SPEC)'.  Then
    the specification SPEC applies only when CONDITION evaluates to a
    non-`nil' value.  During the evaluation, `object' is bound to the
    string or buffer having the conditional `display' property.  `position'
    and `buffer-position' are bound to the position within `object' and the
    buffer position where the `display' property was found, respectively.
    Both positions can be different when `object' is a string.

Since CONDITION gets evaluated, you can use it for pretty much anything
you like, even if you decide to let it result in nil always.

That's pretty much butt-ugly but works.  preview-latex uses this to
prioritize its image rendering in order to have on-screen images
rendered first.

One of the rare cases where XEmacs has a nicer API for some job.  This
use of the Emacs API can't be called more than a hack.

Using the Emacs API, in prv-emacs.el:

(defun preview-add-urgentization (fun ov &rest rest)
  "Cause FUN (function call form) to be called when redisplayed.
FUN must be a form with OV as first argument,
REST as the remainder, returning T."
  (let ((dispro (overlay-get ov 'display)))
    (unless (eq (car dispro) 'when)
      (overlay-put ov 'display `(when (,fun ,ov ,@rest)  . ,dispro)))))

(defun preview-remove-urgentization (ov)
  "Undo urgentization of OV by `preview-add-urgentization'.
Returns the old arguments to `preview-add-urgentization'
if there was any urgentization."
  (let ((dispro (overlay-get ov 'display)))
    (when (eq (car-safe dispro) 'when)
      (prog1
	  (car (cdr dispro))
	(overlay-put ov 'display (cdr (cdr dispro)))))))

Using the XEmacs API, in prv-xemacs.el:

(defun preview-add-urgentization (fun ov &rest rest)
  "Cause FUN (function call form) to be called when redisplayed.
FUN must be a form with OV as first argument,
REST as the remainder, returning T.  An alternative is to give
what `preview-remove-urgentization' returns, this will reinstate
the previous state."
  (set-extent-initial-redisplay-function
   ov
   (if (null rest)
       fun
     `(lambda (ov) (,fun ,ov ,@rest)))))

(defun preview-remove-urgentization (ov)
  "Undo urgentization of OV by `preview-add-urgentization'.
Returns the old arguments to `preview-add-urgentization'
if there was any urgentization."
  (prog1 (list (extent-property ov 'initial-redisplay-function) ov)
    (set-extent-initial-redisplay-function ov nil)))


-- 
David Kastrup

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, Lennart,

On Wed, Jan 27, 2010 at 02:53:35PM +0100, Lennart Borgman wrote:
> On Wed, Jan 27, 2010 at 2:57 PM, Alan Mackenzie <acm@muc.de> wrote:

> > Is there some hook called each time something's about to be displayed
> > on the screen (regardless of whether or not font-lock is enabled)?

> > If there is, I could use it to apply the appropriate text properties
> > to C++ template delimiters as they're about to be displayed, thus
> > potentially speeding up startup for C++ (and like languages).

> > Yes, I've tried searching for it and not found it.


> Just curious, but why don't you want to use font-lock for it?

Because font-lock is merely a user option which not every user likes,
even though most do.  There is, I believe, at least one Emacs developer
who runs Emacs without it.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Davis Herring]

> Is there some hook called each time something's about to be displayed on
> the screen (regardless of whether or not font-lock is enabled)?

The closest I know is the pair `window-scroll-functions' and
`window-size-change-functions'.  But you'd have to also consider text
brought into the window because of text deletions, so you'd have to use
`after-change-functions' too and it's getting quite ugly at that point.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

>> Is there some hook called each time something's about to be displayed on
>> the screen (regardless of whether or not font-lock is enabled)?

There's fontification-functions, which is the hook around which jit-lock
is implemented.  

I'd recommend you use jit-lock instead (via jit-lock-register), tho,
because fontification-functions (despite its name) really only works
well with a single function (at least I don't know how to make it work
well with more than one, based on how it's currently defined).

>> If there is, I could use it to apply the appropriate text properties to
>> C++ template delimiters as they're about to be displayed, thus
>> potentially speeding up startup for C++ (and like languages).

Of course, if these are syntax-table properties, it will not solve all
problems, since the buffer-movement may still get buggy when it depends
on text that hasn't been displayed yet.

>> Yes, I've tried searching for it and not found it.
> Just curious, but why don't you want to use font-lock for it?

He probably wants it to work even when font-lock is disabled.


        Stefan

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, David,

On Wed, Jan 27, 2010 at 03:27:16PM +0100, David Kastrup wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > Is there some hook called each time something's about to be displayed
> > on the screen (regardless of whether or not font-lock is enabled)?

> > If there is, I could use it to apply the appropriate text properties
> > to C++ template delimiters as they're about to be displayed, thus
> > potentially speeding up startup for C++ (and like languages).

> > Yes, I've tried searching for it and not found it.

> (info "(elisp) Other display specs")

>        You can make any display specification conditional.  To do that,
>     package it in another list of the form `(when CONDITION . SPEC)'.  Then
>     the specification SPEC applies only when CONDITION evaluates to a
>     non-`nil' value.  During the evaluation, `object' is bound to the
>     string or buffer having the conditional `display' property.  `position'
>     and `buffer-position' are bound to the position within `object' and the
>     buffer position where the `display' property was found, respectively.
>     Both positions can be different when `object' is a string.

> Since CONDITION gets evaluated, you can use it for pretty much anything
> you like, even if you decide to let it result in nil always.

> That's pretty much butt-ugly but works.  preview-latex uses this to
> prioritize its image rendering in order to have on-screen images
> rendered first.

Yep, that's utterly ghastly.  ;-)  Nevertheless, thanks muchly!  Thanks
also for the XEmacs version (how did you guess I wanted that too?).

> -- 
> David Kastrup

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, Stefan,

On Wed, Jan 27, 2010 at 10:11:27AM -0500, Stefan Monnier wrote:
> >> Is there some hook called each time something's about to be
> >> displayed on the screen (regardless of whether or not font-lock is
> >> enabled)?

> There's fontification-functions, which is the hook around which
> jit-lock is implemented.  

Thanks for this tip.

The documentation (elisp manual) and doc string for this are poor.  They
only say what you should use the hook for, not when it's called.  In
particular, they don't say explicitly what happens when font lock isn't
enabled, or when a different font-lock-support-mode is active.  I take it
the hook is called regardless of these things.

> I'd recommend you use jit-lock instead (via jit-lock-register), tho,
> because fontification-functions (despite its name) really only works
> well with a single function (at least I don't know how to make it work
> well with more than one, based on how it's currently defined).

Is that because it's got to set 'fontified' properties?

> >> If there is, I could use it to apply the appropriate text properties
> >> to C++ template delimiters as they're about to be displayed, thus
> >> potentially speeding up startup for C++ (and like languages).

> Of course, if these are syntax-table properties, it will not solve all
> problems, since the buffer-movement may still get buggy when it depends
> on text that hasn't been displayed yet.

No, that's OK.  C++'s template delimiters < > cannot enclose a brace or
semicolon, so their effect is local.  I would set these properties on
complete "blocks" (regions delimited by {, }, ;).

> >> Yes, I've tried searching for it and not found it.
> > Just curious, but why don't you want to use font-lock for it?

> He probably wants it to work even when font-lock is disabled.

Indeed.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Stephen J. Turnbull]

David Kastrup writes:

 > One of the rare cases where XEmacs has a nicer API for some job.  This
 > use of the Emacs API can't be called more than a hack.

Ouch.  IIRC, what we call `set-extent-initial-redisplay-function' is
"kludge".  I guess I should go fix the documentation so Ben won't
remove it in a fit of code cleanliness mania. ;-)

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> Date: Wed, 27 Jan 2010 15:37:33 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: Lennart Borgman <lennart.borgman@gmail.com>, emacs-devel@gnu.org
> 
> > I'd recommend you use jit-lock instead (via jit-lock-register), tho,
> > because fontification-functions (despite its name) really only works
> > well with a single function (at least I don't know how to make it work
> > well with more than one, based on how it's currently defined).
> 
> Is that because it's got to set 'fontified' properties?

No, I don't think so.  The handler of the `fontified' property checks
for the property's value being nil only once, and then runs all the
functions in fontification-functions in a loop.  The relevant code is
below.

Stefan, could you perhaps show a reproducible test case for this?

 ----------------------------------------------------------------------
  /* Get the value of the `fontified' property at IT's current buffer
     position.  (The `fontified' property doesn't have a special
     meaning in strings.)  If the value is nil, call functions from
     Qfontification_functions.  */
  if (!STRINGP (it->string)
      && it->s == NULL
      && !NILP (Vfontification_functions)
      && !NILP (Vrun_hooks)
      && (pos = make_number (IT_CHARPOS (*it)),
	  prop = Fget_char_property (pos, Qfontified, Qnil),
	  /* Ignore the special cased nil value always present at EOB since
	     no amount of fontifying will be able to change it.  */
	  NILP (prop) && IT_CHARPOS (*it) < Z))
    {
      int count = SPECPDL_INDEX ();
      Lisp_Object val;

      val = Vfontification_functions;
      specbind (Qfontification_functions, Qnil);

      if (!CONSP (val) || EQ (XCAR (val), Qlambda))
	safe_call1 (val, pos);
      else
	{
	  Lisp_Object globals, fn;
	  struct gcpro gcpro1, gcpro2;

	  globals = Qnil;
	  GCPRO2 (val, globals);

	  for (; CONSP (val); val = XCDR (val))
	    {
	      fn = XCAR (val);

	      if (EQ (fn, Qt))
		{
		  /* A value of t indicates this hook has a local
		     binding; it means to run the global binding too.
		     In a global value, t should not occur.  If it
		     does, we must ignore it to avoid an endless
		     loop.  */
		  for (globals = Fdefault_value (Qfontification_functions);
		       CONSP (globals);
		       globals = XCDR (globals))
		    {
		      fn = XCAR (globals);
		      if (!EQ (fn, Qt))
			safe_call1 (fn, pos);
		    }
		}
	      else
		safe_call1 (fn, pos);
	    }

	  UNGCPRO;
	}

      unbind_to (count, Qnil);

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> Date: Wed, 27 Jan 2010 15:37:33 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: Lennart Borgman <lennart.borgman@gmail.com>, emacs-devel@gnu.org
> 
> On Wed, Jan 27, 2010 at 10:11:27AM -0500, Stefan Monnier wrote:
> > >> Is there some hook called each time something's about to be
> > >> displayed on the screen (regardless of whether or not font-lock is
> > >> enabled)?
> 
> > There's fontification-functions, which is the hook around which
> > jit-lock is implemented.  
> 
> Thanks for this tip.
> 
> The documentation (elisp manual) and doc string for this are poor.  They
> only say what you should use the hook for, not when it's called.  In
> particular, they don't say explicitly what happens when font lock isn't
> enabled, or when a different font-lock-support-mode is active.  I take it
> the hook is called regardless of these things.

Yes, it's called regardless.  It's actually part of the normal
redisplay operation, whereby Emacs traverses the visible portion of
every buffer that is displayed in some window, and does whatever is
necessary to redisplay it.  During this traverse, the display code
stops at each buffer position which has some change in text properties
or overlays, and calls the handler appropriate for whatever that
buffer position needs.  (Well, actually it calls all the possible
handlers, and those which have nothing to do simply return
immediately.)  The handler for the `fontified' property is the one
that runs from fontification-functions, if that value's property at
current position is nil.  The only other setting that stops Emacs from
running fontification-functions is that run-hooks is nil (not sure why
this is so, but the code is quite explicit).

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

>> > I'd recommend you use jit-lock instead (via jit-lock-register), tho,
>> > because fontification-functions (despite its name) really only works
>> > well with a single function (at least I don't know how to make it work
>> > well with more than one, based on how it's currently defined).
>> Is that because it's got to set 'fontified' properties?

Yes.

> No, I don't think so.  The handler of the `fontified' property checks
> for the property's value being nil only once, and then runs all the
> functions in fontification-functions in a loop.  The relevant code is
> below.

That's not the problem.  The problem is to figure out which of the
functions should set this property and over which part of the buffer,
since that property should only be applied to the part of the buffer
that's fontified by all the functions (if the first functions fontifies
256 chars at a time and the second 1000 chars at a time, the property
should only be set on the first 256 bytes).


        Stefan

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Alan Mackenzie <acm@muc.de>,  emacs-devel@gnu.org
> Date: Wed, 27 Jan 2010 14:24:18 -0500
> 
> >> > I'd recommend you use jit-lock instead (via jit-lock-register), tho,
> >> > because fontification-functions (despite its name) really only works
> >> > well with a single function (at least I don't know how to make it work
> >> > well with more than one, based on how it's currently defined).
> >> Is that because it's got to set 'fontified' properties?
> 
> Yes.

I see that I misunderstood what Alan was asking, and answered the
wrong question.  Sorry.

> The problem is to figure out which of the functions should set this
> property

As you see from the fragment I posted, the functions are invoked in
strict order of the list.  So I guess the last one should do this?

> and over which part of the buffer, since that property
> should only be applied to the part of the buffer that's fontified by
> all the functions (if the first functions fontifies 256 chars at a
> time and the second 1000 chars at a time, the property should only
> be set on the first 256 bytes).

Sorry, I still don't get this: why do you need several functions that
fontify different stretches of text, and why only the first stretch
should be marked as fontified.  Could you please tell some more about
the context?

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

>> and over which part of the buffer, since that property
>> should only be applied to the part of the buffer that's fontified by
>> all the functions (if the first functions fontifies 256 chars at a
>> time and the second 1000 chars at a time, the property should only
>> be set on the first 256 bytes).

> Sorry, I still don't get this: why do you need several functions that
> fontify different stretches of text,

Presumably if you have several functions on fontification-functions,
it's because you need several functions that "fontify" the same text
(e.g. font-lock on the one hand and Alan's hypothetical code on the other).

> and why only the first stretch should be marked as fontified.

Unless the two functions cooperate, there's no reason to presume that
the two functions will fontify the same amount of text, since the amount
is mostly arbitrarily chosen by the function themselves (and may need
rounding up because of line-granularity or other
foo-region-extend-function).

So if each function is written independently, it will (in general) not
fontify the same amount of text as the others, and it won't know how
much the others (if any) have fontified.  So none of the functions will
actually know on which portion of the text the `fontified' property
should be applied.


        Stefan

Re: Is there something like `on-display-functions'?

[Daniel Colascione]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 1/27/10 8:57 AM, Alan Mackenzie wrote:
> Is there some hook called each time something's about to be displayed on
> the screen (regardless of whether or not font-lock is enabled)?
> 
> If there is, I could use it to apply the appropriate text properties to
> C++ template delimiters as they're about to be displayed, thus
> potentially speeding up startup for C++ (and like languages).

Wouldn't that do the wrong thing if, say, the first half of a template
argument list is at the bottom of the visible portion of the buffer, and
I want to jump to its end?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (Darwin)

iEYEARECAAYFAktg60cACgkQ17c2LVA10VvRXQCeItXUcp4pn+zu/wm/+YGBIfPZ
XpsAoMewsQP9ec9jD1aFcJsOqC+r6Vk4
=iy+z
-----END PGP SIGNATURE-----

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: acm@muc.de,  emacs-devel@gnu.org
> Date: Wed, 27 Jan 2010 16:04:07 -0500
> 
> Unless the two functions cooperate, there's no reason to presume that
> the two functions will fontify the same amount of text, since the amount
> is mostly arbitrarily chosen by the function themselves (and may need
> rounding up because of line-granularity or other
> foo-region-extend-function).
> 
> So if each function is written independently, it will (in general) not
> fontify the same amount of text as the others, and it won't know how
> much the others (if any) have fontified.  So none of the functions will
> actually know on which portion of the text the `fontified' property
> should be applied.

The place where `fontified' property with a nil value is left after
all fontification-functions are run is not very important.  It's just
the place where redisplay will stop next and call those functions
again.  If a function on that list needs to be sure it does not miss
any text due to the position where a nil-valued `fontified' is placed,
it should simply start from some earlier position, not from the one
with which it is invoked.  Then it could search for the first position
where it needs to do its job, and resume there.

Note that the JIT Lock function that gets invoked via
fontification-functions always starts from the beginning of the line
to which the position it was invoked with belongs.  So it is already
doing something similar, albeit for different reasons.

OTOH, it would be a simple change to have the loop in
handle_fontified_prop maintain the lowest buffer position of those
where the `fontified' property was placed by any of the functions it
invokes, and reset the value of that property to nil for that position
after the loop finishes.  Would that resolve the difficulty?

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: acm@muc.de,  emacs-devel@gnu.org
> Date: Wed, 27 Jan 2010 16:04:07 -0500
> 
> > Sorry, I still don't get this: why do you need several functions that
> > fontify different stretches of text,
> 
> Presumably if you have several functions on fontification-functions,
> it's because you need several functions that "fontify" the same text
> (e.g. font-lock on the one hand and Alan's hypothetical code on the other).

Btw, it would not be too difficult to add a hook like what Alan
wanted, I think.  Perhaps Alan could post a list of requirements for
such a feature (as I know close to nothing about CC Mode's internals,
it is not entirely clear to me what is needed), and we could then
generalize it.

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, Daniel!

On Wed, Jan 27, 2010 at 08:41:27PM -0500, Daniel Colascione wrote:
> On 1/27/10 8:57 AM, Alan Mackenzie wrote:
> > Is there some hook called each time something's about to be displayed
> > on the screen (regardless of whether or not font-lock is enabled)?

> > If there is, I could use it to apply the appropriate text properties
> > to C++ template delimiters as they're about to be displayed, thus
> > potentially speeding up startup for C++ (and like languages).

> Wouldn't that do the wrong thing if, say, the first half of a template
> argument list is at the bottom of the visible portion of the buffer,
> and I want to jump to its end?

No.  :-)  I'm intending to mark the template delimiters ONLY in pairs
from now on.  This is due to difficulties with the current approach,
which is to mark an opening < when it looks like a template opener.

It is rarely (?never) necessary to look very far for a matching >, since
a <..> pair can never enclose a ;, {, or } [*].  The "never" might happen
when you type a < shortly before an infinite number of #define lines.
But C++ hackers don't really use #define, do they?  ;-)

[*] Any rough estimate of how large the gap between < and > gets in real
C++ programs would be welcome.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

On Wed, Jan 27, 2010 at 07:55:29PM +0200, Eli Zaretskii wrote:

> Yes, it[fontification_functions]'s called regardless.  It's actually
> part of the normal redisplay operation, whereby ....

Thanks!  That was a very clear explanation.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, Eli,

On Thu, Jan 28, 2010 at 01:55:13AM -0500, Eli Zaretskii wrote:
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: acm@muc.de,  emacs-devel@gnu.org
> > Date: Wed, 27 Jan 2010 16:04:07 -0500

> Btw, it would not be too difficult to add a hook like what Alan
> wanted, I think.  Perhaps Alan could post a list of requirements for
> such a feature (as I know close to nothing about CC Mode's internals,
> it is not entirely clear to me what is needed), and we could then
> generalize it.

As a zero order approximation, something like
`before-display-functions', where an element would look like this:

   (defun foo-before-display (buffer beg end) .....)

, BEG and END defining the piece of BUFFER about to be displayed.  The
hook would be called before fontification.  (Is it possible in general
to determine BEG and END before fontification?).

-- 
Alan Mackenzie (Nuremberg, Germany).

Doc patch for `fontification-functions': [was: Is there something like `on-display-functions'?]

[Alan Mackenzie]

Hi, Stefan,

On Wed, Jan 27, 2010 at 03:37:33PM +0000, Alan Mackenzie wrote:

> The documentation (elisp manual) and doc string for this are poor.
> They only say what you should use the hook for, not when it's called.
> In particular, they don't say explicitly what happens when font lock
> isn't enabled, or when a different font-lock-support-mode is active.
> I take it the hook is called regardless of these things.

Here's a patch to fix this.  I've also converted some "each function"s
to "the functions [collectively]".  May I commit the patch (with a
proper ChangeLog entry, of course)?



=== modified file 'doc/lispref/display.texi'
*** doc/lispref/display.texi	2010-01-13 08:35:10 +0000
--- doc/lispref/display.texi	2010-01-28 11:22:57 +0000
***************
*** 2654,2676 ****
  
  @defvar fontification-functions
  This variable holds a list of functions that are called by Emacs
! redisplay as needed to assign faces automatically to text in the buffer.
  
  The functions are called in the order listed, with one argument, a
! buffer position @var{pos}.  Each function should attempt to assign faces
! to the text in the current buffer starting at @var{pos}.
  
! Each function should record the faces they assign by setting the
! @code{face} property.  It should also add a non-@code{nil}
! @code{fontified} property for all the text it has assigned faces to.
  That property tells redisplay that faces have been assigned to that text
  already.
  
! It is probably a good idea for each function to do nothing if the
  character after @var{pos} already has a non-@code{nil} @code{fontified}
  property, but this is not required.  If one function overrides the
! assignments made by a previous one, the properties as they are
! after the last function finishes are the ones that really matter.
  
  For efficiency, we recommend writing these functions so that they
  usually assign faces to around 400 to 600 characters at each call.
--- 2654,2678 ----
  
  @defvar fontification-functions
  This variable holds a list of functions that are called by Emacs
! redisplay as needed, just before doing redisplay.  They are called even
! when Font-lock isn't enabled.  When Font-lock is enabled, this variable
! usually holds just one function, @code{jit-lock-function}.
  
  The functions are called in the order listed, with one argument, a
! buffer position @var{pos}.  Collectively they should attempt to assign
! faces to the text in the current buffer starting at @var{pos}.
  
! The functions should record the faces they assign by setting the
! @code{face} property.  They should also add a non-@code{nil}
! @code{fontified} property for all the text they have assigned faces to.
  That property tells redisplay that faces have been assigned to that text
  already.
  
! It is probably a good idea for the functions to do nothing if the
  character after @var{pos} already has a non-@code{nil} @code{fontified}
  property, but this is not required.  If one function overrides the
! assignments made by a previous one, the properties as they are after the
! last function finishes are the ones that really matter.
  
  For efficiency, we recommend writing these functions so that they
  usually assign faces to around 400 to 600 characters at each call.



-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> Date: Thu, 28 Jan 2010 10:38:56 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> As a zero order approximation, something like
> `before-display-functions', where an element would look like this:
> 
>    (defun foo-before-display (buffer beg end) .....)
> 
> , BEG and END defining the piece of BUFFER about to be displayed.  The
> hook would be called before fontification.

What do you mean by ``before fontification'', and why does it have to
be before that?

> (Is it possible in general to determine BEG and END before
> fontification?).

Only approximately.  E.g., imagine a `display' property whose value is
a long string -- without computing how it will be displayed, there's
no way of knowing how much of the screen estate it will take.  As
another, less exotic example, imagine font-lock definitions that
enlarge the font significantly: until you fontify that part, you
cannot know how much screen space it will take.

BEG and END are well known _after_ redisplay prepares what will be
shown on the glass, though.  Therefore, please tell why do you ask to
call this hook _before_ fontifications.

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, Eli,

On Thu, Jan 28, 2010 at 07:54:34AM -0500, Eli Zaretskii wrote:
> > Date: Thu, 28 Jan 2010 10:38:56 +0000
> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > As a zero order approximation, something like
> > `before-display-functions', where an element would look like this:

> >    (defun foo-before-display (buffer beg end) .....)

> > , BEG and END defining the piece of BUFFER about to be displayed.
> > The hook would be called before fontification.

> What do you mean by ``before fontification'', and why does it have to
> be before that?

I don't really know what I mean by "before fontification".  I want it so
that if in `foo-before-display' I put syntax-table TP on < and >, and
point happens to be at one of them, show-paren-mode will work to
highlight them.

> > (Is it possible in general to determine BEG and END before
> > fontification?).

> Only approximately.  E.g., imagine a `display' property whose value is
> a long string -- without computing how it will be displayed, there's
> no way of knowing how much of the screen estate it will take.  As
> another, less exotic example, imagine font-lock definitions that
> enlarge the font significantly: until you fontify that part, you
> cannot know how much screen space it will take.

> BEG and END are well known _after_ redisplay prepares what will be
> shown on the glass, though.  Therefore, please tell why do you ask to
> call this hook _before_ fontifications.

See above.

Maybe an alternative to a hook here would be to allow an application to
specify that some text property (say, 'c-templatified) is to be
"special", and when the displayer detects the TP has changed, it calls an
associated function.  'c-templatified would be analogous to 'fontified.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Doc patch for `fontification-functions':

[Chong Yidong]

Alan Mackenzie <acm@muc.de> writes:

> Here's a patch to fix this.  I've also converted some "each function"s
> to "the functions [collectively]".  May I commit the patch (with a
> proper ChangeLog entry, of course)?

OK, but

> ! redisplay as needed, just before doing redisplay.  They are called even
> ! when Font-lock isn't enabled.  When Font-lock is enabled, this variable
> ! usually holds just one function, @code{jit-lock-function}.

"Font-lock" should be "Font Lock mode".

> ! @code{face} property.  They should also add a non-@code{nil}
> ! @code{fontified} property for all the text they have assigned faces to.

"for" should be "to".

> ! assignments made by a previous one, the properties as they are after the
> ! last function finishes are the ones that really matter.

You don't need "as they are".

Re: Doc patch for `fontification-functions':

[Alan Mackenzie]

Hi, Yidong,

On Thu, Jan 28, 2010 at 10:34:42AM -0500, Chong Yidong wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > Here's a patch to fix this.  I've also converted some "each function"s
> > to "the functions [collectively]".  May I commit the patch (with a
> > proper ChangeLog entry, of course)?

> OK, but .....

I've commited this, having incorporated all your suggested further
improvements.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Doc patch for `fontification-functions': [was: Is there something like `on-display-functions'?]

[Eli Zaretskii]

> Date: Thu, 28 Jan 2010 11:30:56 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: Lennart Borgman <lennart.borgman@gmail.com>, emacs-devel@gnu.org
> 
> On Wed, Jan 27, 2010 at 03:37:33PM +0000, Alan Mackenzie wrote:
> 
> > The documentation (elisp manual) and doc string for this are poor.
> > They only say what you should use the hook for, not when it's called.
> > In particular, they don't say explicitly what happens when font lock
> > isn't enabled, or when a different font-lock-support-mode is active.
> > I take it the hook is called regardless of these things.
> 
> Here's a patch to fix this.

Thanks.

>   @defvar fontification-functions
>   This variable holds a list of functions that are called by Emacs
> ! redisplay as needed, just before doing redisplay.  They are called even

I suggest to rephrase:

 This variable holds a list of functions that Emacs calls as part of
 doing redisplay.

This (a) avoids passive tense, (b) does not use "redisplay" twice in 2
different senses, and (3) is more accurate, because "just before doing
redisplay" is factually incorrect.

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> Date: Thu, 28 Jan 2010 14:47:07 +0000
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > >    (defun foo-before-display (buffer beg end) .....)
> 
> > > , BEG and END defining the piece of BUFFER about to be displayed.
> > > The hook would be called before fontification.
> 
> > What do you mean by ``before fontification'', and why does it have to
> > be before that?
> 
> I don't really know what I mean by "before fontification".  I want it so
> that if in `foo-before-display' I put syntax-table TP on < and >, and
> point happens to be at one of them, show-paren-mode will work to
> highlight them.

Why do you need to do that only when the text is about to be
displayed?  Is it too expensive to do that in the whole buffer or
something?

Anyway, show-paren-mode works off an idle timer, so I think it is
not necessary to run the kind of function you have in mind before
redisplay; near its end is okay.

> Maybe an alternative to a hook here would be to allow an application to
> specify that some text property (say, 'c-templatified) is to be
> "special", and when the displayer detects the TP has changed, it calls an
> associated function.  'c-templatified would be analogous to 'fontified.

Don't we already have text properties that cause Lisp forms to be
evaluated?

And again, why can't you put your text property everywhere in the
buffer where you want this effect?

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

>> Unless the two functions cooperate, there's no reason to presume that
>> the two functions will fontify the same amount of text, since the amount
>> is mostly arbitrarily chosen by the function themselves (and may need
>> rounding up because of line-granularity or other
>> foo-region-extend-function).
>> 
>> So if each function is written independently, it will (in general) not
>> fontify the same amount of text as the others, and it won't know how
>> much the others (if any) have fontified.  So none of the functions will
>> actually know on which portion of the text the `fontified' property
>> should be applied.
> The place where `fontified' property with a nil value is left after
> all fontification-functions are run is not very important.

On the contrary, it's very important.

> It's just the place where redisplay will stop next and call those
> functions again.

If function foo1 fontified to location 123 and foo2 to location 245 and
the property is set until 245 and the window ends at 200, then foo1 will
not be called again, and the display will show 123-200 without
foo1's fontification.

> If a function on that list needs to be sure it does
> not miss any text due to the position where a nil-valued `fontified'
> is placed, it should simply start from some earlier position, not from
> the one with which it is invoked.  Then it could search for the first
> position where it needs to do its job, and resume there.

Which "earlier position"?  How does the function know how far back it
would need to look?

> Note that the JIT Lock function that gets invoked via
> fontification-functions always starts from the beginning of the line
> to which the position it was invoked with belongs.  So it is already
> doing something similar, albeit for different reasons.

That used to be the case, but now this region-extension has been moved
to font-lock (where it belongs).

> OTOH, it would be a simple change to have the loop in
> handle_fontified_prop maintain the lowest buffer position of those
> where the `fontified' property was placed by any of the functions it
> invokes, and reset the value of that property to nil for that position
> after the loop finishes.  Would that resolve the difficulty?

We could try and solve the problem this way, yes.  Then again, it could
result in serious inefficiency if one of the functions works on 100B
chunks and the other on 1KB chunks, forcing the 1KB chunks to be
recomputed redundantly 10 times.

This said, I don't think it's a serious problem.  The way I look at it,
we should have used fontification-function rather than
fontification-functions, and that's it.

The "multiplexing" of several functions on this hook can be (and is)
performed by jit-lock, where it's easier to be more clever (although
currently jit-lock is not terribly clever either).


        Stefan

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

> As a zero order approximation, something like
> `before-display-functions', where an element would look like this:

>    (defun foo-before-display (buffer beg end) .....)

> , BEG and END defining the piece of BUFFER about to be displayed.  The
> hook would be called before fontification.  (Is it possible in general
> to determine BEG and END before fontification?).

That's exactly what jit-lock-register offers.


        Stefan

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

> It is rarely (?never) necessary to look very far for a matching >, since
> a <..> pair can never enclose a ;, {, or } [*].  The "never" might happen
> when you type a < shortly before an infinite number of #define lines.
> But C++ hackers don't really use #define, do they?  ;-)

Of course, never can also happen in the presence of comments (maybe
strings as well?).


        Stefan

Re: Doc patch for `fontification-functions':

[Stefan Monnier]

>> The documentation (elisp manual) and doc string for this are poor.
>> They only say what you should use the hook for, not when it's called.
>> In particular, they don't say explicitly what happens when font lock
>> isn't enabled, or when a different font-lock-support-mode is active.
>> I take it the hook is called regardless of these things.

> Here's a patch to fix this.  I've also converted some "each function"s
> to "the functions [collectively]".  May I commit the patch (with a
> proper ChangeLog entry, of course)?

If you feel like adding a pointer to jit-lock-register, that would be
great, since it's an interface that's a lot easier to use.


        Stefan

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: acm@muc.de, emacs-devel@gnu.org
> Date: Thu, 28 Jan 2010 14:37:11 -0500
> 
> > If a function on that list needs to be sure it does
> > not miss any text due to the position where a nil-valued `fontified'
> > is placed, it should simply start from some earlier position, not from
> > the one with which it is invoked.  Then it could search for the first
> > position where it needs to do its job, and resume there.
> 
> Which "earlier position"?  How does the function know how far back it
> would need to look?

It could simply record that position in some variable.

> > Note that the JIT Lock function that gets invoked via
> > fontification-functions always starts from the beginning of the line
> > to which the position it was invoked with belongs.  So it is already
> > doing something similar, albeit for different reasons.
> 
> That used to be the case, but now this region-extension has been moved
> to font-lock (where it belongs).

The effect is the same.

> > OTOH, it would be a simple change to have the loop in
> > handle_fontified_prop maintain the lowest buffer position of those
> > where the `fontified' property was placed by any of the functions it
> > invokes, and reset the value of that property to nil for that position
> > after the loop finishes.  Would that resolve the difficulty?
> 
> We could try and solve the problem this way, yes.  Then again, it could
> result in serious inefficiency if one of the functions works on 100B
> chunks and the other on 1KB chunks, forcing the 1KB chunks to be
> recomputed redundantly 10 times.

That would just mean that the 1KB function gets called, detects that
it has nothing to do, and returns.  (Assuming it records its last
chunk in some variable.)

Anyway, in my testing, this inefficiency is not serious.  The bidi
redisplay sometimes needs to traverse the same ``stop point''
(stop_charpos in the iterator structure) several times, first from
left to right, then right to left, etc., because it must do that in
the visual order, and still the display is fast enough to not annoy or
even show a noticeable delay (at least on a machine such as
fencepost)[1].

> This said, I don't think it's a serious problem.  The way I look at it,
> we should have used fontification-function rather than
> fontification-functions, and that's it.
> 
> The "multiplexing" of several functions on this hook can be (and is)
> performed by jit-lock, where it's easier to be more clever (although
> currently jit-lock is not terribly clever either).

Fine with me.  But I do think we may wish to have some more general
redisplay-hook feature, that isn't tailored to fontification.

----------------------
[1] As usual, our mostly wrong impression about the speed of our our
machines stems from the fact that most of the time we see them being
busy moving pixels from here to there.  When time comes to exercise
raw computing power, modern machines are lightning fast.

A case in point is the bidi reordering code.  Its processing core is
no less than 1500 lines of C, which replace these two lines in the
current unidirectional display engine:

	  IT_BYTEPOS (*it) += it->len;
	  IT_CHARPOS (*it) += 1;

(For those who are not familiar with the display engine, this advances
the iterator by one character and the corresponding number of bytes,
as determined by the length of its multibyte sequence.)

IOW, instead of two increments, we run a 1500-line monster.  Mind you,
this runs in the innermost loop of the display engine!  Would you be
surprised to know that I got scared when I finished implementing the
reordering code?  I was afraid that the resulting redisplay will
crawl.  So I timed the code, by writing a trivial wc workalike, and
timing it against the real wc.  To my surprise it worked at half the
speed of wc, which is pretty darn fast, concerning what it needs to
do instead of just incrementing a pointer.

Morale: never underestimate what a modern CPU can do when it does not
need to move pixels ;-)

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

>> > If a function on that list needs to be sure it does
>> > not miss any text due to the position where a nil-valued `fontified'
>> > is placed, it should simply start from some earlier position, not from
>> > the one with which it is invoked.  Then it could search for the first
>> > position where it needs to do its job, and resume there.
>> Which "earlier position"?  How does the function know how far back it
>> would need to look?
> It could simply record that position in some variable.

If the buffer is shown in two windows, that "last position" might be
megabytes away.

>> > Note that the JIT Lock function that gets invoked via
>> > fontification-functions always starts from the beginning of the line
>> > to which the position it was invoked with belongs.  So it is already
>> > doing something similar, albeit for different reasons.
>> That used to be the case, but now this region-extension has been moved
>> to font-lock (where it belongs).
> The effect is the same.

Not quite: there are other users of jit-lock than font-lock (mostly
glasses-mode, but Alan might become the third user, linum could be
another user, ...).

>> > OTOH, it would be a simple change to have the loop in
>> > handle_fontified_prop maintain the lowest buffer position of those
>> > where the `fontified' property was placed by any of the functions it
>> > invokes, and reset the value of that property to nil for that position
>> > after the loop finishes.  Would that resolve the difficulty?
>> 
>> We could try and solve the problem this way, yes.  Then again, it could
>> result in serious inefficiency if one of the functions works on 100B
>> chunks and the other on 1KB chunks, forcing the 1KB chunks to be
>> recomputed redundantly 10 times.

> That would just mean that the 1KB function gets called, detects that
> it has nothing to do, and returns.  (Assuming it records its last
> chunk in some variable.)

This assumption is currently not true.  Recording which part is already
handled and which part is not yet handled is usually done either by the
`fontified' property (if you use fontification-functions) or somehow by
`jit-lock'.

If such a problem of efficiency needs to be solved, we could indeed
probably hack it up via some such variable, tho it would be mighty ugly,
since the function wouldn't necessarily know what happened between the
last time it was called and the current time, so it would need to
somehow figure out whether "nothing happened and I can just return" or
whether in the contrary the buffer has been modified and the
fontification that was computed last time needs to be refreshed.

All doable, but ugly and messy, duplicating what jit-lock does.

> Fine with me.  But I do think we may wish to have some more general
> redisplay-hook feature, that isn't tailored to fontification.

Not sure what you're thinking of.  I'd be interested in
a before-redisplay-hook, admittedly, which would be an alternative to
post-command-hook: several post-command-hooks would actually also like
to be run at times where no command was performed (e.g. timer event,
filter, sit-for, ...).  I'm thinking of things like reveal-mode, or
changing the cursor color depending on context.
Anything else in your mind?

> ----------------------
> [1] As usual, our mostly wrong impression about the speed of our our
> machines stems from the fact that most of the time we see them being
> busy moving pixels from here to there.  When time comes to exercise
> raw computing power, modern machines are lightning fast.

> A case in point is the bidi reordering code.  Its processing core is
> no less than 1500 lines of C, which replace these two lines in the
> current unidirectional display engine:

> 	  IT_BYTEPOS (*it) += it->len;
> 	  IT_CHARPOS (*it) += 1;

> (For those who are not familiar with the display engine, this advances
> the iterator by one character and the corresponding number of bytes,
> as determined by the length of its multibyte sequence.)

> IOW, instead of two increments, we run a 1500-line monster.  Mind you,
> this runs in the innermost loop of the display engine!  Would you be
> surprised to know that I got scared when I finished implementing the
> reordering code?  I was afraid that the resulting redisplay will
> crawl.  So I timed the code, by writing a trivial wc workalike, and
> timing it against the real wc.  To my surprise it worked at half the
> speed of wc, which is pretty darn fast, concerning what it needs to
> do instead of just incrementing a pointer.

> Morale: never underestimate what a modern CPU can do when it does not
> need to move pixels ;-)

Amen!


        Stefan

Re: Is there something like `on-display-functions'?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: acm@muc.de, emacs-devel@gnu.org
> Date: Thu, 28 Jan 2010 18:12:11 -0500
> 
> >> > If a function on that list needs to be sure it does
> >> > not miss any text due to the position where a nil-valued `fontified'
> >> > is placed, it should simply start from some earlier position, not from
> >> > the one with which it is invoked.  Then it could search for the first
> >> > position where it needs to do its job, and resume there.
> >> Which "earlier position"?  How does the function know how far back it
> >> would need to look?
> > It could simply record that position in some variable.
> 
> If the buffer is shown in two windows, that "last position" might be
> megabytes away.

It doesn't matter, because the redisplay works on windows one by one,
and the iterator structure is initialized anew for each one of them.

> >> > Note that the JIT Lock function that gets invoked via
> >> > fontification-functions always starts from the beginning of the line
> >> > to which the position it was invoked with belongs.  So it is already
> >> > doing something similar, albeit for different reasons.
> >> That used to be the case, but now this region-extension has been moved
> >> to font-lock (where it belongs).
> > The effect is the same.
> 
> Not quite: there are other users of jit-lock than font-lock (mostly
> glasses-mode, but Alan might become the third user, linum could be
> another user, ...).

My point was that the Lisp function which is invoked via this
mechanism can do its work in any position it finds fit, not
necessarily the position passed to it by the display engine.

> >> We could try and solve the problem this way, yes.  Then again, it could
> >> result in serious inefficiency if one of the functions works on 100B
> >> chunks and the other on 1KB chunks, forcing the 1KB chunks to be
> >> recomputed redundantly 10 times.
> 
> > That would just mean that the 1KB function gets called, detects that
> > it has nothing to do, and returns.  (Assuming it records its last
> > chunk in some variable.)
> 
> This assumption is currently not true.  Recording which part is already
> handled and which part is not yet handled is usually done either by the
> `fontified' property (if you use fontification-functions) or somehow by
> `jit-lock'.

That ``somehow'' is what I meant when I mentioned that the function
could record where it left off in some variable.

> If such a problem of efficiency needs to be solved, we could indeed
> probably hack it up via some such variable, tho it would be mighty ugly,
> since the function wouldn't necessarily know what happened between the
> last time it was called and the current time, so it would need to
> somehow figure out whether "nothing happened and I can just return" or
> whether in the contrary the buffer has been modified and the
> fontification that was computed last time needs to be refreshed.

Anything that's needed for such a decision except the position and the
buffer or window id?

> > Fine with me.  But I do think we may wish to have some more general
> > redisplay-hook feature, that isn't tailored to fontification.
> 
> Not sure what you're thinking of.

I was thinking about a hook that does not depend on the `fontified'
property.

> I'd be interested in a before-redisplay-hook

Why ``before''?  There isn't much a Lisp code can do to affect
redisplay, if it is called _by_ redisplay.  I'd say calling the hook
when redisplay is done is more useful.  But I didn't analyze the
possible uses of that, so I may be mistaken.

> admittedly, which would be an alternative to post-command-hook

Redisplay is not always triggered by commands.  Input from a process
is a notable example when it isn't.  Mouse movement is another.
Window size changes are yet another.

> several post-command-hooks would actually also like to be run at
> times where no command was performed (e.g. timer event, filter,
> sit-for, ...).

Exactly.

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, Eli,

On Thu, Jan 28, 2010 at 09:18:05PM +0200, Eli Zaretskii wrote:
> > Date: Thu, 28 Jan 2010 14:47:07 +0000
> > Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > >    (defun foo-before-display (buffer beg end) .....)

> > > > , BEG and END defining the piece of BUFFER about to be displayed.
> > > > The hook would be called before fontification.

> > > What do you mean by ``before fontification'', and why does it have to
> > > be before that?

> > I don't really know what I mean by "before fontification".  I want it
> > so that if in `foo-before-display' I put syntax-table TP on < and >,
> > and point happens to be at one of them, show-paren-mode will work to
> > highlight them.

> Why do you need to do that only when the text is about to be displayed?
> Is it too expensive to do that in the whole buffer or something?

People have complained before about "sluggish start up time" when I've
added things that scan the whole buffer.  Adding 0.5s to startup time
could be serious if you've got 50 big C++ buffers in your .emacs.desktop.
Maybe.  I'm more exploring the possibilities; I might well end up just
scanning the whole buffer - PCs aren't getting any slower as time goes
by.  Currently, the TPs are being added in CC Mode's font locking stuff,
which is kind of irritating (to me at least).

> Anyway, show-paren-mode works off an idle timer, so I think it is
> not necessary to run the kind of function you have in mind before
> redisplay; near its end is okay.

Thanks.  I didn't know that.  Maybe `after-redisplay-functions' is really
what I'm looking for.

> > Maybe an alternative to a hook here would be to allow an application
> > to specify that some text property (say, 'c-templatified) is to be
> > "special", and when the displayer detects the TP has changed, it
> > calls an associated function.  'c-templatified would be analogous to
> > 'fontified.

> Don't we already have text properties that cause Lisp forms to be
> evaluated?

Er, yes.  help-echo, display (as David pointed out).  So it seems I've
got what I need, if I need it.

> And again, why can't you put your text property everywhere in the
> buffer where you want this effect?

I can.  Maybe I will.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

Hi, Stefan,

On Thu, Jan 28, 2010 at 02:37:52PM -0500, Stefan Monnier wrote:
> > As a zero order approximation, something like
> > `before-display-functions', where an element would look like this:

> >    (defun foo-before-display (buffer beg end) .....)

> > , BEG and END defining the piece of BUFFER about to be displayed.  The
> > hook would be called before fontification.  (Is it possible in general
> > to determine BEG and END before fontification?).

> That's exactly what jit-lock-register offers.

jit-lock-register's function only gets called when font lock is enabled,
or at least that's what the fine manual says and the doc string implies.
Yes, I know, I said "before fontification" too, when I really meant
"before redisplay".  This is confusing stuff.  ;-)

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

>> >> > Note that the JIT Lock function that gets invoked via
>> >> > fontification-functions always starts from the beginning of the line
>> >> > to which the position it was invoked with belongs.  So it is already
>> >> > doing something similar, albeit for different reasons.
>> >> That used to be the case, but now this region-extension has been moved
>> >> to font-lock (where it belongs).
>> > The effect is the same.
>> Not quite: there are other users of jit-lock than font-lock (mostly
>> glasses-mode, but Alan might become the third user, linum could be
>> another user, ...).
> My point was that the Lisp function which is invoked via this
> mechanism can do its work in any position it finds fit, not
> necessarily the position passed to it by the display engine.

Right, but for efficiency reason, we'd like to make sure it does its job
at a place that's useful.  With a single fontification-function, the
`fontified' text property provides this information very precisely.

>> This assumption is currently not true.  Recording which part is already
>> handled and which part is not yet handled is usually done either by the
>> `fontified' property (if you use fontification-functions) or somehow by
>> `jit-lock'.
> That ``somehow'' is what I meant when I mentioned that the function
> could record where it left off in some variable.

jit-lock uses the `fontified' text property for that, and then forces
all its clients to fontify (at least) the same part of the buffer, which
is what fontification-functions can't do because it only provides the
`start' but not the `end' argument.

>> If such a problem of efficiency needs to be solved, we could indeed
>> probably hack it up via some such variable, tho it would be mighty ugly,
>> since the function wouldn't necessarily know what happened between the
>> last time it was called and the current time, so it would need to
>> somehow figure out whether "nothing happened and I can just return" or
>> whether in the contrary the buffer has been modified and the
>> fontification that was computed last time needs to be refreshed.
> Anything that's needed for such a decision except the position and the
> buffer or window id?

MODIFF, probably.

>> > Fine with me.  But I do think we may wish to have some more general
>> > redisplay-hook feature, that isn't tailored to fontification.
>> Not sure what you're thinking of.
> I was thinking about a hook that does not depend on the `fontified'
> property.

What would it depend on and what info would it provide to its functions?

>> I'd be interested in a before-redisplay-hook
> Why ``before''?  There isn't much a Lisp code can do to affect
> redisplay, if it is called _by_ redisplay.

Just like fontification-functions, it can affect the faces, the color,
... everything.  See my examples of reveal-mode and cursor color.

> I'd say calling the hook when redisplay is done is more useful.

That's an alternative, indeed, tho I'm not sure what it could be used
for (I do see that it could receive extra info such as which part of
which buffer needed redisply and things like that, but again, no
use-case springs to mind).


        Stefan

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

> jit-lock-register's function only gets called when font lock is enabled,

No: jit-lock has nothing to do with font-lock, except for the fact that
font-lock is its main client (and that historically jit-lock was written
only for font-lock), so it's naturally tuned towards serving font-lock
better than other potential clients.

I.e. jit-lock-register's function will be called regardless of whether
font-lock is enabled or not.

> or at least that's what the fine manual says and the doc string implies.

The docstring doesn't mention font-lock.


        Stefan

Re: Is there something like `on-display-functions'?

[Alan Mackenzie]

HI, Stefan,

On Fri, Jan 29, 2010 at 01:13:42PM -0500, Stefan Monnier wrote:
> > jit-lock-register's function only gets called when font lock is
> > enabled,

> No: jit-lock has nothing to do with font-lock, except for the fact that
> font-lock is its main client (and that historically jit-lock was
> written only for font-lock), so it's naturally tuned towards serving
> font-lock better than other potential clients.

Is this nothing-to-do-with-ness documented anywhere?  In the elisp
manual, jit-lock is found twice, both occurrences in "Other Font Lock
Variables".  Without reading the source code in great detail, you'd be
hard pressed to find out that the jit-lock mechanism is operative when
font-lock mode is disabled.

> I.e. jit-lock-register's function will be called regardless of whether
> font-lock is enabled or not.

> > or at least that's what the fine manual says and the doc string
> > implies.

> The docstring doesn't mention font-lock.

It mentions "fontification".  Isn't that a synonym for font-lock?  Or is
it the part of redisplay that converts characters into glyphs and pixels,
regardless of whether font-lock is enabled?

jit-lock-register says that FUN is "registered as a fontification
function".  I think Somebody (tm) could usefully separate out jit-lock
from font-lock in the doc strings and manual.

Just as a matter of interest, how does jit-lock know the END parameter
which it passes to FUN?  Or is this just a random value, a few hundred
bytes after BEG?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is there something like `on-display-functions'?

[Stefan Monnier]

> Is this nothing-to-do-with-ness documented anywhere?  In the elisp
> manual, jit-lock is found twice, both occurrences in "Other Font Lock
> Variables".  Without reading the source code in great detail, you'd be
> hard pressed to find out that the jit-lock mechanism is operative when
> font-lock mode is disabled.

Yes, that's inherited from jit-lock's origin as a "font-lock support mode".
It deserves to be better documented.

>> The docstring doesn't mention font-lock.

> It mentions "fontification".  Isn't that a synonym for font-lock?  Or is
> it the part of redisplay that converts characters into glyphs and pixels,
> regardless of whether font-lock is enabled?

"fontification" usually refers to the act of adding `face' properties
(and similar things) to enhance the visual appearance of the text.
Font-lock is the canonical package that does fontification, hilit19 was
another one.  Some major modes do/did it without using font-lock.

> jit-lock-register says that FUN is "registered as a fontification
> function".  I think Somebody (tm) could usefully separate out jit-lock
> from font-lock in the doc strings and manual.

Agreed.

> Just as a matter of interest, how does jit-lock know the END parameter
> which it passes to FUN?  Or is this just a random value, a few hundred
> bytes after BEG?

It's a somewhat arbitrary value, computed based on jit-lock-chunk-size.


        Stefan