bug#18856: 24.4; *grep* output buffer not getting fontified when jit-lock-defer-time is used

[Eli Zaretskii <eliz@gnu.org>]

> From: David Engster <deng@randomsample.de>
> Date: Mon, 27 Oct 2014 20:33:48 +0100
> 
> Recipe:
> 
> * emacs -Q
> 
> * M-x set-variable RET jit-lock-defer-time RET 0.05 RET
> 
> * Call M-x rgrep and search for some string in some directory
> 
> You should be able to see that the *grep* buffer capturing grep's output
> is not getting fontified until you hit a key. I would expect that it
> gets fontified automatically when new output arrives.

First, my crystal ball says you omitted something from this recipe,
because if I strictly follow these steps, I cannot even see the
matches produced by Grep, because the shell command inserted by rgrep
into the *grep* buffer is longer than the visible portion of the
window displaying the buffer.  Moreover, rgrep doesn't seem to obey
grep-scroll-output (which will soon be a separate bug report), so you
cannot see the output.

Nevertheless, the problem does exist.  I used the following recipe to
reproduce it:

  emacs -Q
  M-x set-variable RET jit-lock-defer-time RET 0.05 RET
  M-: (setq-default grep-scroll-output t) RET
  M-x grep RET <type a suitable Grep command arguments here> RET

Here's what I saw while digging into this problem:

 . It looks like the idleness state is not reset when we receive input
   from a subprocess.  At least, what I see is that
   jit-lock-deferred-fontify is called only once, prior to any input
   is received from Grep, and never called again, until you press a
   key or send some other input event Emacs's way.

   I can overcome this problem if I add a call to
   internal-timer-start-idle at the end of compilation-handle-exit.
   Not sure this is TRT, though.  If not, what else?

 . But even after the call to internal-timer-start-idle is made when
   Grep exits, and jit-lock-deferred-fontify _is_ called, there's no
   fontification.  Why? because input-pending-p, called by sit-for,
   returns non-nil, and we don't call redisplay.  This happens to me
   on MS-Windows in a GUI session only.  If I put a breakpoint in
   readable_events, here:

  static bool
  readable_events (int flags)
  {
    if (flags & READABLE_EVENTS_DO_TIMERS_NOW)
      timer_check ();

    /* If the buffer contains only FOCUS_IN_EVENT events, and
       READABLE_EVENTS_FILTER_EVENTS is set, report it as empty.  */
    if (kbd_fetch_ptr != kbd_store_ptr)
      {
	if (flags & (READABLE_EVENTS_FILTER_EVENTS
  #ifdef USE_TOOLKIT_SCROLL_BARS
		     | READABLE_EVENTS_IGNORE_SQUEEZABLES
  #endif
		     ))
	  {
	    struct input_event *event;

	    event = ((kbd_fetch_ptr < kbd_buffer + KBD_BUFFER_SIZE)
		     ? kbd_fetch_ptr
		     : kbd_buffer);

	    do
	      {
		if (!(        <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
  #ifdef USE_TOOLKIT_SCROLL_BARS
		      (flags & READABLE_EVENTS_FILTER_EVENTS) &&
  #endif
		      event->kind == FOCUS_IN_EVENT)
  #ifdef USE_TOOLKIT_SCROLL_BARS
		    && !((flags & READABLE_EVENTS_IGNORE_SQUEEZABLES)
			 && (event->kind == SCROLL_BAR_CLICK_EVENT
			     || event->kind == HORIZONTAL_SCROLL_BAR_CLICK_EVENT)
			 && event->part == scroll_bar_handle
			 && event->modifiers == 0)
  #endif
		    )
		  return 1;
		event++;

then I see a single event, BUFFER_SWITCH_EVENT, being reported.
Should input-pending-p ignore such events?

Finally, if I disable blink-cursor-mode, the problem with
input-pending-p doesn't happen, and the only change that is needed to
fix this is a call to internal-timer-start-idle mentioned above.