bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Eli Zaretskii]

To reproduce, in "emacs -Q" evaluate the following:

  (setq jit-lock-stealth-time 16)
  (setq jit-lock-stealth-nice 0.5)
  (setq jit-lock-stealth-verbose t)
  (setq jit-lock-defer-contextually t)
  (setq jit-lock-stealth-load 20)

Then visit buffer.c from the current trunk, and wait for about 20 to
30 sec.  After a while, you should see the JIT Stealth messages
blinking in the echo area, and after some more time, you will see an
error message:

  Error running timer `jit-lock-stealth-fontify': (args-out-of-range #<buffer buffer.c> 71003 71397)

This happens because some code in CC mode that handles fontification
narrows buffer.c such that the accessible region ends at position
71145, inside a call to jit-lock-fontify-now that wants to fontify
between the two positions shown in the error message.  Trying to
access buffer positions beyond 71145 triggers the error.

The C and Lisp backtraces, both from the call to narrow-to-region
which performed the narrowing, and the call to parse-partial-sexp
which signals the error, are below.  AFAICS, the narrowing is done in
this snippet from c-find-decl-spots:

	       (if (zerop cfd-macro-end)
		   t
		 (if (> cfd-macro-end (point))
		     (progn (narrow-to-region (point-min) cfd-macro-end)
			    t)
		   ;; The matched token was the last thing in the macro,
		   ;; so the whole match is bogus.
		   (setq cfd-macro-end 0)
		   nil))))

The function then proceeds to call cfd-fun, which, several levels
below, calls parse-partial-sexp that barfs.

I don't know enough about this stuff to decide whether its OK to
constrain cfd-match-pos to not exceed cfd-macro-end, or maybe extend
cfd-macro-end to include cfd-match-pos (or something else).

Curiously, I cannot reproduce this on the trunk, only with Emacs 24.4,
but if that means the problem was somehow solved on the trunk, it
would be nice to have that solution on the branch as well.

Here's the backtrace from the call to narrow-to-region:

  Fnarrow_to_region (start=4, end=284580) at editfns.c:3290
  3290      if (PT < XFASTINT (start))
  (gdb) bt 20
  #0  Fnarrow_to_region (start=4, end=284580) at editfns.c:3290
  #1  0x01184efd in eval_sub (form=87559934) at eval.c:2185
  #2  0x01180aca in Fprogn (body=87559902) at eval.c:462
  #3  0x01184c42 in eval_sub (form=87559942) at eval.c:2132
  #4  0x01180854 in Fif (args=87559950) at eval.c:413
  #5  0x01184c42 in eval_sub (form=87559990) at eval.c:2132
  #6  0x01180aca in Fprogn (body=87559822) at eval.c:462
  #7  0x01180912 in Fif (args=87560006) at eval.c:414
  #8  0x01184c42 in eval_sub (form=87560030) at eval.c:2132
  #9  0x01180aca in Fprogn (body=87559806) at eval.c:462
  #10 0x01184c42 in eval_sub (form=87616222) at eval.c:2132
  #11 0x0118074b in Fand (args=87616038) at eval.c:387
  #12 0x01184c42 in eval_sub (form=87616054) at eval.c:2132
  #13 0x011807fa in Fif (args=87616022) at eval.c:409
  #14 0x01184c42 in eval_sub (form=87616030) at eval.c:2132
  #15 0x01180aca in Fprogn (body=87615774) at eval.c:462
  #16 0x01182360 in Fwhile (args=87616726) at eval.c:992
  #17 0x01184c42 in eval_sub (form=87616734) at eval.c:2132
  #18 0x01180aca in Fprogn (body=87613150) at eval.c:462
  #19 0x01182262 in Flet (args=87613142) at eval.c:970
  (More stack frames follow...)

  Lisp Backtrace:
  "narrow-to-region" (0x88be0c)
  "progn" (0x88bf70)
  "if" (0x88c090)
  "if" (0x88c1e0)
  "progn" (0x88c300)
  "and" (0x88c420)
  "if" (0x88c540)
  "while" (0x88c6a0)
  "let" (0x88c880)
  "c-find-decl-spots" (0x88c980)
  "let" (0x88cc80)
  "save-restriction" (0x88cdd0)
  "c-font-lock-declarations" (0x88cf68)
  "font-lock-fontify-keywords-region" (0x88d298)
  "font-lock-default-fontify-region" (0x88d5c4)
  "funcall" (0x88d5c0)
  "let" (0x88d820)
  "c-font-lock-fontify-region" (0x88d9b8)
  "font-lock-fontify-region" (0x88de0c)
  "run-hook-with-args" (0x88de08)
  0x52e3590 PVEC_COMPILED
  "funcall" (0x88e130)
  "jit-lock-fontify-now" (0x88e638)
  "jit-lock-stealth-fontify" (0x88ea5c)
  "apply" (0x88ea58)
  "byte-code" (0x88ecec)
  "timer-event-handler" (0x88f15c)
  (gdb) up
  #1  0x01184efd in eval_sub (form=87559934) at eval.c:2185
  2185                  val = (XSUBR (fun)->function.a2 (argvals[0], argvals[1]));
  (gdb) p form
  $14 = 87559934
  (gdb) xtype
  Lisp_Cons
  (gdb) pp form
  (narrow-to-region (point-min) cfd-macro-end)

And here's the backtrace from the call to parse-partial-sexp:

  #0  args_out_of_range_3 (a1=63625221, a2=284012, a3=285588) at data.c:223
  #1  0x0111ec11 in validate_region (b=0x4556c, e=0x45b94) at buffer.c:2268
  #2  0x011c3f1c in Fparse_partial_sexp (from=284012, to=285588, targetdepth=0,
      stopbefore=58931234, oldstate=88488678, commentstop=58931234)
      at syntax.c:3460
  #3  0x01185025 in eval_sub (form=59559654) at eval.c:2201
  #4  0x01180d26 in Fsetq (args=59559662) at eval.c:539
  #5  0x01184c42 in eval_sub (form=59559670) at eval.c:2132
  #6  0x01180aca in Fprogn (body=59559486) at eval.c:462
  #7  0x01180912 in Fif (args=59559774) at eval.c:414
  #8  0x01184c42 in eval_sub (form=59559814) at eval.c:2132
  #9  0x01180854 in Fif (args=59559822) at eval.c:413
  #10 0x01184c42 in eval_sub (form=59559878) at eval.c:2132
  #11 0x01180aca in Fprogn (body=59559190) at eval.c:462
  #12 0x01180a17 in Fcond (args=59698630) at eval.c:440
  #13 0x01184c42 in eval_sub (form=59697590) at eval.c:2132
  #14 0x01180aca in Fprogn (body=59697486) at eval.c:462
  #15 0x01184c42 in eval_sub (form=59697582) at eval.c:2132
  #16 0x0118074b in Fand (args=59697462) at eval.c:387
  #17 0x01184c42 in eval_sub (form=59697478) at eval.c:2132
  #18 0x0118236b in Fwhile (args=59697262) at eval.c:989
  #19 0x01184c42 in eval_sub (form=59697270) at eval.c:2132
  #20 0x011830ac in internal_lisp_condition_case (var=60678690,
      bodyform=59697270, handlers=59703038) at eval.c:1317
  #21 0x01182b94 in Fcondition_case (args=59697190) at eval.c:1245
  #22 0x01184c42 in eval_sub (form=59697182) at eval.c:2132
  #23 0x01180aca in Fprogn (body=59697174) at eval.c:462
  #24 0x01182262 in Flet (args=59696854) at eval.c:970
  #25 0x01184c42 in eval_sub (form=59696846) at eval.c:2132
  #26 0x01180aca in Fprogn (body=59696782) at eval.c:462
  #27 0x01186dd7 in funcall_lambda (fun=59696822, nargs=4, arg_vector=0x88a6d0)
      at eval.c:3038
  #28 0x011867a9 in apply_lambda (fun=59696822, args=86996902, count=155)
      at eval.c:2920
  #29 0x011852d2 in eval_sub (form=86996910) at eval.c:2257
  #30 0x0118074b in Fand (args=86996854) at eval.c:387
  #31 0x01184c42 in eval_sub (form=86987854) at eval.c:2132
  #32 0x01180aca in Fprogn (body=86987782) at eval.c:462
  #33 0x01180a17 in Fcond (args=86987702) at eval.c:440
  #34 0x01184c42 in eval_sub (form=86987262) at eval.c:2132
  #35 0x01180aca in Fprogn (body=86987246) at eval.c:462
  #36 0x01184c42 in eval_sub (form=86987254) at eval.c:2132
  #37 0x01180854 in Fif (args=87027270) at eval.c:413
  #38 0x01184c42 in eval_sub (form=87027278) at eval.c:2132
  #39 0x01180aca in Fprogn (body=87027238) at eval.c:462
  #40 0x01182360 in Fwhile (args=86990310) at eval.c:992
  #41 0x01184c42 in eval_sub (form=86990318) at eval.c:2132
  #42 0x01180aca in Fprogn (body=87027222) at eval.c:462
  #43 0x01184c42 in eval_sub (form=87027230) at eval.c:2132
  #44 0x01180b8a in Fprog1 (args=87027206) at eval.c:494
  #45 0x01184c42 in eval_sub (form=87027214) at eval.c:2132
  #46 0x01180aca in Fprogn (body=87027190) at eval.c:462
  #47 0x01182262 in Flet (args=87027182) at eval.c:970
  #48 0x01184c42 in eval_sub (form=87027174) at eval.c:2132
  #49 0x01180aca in Fprogn (body=87027150) at eval.c:462
  #50 0x01186dd7 in funcall_lambda (fun=87027166, nargs=3, arg_vector=0x88b330)
      at eval.c:3038
  #51 0x011867a9 in apply_lambda (fun=87027166, args=87017606, count=132)
      at eval.c:2920
  #52 0x011852d2 in eval_sub (form=87017614) at eval.c:2257
  #53 0x01180aca in Fprogn (body=87017582) at eval.c:462
  #54 0x01184c42 in eval_sub (form=87017622) at eval.c:2132
  #55 0x01180854 in Fif (args=87017630) at eval.c:413
  #56 0x01184c42 in eval_sub (form=87017718) at eval.c:2132
  #57 0x01180aca in Fprogn (body=87017526) at eval.c:462
  #58 0x01184c42 in eval_sub (form=87045982) at eval.c:2132
  #59 0x01180854 in Fif (args=87045566) at eval.c:413
  #60 0x01184c42 in eval_sub (form=87045574) at eval.c:2132
  #61 0x01180aca in Fprogn (body=87045534) at eval.c:462
  #62 0x0117463d in Fsave_excursion (args=87046534) at editfns.c:941
  #63 0x01184c42 in eval_sub (form=87046542) at eval.c:2132
  #64 0x01180aca in Fprogn (body=87045526) at eval.c:462
  #65 0x01182262 in Flet (args=87045518) at eval.c:970
  #66 0x01184c42 in eval_sub (form=87045510) at eval.c:2132
  #67 0x01180aca in Fprogn (body=87045494) at eval.c:462
  #68 0x01180a17 in Fcond (args=87045470) at eval.c:440
  #69 0x01184c42 in eval_sub (form=87045462) at eval.c:2132
  #70 0x01180aca in Fprogn (body=87045446) at eval.c:462
  #71 0x01180912 in Fif (args=87013070) at eval.c:414
  #72 0x01184c42 in eval_sub (form=87013078) at eval.c:2132
  #73 0x011807fa in Fif (args=87045430) at eval.c:409
  #74 0x01184c42 in eval_sub (form=87045438) at eval.c:2132
  #75 0x01180aca in Fprogn (body=87041606) at eval.c:462
  #76 0x01186dd7 in funcall_lambda (fun=87041630, nargs=2, arg_vector=0x88c154)
      at eval.c:3038
  #77 0x011865a8 in Ffuncall (nargs=3, args=0x88c150) at eval.c:2873
  #78 0x01184dd4 in eval_sub (form=59936774) at eval.c:2155
  #79 0x011807fa in Fif (args=59936302) at eval.c:409
  #80 0x01184c42 in eval_sub (form=59936782) at eval.c:2132
  #81 0x01180aca in Fprogn (body=61764958) at eval.c:462
  #82 0x01184c42 in eval_sub (form=61765246) at eval.c:2132
  #83 0x01180854 in Fif (args=61765286) at eval.c:413
  #84 0x01184c42 in eval_sub (form=61765294) at eval.c:2132
  #85 0x01180aca in Fprogn (body=61764934) at eval.c:462
  #86 0x01182360 in Fwhile (args=61766134) at eval.c:992
  #87 0x01184c42 in eval_sub (form=61766142) at eval.c:2132
  #88 0x01180aca in Fprogn (body=61761366) at eval.c:462
  #89 0x01182262 in Flet (args=61761358) at eval.c:970
  #90 0x01184c42 in eval_sub (form=61761350) at eval.c:2132
  #91 0x01180aca in Fprogn (body=61761326) at eval.c:462
  #92 0x01186dd7 in funcall_lambda (fun=61761342, nargs=4, arg_vector=0x88c980)
      at eval.c:3038
  #93 0x011867a9 in apply_lambda (fun=61761342, args=87041574, count=93)
      at eval.c:2920
  #94 0x011852d2 in eval_sub (form=87041582) at eval.c:2257
  #95 0x01180aca in Fprogn (body=87041542) at eval.c:462
  #96 0x01182262 in Flet (args=87041526) at eval.c:970
  #97 0x01184c42 in eval_sub (form=87041518) at eval.c:2132
  #98 0x01180aca in Fprogn (body=87041502) at eval.c:462
  #99 0x01179c5a in Fsave_restriction (body=87041502) at editfns.c:3415
  #100 0x01184c42 in eval_sub (form=87041510) at eval.c:2132
  #101 0x01180aca in Fprogn (body=87041478) at eval.c:462
  #102 0x01186dd7 in funcall_lambda (fun=87041494, nargs=1, arg_vector=0x88cf68)
      at eval.c:3038
  #103 0x011865a8 in Ffuncall (nargs=2, args=0x88cf64) at eval.c:2873
  #104 0x011c6fb8 in exec_byte_code (bytestr=20036361, vector=20036381,
      maxdepth=36, args_template=58931234, nargs=0, args=0x0) at bytecode.c:916
  #105 0x01186e6c in funcall_lambda (fun=20036341, nargs=3,
      arg_vector=0x131bb1d <pure+655645>) at eval.c:3045
  #106 0x011864d5 in Ffuncall (nargs=4, args=0x88d294) at eval.c:2861
  #107 0x011c6fb8 in exec_byte_code (bytestr=20033889, vector=20033909,
      maxdepth=20, args_template=58931234, nargs=0, args=0x0) at bytecode.c:916
  #108 0x01186e6c in funcall_lambda (fun=20033853, nargs=3,
      arg_vector=0x131b175 <pure+653173>) at eval.c:3045
  #109 0x011864d5 in Ffuncall (nargs=4, args=0x88d5c0) at eval.c:2861
  #110 0x01184dd4 in eval_sub (form=87464886) at eval.c:2155
  #111 0x01180aca in Fprogn (body=87464814) at eval.c:462
  #112 0x01182262 in Flet (args=87464110) at eval.c:970
  #113 0x01184c42 in eval_sub (form=87464102) at eval.c:2132
  #114 0x01180aca in Fprogn (body=87464078) at eval.c:462
  #115 0x01186dd7 in funcall_lambda (fun=87464094, nargs=3, arg_vector=0x88d9b8)
      at eval.c:3038
  #116 0x011865a8 in Ffuncall (nargs=4, args=0x88d9b4) at eval.c:2873
  #117 0x011c6fb8 in exec_byte_code (bytestr=20032865, vector=20032885,
      maxdepth=16, args_template=58931234, nargs=0, args=0x0) at bytecode.c:916
  #118 0x01186e6c in funcall_lambda (fun=20032813, nargs=2,
      arg_vector=0x131ad75 <pure+652149>) at eval.c:3045
  #119 0x011864d5 in Ffuncall (nargs=3, args=0x88de08) at eval.c:2861
  #120 0x01185833 in funcall_nil (nargs=3, args=0x88de08) at eval.c:2363
  #121 0x01185c3a in run_hook_with_args (nargs=3, args=0x88de08,
      funcall=0x118581b <funcall_nil>) at eval.c:2548
  #122 0x011858aa in Frun_hook_with_args (nargs=3, args=0x88de08) at eval.c:2409
  #123 0x01186189 in Ffuncall (nargs=4, args=0x88de04) at eval.c:2793
  #124 0x011c6fb8 in exec_byte_code (bytestr=20044041, vector=61950333,
      maxdepth=16, args_template=0, nargs=0, args=0x88e134) at bytecode.c:916
  #125 0x01186aa8 in funcall_lambda (fun=61950357, nargs=0, arg_vector=0x88e134)
      at eval.c:2979
  #126 0x011864d5 in Ffuncall (nargs=1, args=0x88e130) at eval.c:2861
  #127 0x01184dd4 in eval_sub (form=87611958) at eval.c:2155
  #128 0x011830ac in internal_lisp_condition_case (var=63281570,
      bodyform=87611958, handlers=87611910) at eval.c:1317
  #129 0x011c7e84 in exec_byte_code (bytestr=20043849, vector=20043869,
      maxdepth=76, args_template=2048, nargs=2, args=0x88e640)
      at bytecode.c:1162
  #130 0x01186aa8 in funcall_lambda (fun=20043829, nargs=2, arg_vector=0x88e638)
      at eval.c:2979
  #131 0x011864d5 in Ffuncall (nargs=3, args=0x88e634) at eval.c:2861
  #132 0x011c6fb8 in exec_byte_code (bytestr=20044417, vector=20044437,
      maxdepth=64, args_template=1024, nargs=1, args=0x88ea60) at bytecode.c:916
  #133 0x01186aa8 in funcall_lambda (fun=20044397, nargs=1, arg_vector=0x88ea5c)
      at eval.c:2979
  #134 0x011864d5 in Ffuncall (nargs=2, args=0x88ea58) at eval.c:2861
  #135 0x0118541f in Fapply (nargs=2, args=0x88ea58) at eval.c:2298
  #136 0x01186189 in Ffuncall (nargs=3, args=0x88ea54) at eval.c:2793
  #137 0x011c6fb8 in exec_byte_code (bytestr=20067945, vector=20067973,
      maxdepth=16, args_template=58931234, nargs=0, args=0x0) at bytecode.c:916
  #138 0x011c6408 in Fbyte_code (bytestr=20067945, vector=20067973, maxdepth=16)
      at bytecode.c:482
  #139 0x01184f32 in eval_sub (form=20067934) at eval.c:2188
  #140 0x011830ac in internal_lisp_condition_case (var=60678690,
      bodyform=20067934, handlers=20067998) at eval.c:1317
  #141 0x011c7e84 in exec_byte_code (bytestr=20067793, vector=20067813,
      maxdepth=20, args_template=58931234, nargs=0, args=0x0) at bytecode.c:1162
  #142 0x01186e6c in funcall_lambda (fun=20067773, nargs=1,
      arg_vector=0x13235e5 <pure+687077>) at eval.c:3045
  #143 0x011864d5 in Ffuncall (nargs=2, args=0x88f158) at eval.c:2861
  #144 0x01185d7a in call1 (fn=58972770, arg1=59339869) at eval.c:2611
  #145 0x010ffb26 in timer_check_2 (timers=58931234, idle_timers=58931234)
      at keyboard.c:4514
  #146 0x010ffc11 in timer_check () at keyboard.c:4581
  #147 0x010fdc6f in readable_events (flags=1) at keyboard.c:3447
  #148 0x0110413d in get_input_pending (flags=1) at keyboard.c:6765
  #149 0x0110a564 in detect_input_pending_run_timers (do_display=true)
      at keyboard.c:9894
  #150 0x011d0d49 in wait_reading_process_output (time_limit=0, nsecs=0,
      read_kbd=-1, do_display=true, wait_for_cell=58931234, wait_proc=0x0,
      just_wait_proc=0) at process.c:4702
  #151 0x010fe91c in kbd_buffer_get_event (kbp=0x88f484,
      used_mouse_menu=0x88f793, end_time=0x0) at keyboard.c:3906
  #152 0x010fae15 in read_event_from_main_queue (end_time=0x0,
      local_getcjmp=0x88f5ec, used_mouse_menu=0x88f793) at keyboard.c:2246
  #153 0x010fb057 in read_decoded_event_from_main_queue (end_time=0x0,
      local_getcjmp=0x88f5ec, prev_event=58931234, used_mouse_menu=0x88f793)
      at keyboard.c:2309
  #154 0x010fc3d0 in read_char (commandflag=1, map=88391966,
      prev_event=58931234, used_mouse_menu=0x88f793, end_time=0x0)
      at keyboard.c:2895
  #155 0x01108c84 in read_key_sequence (keybuf=0x88f8b0, bufsize=30,
      prompt=58931234, dont_downcase_last=false, can_return_switch_frame=true,
      fix_current_buffer=true, prevent_redisplay=false) at keyboard.c:9088
  #156 0x010f9351 in command_loop_1 () at keyboard.c:1452
  #157 0x011831b7 in internal_condition_case (bfun=0x10f8fd1 <command_loop_1>,
      handlers=58982754, hfun=0x10f8837 <cmd_error>) at eval.c:1348
  #158 0x010f8c86 in command_loop_2 (ignore=58931234) at keyboard.c:1177
  #159 0x01182764 in internal_catch (tag=58978114,
      func=0x10f8c62 <command_loop_2>, arg=58931234) at eval.c:1112
  #160 0x010f8c3e in command_loop () at keyboard.c:1156
  #161 0x010f83d4 in recursive_edit_1 () at keyboard.c:777
  #162 0x010f8590 in Frecursive_edit () at keyboard.c:848
  #163 0x010f6794 in main (argc=2, argv=0x962058) at emacs.c:1646

  Lisp Backtrace:
  "parse-partial-sexp" (0x88998c)
  "setq" (0x889b00)
  "if" (0x889c50)
  "if" (0x889d70)
  "cond" (0x889ed0)
  "progn" (0x889ff0)
  "and" (0x88a110)
  "while" (0x88a240)
  "condition-case" (0x88a3f0)
  "let" (0x88a5d0)
  "c-syntactic-re-search-forward" (0x88a6d0)
  "and" (0x88a900)
  "cond" (0x88aa60)
  "progn" (0x88ab80)
  "if" (0x88aca0)
  "while" (0x88ae00)
  "progn" (0x88af20)
  "prog1" (0x88b050)
  "let" (0x88b230)
  "c-font-lock-declarators" (0x88b330)
  "progn" (0x88b560)
  "if" (0x88b680)
  "progn" (0x88b7a0)
  "if" (0x88b8c0)
  "save-excursion" (0x88ba10)
  "let" (0x88bbe0)
  "cond" (0x88bd40)
  "if" (0x88be90)
  "if" (0x88bfb0)
  0x5302658 Lisp type 6
  "funcall" (0x88c150)
  "if" (0x88c300)
  "progn" (0x88c420)
  "if" (0x88c540)
  "while" (0x88c6a0)
  "let" (0x88c880)
  "c-find-decl-spots" (0x88c980)
  "let" (0x88cc80)
  "save-restriction" (0x88cdd0)
  "c-font-lock-declarations" (0x88cf68)
  "font-lock-fontify-keywords-region" (0x88d298)
  "font-lock-default-fontify-region" (0x88d5c4)
  "funcall" (0x88d5c0)
  "let" (0x88d820)
  "c-font-lock-fontify-region" (0x88d9b8)
  "font-lock-fontify-region" (0x88de0c)
  "run-hook-with-args" (0x88de08)
  0x3b14990 PVEC_COMPILED
  "funcall" (0x88e130)
  "jit-lock-fontify-now" (0x88e638)
  "jit-lock-stealth-fontify" (0x88ea5c)
  "apply" (0x88ea58)
  "byte-code" (0x88ecec)
  "timer-event-handler" (0x88f15c)


In GNU Emacs 24.4.1 (i686-pc-mingw32)
 of 2014-10-17 on HOME-C4E4A596F7
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --prefix=/d/usr'

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
M-x r e p o r t - e m a c s - b u g <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils time-date tooltip electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp
w32-common-fns disp-table w32-win w32-vars tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process w32notify w32
multi-tty emacs)

Memory information:
((conses 8 74262 7627)
 (symbols 32 17521 0)
 (miscs 32 33 97)
 (strings 16 10806 4314)
 (string-bytes 1 269319)
 (vectors 8 9551)
 (vector-slots 4 384761 3946)
 (floats 8 57 214)
 (intervals 28 232 95)
 (buffers 508 11))

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Stefan Monnier]

>   Error running timer `jit-lock-stealth-fontify': (args-out-of-range #<buffer buffer.c> 71003 71397)

Is the error really signaled by jit-lock, or is it just the message that
comes from jit-lock?  At least, the backtraces you show all have an
appropriate save-restriction, so the jit-lock code itself should never
see the narrowing.


        Stefan

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 18948@debbugs.gnu.org
> Date: Tue, 04 Nov 2014 15:23:17 -0500
> 
> >   Error running timer `jit-lock-stealth-fontify': (args-out-of-range #<buffer buffer.c> 71003 71397)
> 
> Is the error really signaled by jit-lock, or is it just the message that
> comes from jit-lock?

The error is signaled by parse-partial-sexp that is called indirectly
by jit-lock-stealth-fontify, as the backtrace shows.

> At least, the backtraces you show all have an appropriate
> save-restriction, so the jit-lock code itself should never see the
> narrowing.

It doesn't.  It isn't a problem with jit-stealth, this is a problem
with CC mode.

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Alan Mackenzie]

Hello, Eli
In article <mailman.12784.1415120784.1147.bug-gnu-emacs@gnu.org> you wrote:
> To reproduce, in "emacs -Q" evaluate the following:

>  (setq jit-lock-stealth-time 16)
>  (setq jit-lock-stealth-nice 0.5)
>  (setq jit-lock-stealth-verbose t)
>  (setq jit-lock-defer-contextually t)
>  (setq jit-lock-stealth-load 20)

> Then visit buffer.c from the current trunk, and wait for about 20 to
> 30 sec.  After a while, you should see the JIT Stealth messages
> blinking in the echo area, and after some more time, you will see an
> error message:

>  Error running timer `jit-lock-stealth-fontify': (args-out-of-range #<buffer buffer.c> 71003 71397)

[ ... ]

OK, I'll have a look at it.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Alan Mackenzie]

In article <mailman.12784.1415120784.1147.bug-gnu-emacs@gnu.org> you wrote:
> To reproduce, in "emacs -Q" evaluate the following:

>  (setq jit-lock-stealth-time 16)
>  (setq jit-lock-stealth-nice 0.5)
>  (setq jit-lock-stealth-verbose t)
>  (setq jit-lock-defer-contextually t)
>  (setq jit-lock-stealth-load 20)

> Then visit buffer.c from the current trunk, and wait for about 20 to
> 30 sec.  After a while, you should see the JIT Stealth messages
> blinking in the echo area, and after some more time, you will see an
> error message:

>  Error running timer `jit-lock-stealth-fontify': (args-out-of-range #<buffer buffer.c> 71003 71397)

> This happens because some code in CC mode that handles fontification
> narrows buffer.c such that the accessible region ends at position
> 71145, inside a call to jit-lock-fontify-now that wants to fontify
> between the two positions shown in the error message.  Trying to
> access buffer positions beyond 71145 triggers the error.

Thanks for such deep debugging information, Eli!  This made it very easy
to track down the bug.

> The C and Lisp backtraces, both from the call to narrow-to-region
> which performed the narrowing, and the call to parse-partial-sexp
> which signals the error, are below.  AFAICS, the narrowing is done in
> this snippet from c-find-decl-spots:

>               (if (zerop cfd-macro-end)
>                   t
>                 (if (> cfd-macro-end (point))
>                     (progn (narrow-to-region (point-min) cfd-macro-end)
>                            t)
>                   ;; The matched token was the last thing in the macro,
>                   ;; so the whole match is bogus.
>                   (setq cfd-macro-end 0)
>                   nil))))

> The function then proceeds to call cfd-fun, which, several levels
> below, calls parse-partial-sexp that barfs.

At this stage, jit-lock-... has called, indirectly,
c-font-lock-declarations with a `limit'.  This calls c-find-decl-spots
with an internal (largish) lambda function as the `cfd-fun' argument.

c-find-decl-spots calls this lambda (`cdf-fun'), having narrowed the
buffer to the end of a macro.  The lambda then calls
c-font-lock-declarators with `limit' as argument, and this later leads to
the args-out-of-range condition.  Substituting (point-max) for `limit'
fixes it.

[ ... ]

> Curiously, I cannot reproduce this on the trunk, only with Emacs 24.4,
> but if that means the problem was somehow solved on the trunk, it
> would be nice to have that solution on the branch as well.

Also, the bug doesn't show up with Emacs 24.4's buffer.c.  Why it should
trigger so rarely is a mystery.  I suspect the answer has to do with the
"##" CPP operators in the vicinity, and that on the trunk's buffer.c, they
are triggering bug #18749 (where a "##" was spuriously recognised as a
macro start because of narrowing to exactly that point).

[ ... ]

Here's a patch which fixes it.  I'll get it committed to the trunk
sometime today.

Perhaps both this fix and that to 18749 should be backported to the 24.4
branch.


--- cc-fonts.el~	2014-03-21 05:34:40.000000000 +0000
+++ cc-fonts.el	2014-11-05 11:23:05.000000000 +0000
@@ -1507,7 +1507,7 @@
 			;; At a real declaration?
 			(if (memq (c-forward-type t) '(t known found))
 			    (progn
-			      (c-font-lock-declarators limit t is-typedef)
+			      (c-font-lock-declarators (point-max) t is-typedef)
 			      nil)
 			  ;; False alarm.  Return t to go on to the next check.
 			  (goto-char start-pos)



-- 
Alan Mackenzie (Nuremberg, Germany).

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Stefan Monnier]

> Perhaps both this fix and that to 18749 should be backported to the 24.4
> branch.

Yes, please,


        Stefan

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Eli Zaretskii]

> Date: 5 Nov 2014 12:16:59 -0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: 18948@debbugs.gnu.org
> 
> Here's a patch which fixes it.  I'll get it committed to the trunk
> sometime today.

Thanks, but please commit it to emacs-24, it will be merged to trunk
soon.

> Perhaps both this fix and that to 18749 should be backported to the 24.4
> branch.

I agree with Stefan: yes, please.

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Alan Mackenzie]

Hello, Eli.

On Wed, Nov 05, 2014 at 06:09:13PM +0200, Eli Zaretskii wrote:
> > Date: 5 Nov 2014 12:16:59 -0000
> > From: Alan Mackenzie <acm@muc.de>
> > Cc: 18948@debbugs.gnu.org

> > Here's a patch which fixes it.  I'll get it committed to the trunk
> > sometime today.

> Thanks, but please commit it to emacs-24, it will be merged to trunk
> soon.

I've just committed it to Emacs-24, but not the trunk.

> > Perhaps both this fix and that to 18749 should be backported to the 24.4
> > branch.

I've backported the fix to 18749 to the Emacs-24 branch.  Hopefully I've
done it right.

> I agree with Stefan: yes, please.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Eli Zaretskii]

> Date: Wed, 5 Nov 2014 18:51:24 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, 18948@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> On Wed, Nov 05, 2014 at 06:09:13PM +0200, Eli Zaretskii wrote:
> > > Date: 5 Nov 2014 12:16:59 -0000
> > > From: Alan Mackenzie <acm@muc.de>
> > > Cc: 18948@debbugs.gnu.org
> 
> > > Here's a patch which fixes it.  I'll get it committed to the trunk
> > > sometime today.
> 
> > Thanks, but please commit it to emacs-24, it will be merged to trunk
> > soon.
> 
> I've just committed it to Emacs-24, but not the trunk.
> 
> > > Perhaps both this fix and that to 18749 should be backported to the 24.4
> > > branch.
> 
> I've backported the fix to 18749 to the Emacs-24 branch.  Hopefully I've
> done it right.

Thanks.

I'm closing this bug.