bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

This bug is bugging me for quite some time now, and my hopes for it to
be resolved are now gone, so I finally sat down to debug it.

I have jit-lock-stealth turned on in my sessions, so whenever I
restart Emacs (e.g., when I build a new binary, or after a system
restart), and restore my session using desktop.el, Emacs starts
fontifying in the background.  At some point, sometimes more than
once, I get this error:

  Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

Today I ran Emacs under a debugger, and caught this error.  The
details are below, but in a nutshell, CC mode's fontification
functions call re-search-forward with BOUND that is before point.  I
hope the data below is enough to understand why that happens and fix
it; if not, please tell what additional data is needed to diagnose the
problem.

Here're the C and Lisp backtraces from the error, and some relevant
data that explains why the error happened:

  Thread 1 hit Breakpoint 5, search_command (string=...,
      bound=make_number(123806), noerror=..., count=...,
      direction=direction@entry=1, RE=RE@entry=1, posix=posix@entry=false)
      at search.c:1046
  1046            error ("Invalid search bound (wrong side of point)");
  (gdb) bt
  #0  search_command (string=..., bound=make_number(123806), noerror=...,
      count=..., direction=direction@entry=1, RE=RE@entry=1,
      posix=posix@entry=false) at search.c:1046
  #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000ad97598),
      bound=..., noerror=..., count=...) at search.c:2271
  #2  0x0121f9f0 in funcall_subr (subr=<optimized out>,
      subr@entry=0x137fc48 <Sre_search_forward>, numargs=<optimized out>,
      numargs@entry=3, args=<optimized out>, args@entry=0x8898d0) at eval.c:2849
  #3  0x0121ddc6 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x8898c8)
      at eval.c:2766
  #4  0x0129445c in exec_byte_code (bytestr=XIL(0x800000000ad526b8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #5  0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=4,
      arg_vector=arg_vector@entry=0x889ed0) at eval.c:3049
  #6  0x0121de35 in Ffuncall (nargs=nargs@entry=5, args=args@entry=0x889ec8)
      at eval.c:2768
  #7  0x0129445c in exec_byte_code (bytestr=XIL(0x800000000ad97558),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #8  0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88a410) at eval.c:3049
  #9  0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88a408)
      at eval.c:2768
  #10 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000add7b70),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #11 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=5,
      arg_vector=arg_vector@entry=0x88a980) at eval.c:3049
  #12 0x0121de35 in Ffuncall (nargs=nargs@entry=6, args=args@entry=0x88a978)
      at eval.c:2768
  #13 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000add8368),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #14 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=5,
      arg_vector=arg_vector@entry=0x88ae50) at eval.c:3049
  #15 0x0121de35 in Ffuncall (nargs=nargs@entry=6, args=args@entry=0x88ae48)
      at eval.c:2768
  #16 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000add83e8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #17 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88b310) at eval.c:3049
  #18 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88b308)
      at eval.c:2768
  #19 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000ad54fe0),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #20 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=4,
      arg_vector=arg_vector@entry=0x88c040) at eval.c:3049
  #21 0x0121de35 in Ffuncall (nargs=nargs@entry=5, args=args@entry=0x88c038)
      at eval.c:2768
  #22 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000add83c8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #23 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88c410) at eval.c:3049
  #24 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88c408)
      at eval.c:2768
  #25 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146ae20),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #26 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88ca70) at eval.c:3049
  #27 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88ca68)
      at eval.c:2768
  #28 0x0129445c in exec_byte_code (bytestr=XIL(0x8000000001469d78),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #29 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88ce60) at eval.c:3049
  #30 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88ce58)
      at eval.c:2768
  #31 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000adad4c0),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #32 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88d230) at eval.c:3049
  #33 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88d228)
      at eval.c:2768
  #34 0x0129445c in exec_byte_code (bytestr=XIL(0x8000000001469798),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #35 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88d578) at eval.c:3049
  #36 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88d570)
      at eval.c:2768
  #37 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146d9a8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88daf8)
      at bytecode.c:629
  #38 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88daf8) at eval.c:2967
  #39 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88daf0)
      at eval.c:2768
  #40 0x0121dfff in run_hook_wrapped_funcall (nargs=2, args=0x88daf0)
      at eval.c:2493
  #41 0x0121b1e2 in run_hook_with_args (nargs=nargs@entry=2,
      args=args@entry=0x88daf0,
      funcall=funcall@entry=0x121dfcf <run_hook_wrapped_funcall>) at eval.c:2574
  #42 0x0121b38b in Frun_hook_wrapped (nargs=2, args=0x88daf0) at eval.c:2508
  #43 0x0121f8f8 in funcall_subr (
      subr=subr@entry=0x167d5c8 <Srun_hook_wrapped>, numargs=numargs@entry=2,
      args=args@entry=0x88daf0) at eval.c:2821
  #44 0x0121ddc6 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88dae8)
      at eval.c:2766
  #45 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146d938),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=2, args=<optimized out>, args@entry=0x88dee0)
      at bytecode.c:629
  #46 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88dee0) at eval.c:2967
  #47 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88ded8)
      at eval.c:2768
  #48 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146da00),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=2, args=<optimized out>, args@entry=0x88e3e0)
      at bytecode.c:629
  #49 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88e3e0) at eval.c:2967
  #50 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88e3d8)
      at eval.c:2768
  #51 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146dc90),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88e9d0)
      at bytecode.c:629
  #52 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88e9d0) at eval.c:2967
  #53 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88e9c8)
      at eval.c:2768
  #54 0x01220d8a in Fapply (nargs=2, args=0x88e9c8) at eval.c:2343
  #55 0x0121f8f8 in funcall_subr (subr=subr@entry=0x167d668 <Sapply>,
      numargs=numargs@entry=2, args=args@entry=0x88e9c8) at eval.c:2821
  #56 0x0121ddc6 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88e9c0)
      at eval.c:2766
  #57 0x0129445c in exec_byte_code (bytestr=XIL(0x80000000014770f8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88edb8)
      at bytecode.c:629
  #58 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88edb8) at eval.c:2967
  #59 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88edb0)
      at eval.c:2768
  #60 0x0121e088 in call1 (fn=XIL(0xf1f0), arg1=...) at eval.c:2617
  #61 0x0114a180 in timer_check_2 (timers=..., idle_timers=...)
      at keyboard.c:4462
  #62 0x01150644 in timer_check () at keyboard.c:4524
  #63 0x0115069a in readable_events (flags=flags@entry=1) at keyboard.c:3340
  #64 0x011594d2 in get_input_pending (flags=flags@entry=1) at keyboard.c:6824
  #65 0x011596b9 in detect_input_pending_run_timers (
      do_display=do_display@entry=true) at keyboard.c:9951
  #66 0x012a9223 in wait_reading_process_output (time_limit=<optimized out>,
      nsecs=<optimized out>, nsecs@entry=0, read_kbd=<optimized out>,
      read_kbd@entry=-1, do_display=<optimized out>, wait_for_cell=...,
      wait_proc=<optimized out>, wait_proc@entry=0x0,
      just_wait_proc=<optimized out>, just_wait_proc@entry=0) at process.c:5504
  #67 0x01159b56 in kbd_buffer_get_event (kbp=kbp@entry=0x88f47c,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3,
      end_time=end_time@entry=0x0) at keyboard.c:3831
  #68 0x0115aab4 in read_event_from_main_queue (end_time=end_time@entry=0x0,
      local_getcjmp=local_getcjmp@entry=0x88f670,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3) at keyboard.c:2151
  #69 0x0115ae38 in read_decoded_event_from_main_queue (
      end_time=end_time@entry=0x0, local_getcjmp=local_getcjmp@entry=0x88f670,
      prev_event=XIL(0), used_mouse_menu=used_mouse_menu@entry=0x88f7c3)
      at keyboard.c:2214
  #70 0x0115c9ff in read_char (commandflag=1, map=..., prev_event=...,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3,
      end_time=end_time@entry=0x0) at keyboard.c:2802
  #71 0x0115e6be in read_key_sequence (keybuf=keybuf@entry=0x88f870,
      bufsize=bufsize@entry=30, prompt=XIL(0xfb938800000000),
      dont_downcase_last=dont_downcase_last@entry=false,
      can_return_switch_frame=can_return_switch_frame@entry=true,
      fix_current_buffer=fix_current_buffer@entry=true,
      prevent_redisplay=prevent_redisplay@entry=false) at keyboard.c:9147
  #72 0x01161825 in command_loop_1 () at keyboard.c:1368
  #73 0x0121a98e in internal_condition_case (
      bfun=bfun@entry=0x1161517 <command_loop_1>, handlers=...,
      hfun=hfun@entry=0x114d246 <cmd_error>) at eval.c:1332
  #74 0x01142ce3 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110
  #75 0x0121a8f0 in internal_catch (tag=XIL(0xf538),
      func=func@entry=0x1142cbc <command_loop_2>, arg=...) at eval.c:1097
  #76 0x01142c8b in command_loop () at keyboard.c:1089
  #77 0x0114cba6 in recursive_edit_1 () at keyboard.c:695
  #78 0x0114cff7 in Frecursive_edit () at keyboard.c:766
  #79 0x01141ab3 in main (argc=<optimized out>, argv=<optimized out>)
      at emacs.c:1713

  Lisp Backtrace:
  "re-search-forward" (0x8898d0)
  "c-syntactic-re-search-forward" (0x889ed0)
  "c-forward-declarator" (0x88a410)
  "c-font-lock-declarators" (0x88a980)
  "c-font-lock-single-decl" (0x88ae50)
  0xad881a0 PVEC_COMPILED
  "c-find-decl-spots" (0x88c040)
  "c-font-lock-declarations" (0x88c410)
  "font-lock-fontify-keywords-region" (0x88ca70)
  "font-lock-default-fontify-region" (0x88ce60)
  "c-font-lock-fontify-region" (0x88d230)
  "font-lock-fontify-region" (0x88d578)
  0x83d89a8 PVEC_COMPILED
  "run-hook-wrapped" (0x88daf0)
  "jit-lock--run-functions" (0x88dee0)
  "jit-lock-fontify-now" (0x88e3e0)
  "jit-lock-stealth-fontify" (0x88e9d0)
  "apply" (0x88e9c8)
  "timer-event-handler" (0x88edb8)
  (gdb) p n
  $1 = 1
  (gdb) p lim
  $2 = <optimized out>
  (gdb) pp bound
  123806
  (gdb) p PT
  $3 = 123811

So point is 123811 and the BOUND argument of re-search-forward is
123806, too small.

  (gdb) up
  #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000ad97598),
      bound=..., noerror=..., count=...) at search.c:2271
  2271      return search_command (regexp, bound, noerror, count, 1, 1, 0);
  (gdb) pp regexp
  "[;:,]\\|\\s)\\|\\(=\\|\\s(\\)"
  (gdb) p current_buffer
  $4 = (struct buffer *) 0xb362590
  (gdb) pp current_buffer->name_
  "process.c"

These are the regexp argument to re-search-forward and the buffer
which was being fontified.

The problem happens in c-syntactic-re-search-forward in this snippet:

    (condition-case err
	(while
	    (and
	     (progn
	       (setq search-pos (point))
	       (if (re-search-forward regexp bound noerror)  <<<<<<<<<<<
		   t
		 ;; Without the following, when PAREN-LEVEL is non-nil, and
		 ;; NOERROR is not nil or t, and the very first search above
		 ;; has just failed, point would end up at BOUND rather than
		 ;; just before the next close paren.
		 (when (and (eq search-pos start)
			    paren-level
			    (not (memq noerror '(nil t))))
		   (setq state (parse-partial-sexp start bound -1))
		   (if (eq (car state) -1)
		       (setq bound (1- (point)))))
		 nil))

This is called from c-forward-declarator:

	 ;; Search syntactically to the end of the declarator (";",
	 ;; ",", a closing paren, eob etc) or to the beginning of an
	 ;; initializer or function prototype ("=" or "\\s\(").
	 ;; Note that square brackets are now not also treated as
	 ;; initializers, since this broke when there were also
	 ;; initializing brace lists.
	 (let (found)
	   (while
	       (and (progn
		      ;; In the next loop, we keep searching forward whilst
		      ;; we find ":"s which aren't single colons inside C++
		      ;; "for" statements.
		      (while
			  (and
			   (setq found
				 (c-syntactic-re-search-forward   <<<<<<<<<<<
				  "[;:,]\\|\\s)\\|\\(=\\|\\s(\\)"
				  limit t t))


It looks like c-syntactic-re-search-forward calls re-search-forward in
a loop, but perhaps it fails to update the limit to be in sync with
point that moves as the search proceeds?

Let me know what other data I can provide to help fix this annoying
problem.

In GNU Emacs 26.0.90 (build 1, i686-pc-mingw32)
 of 2017-10-12 built on HOME-C4E4A596F7
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --prefix=/d/usr --with-wide-int --with-modules
 --enable-checking=yes,glyphs 'CFLAGS=-Og -gdwarf-4 -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS MODULES LCMS2

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote w32notify w32 lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 100974 11112)
 (symbols 56 21280 1)
 (miscs 48 40 107)
 (strings 16 31517 1999)
 (string-bytes 1 759703)
 (vectors 16 14009)
 (vector-slots 8 645650 16134)
 (floats 8 51 226)
 (intervals 40 268 103)
 (buffers 880 11))

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Eli.

On Sun, Oct 15, 2017 at 19:07:50 +0300, Eli Zaretskii wrote:
> This bug is bugging me for quite some time now, and my hopes for it to
> be resolved are now gone, so I finally sat down to debug it.

> I have jit-lock-stealth turned on in my sessions, so whenever I
> restart Emacs (e.g., when I build a new binary, or after a system
> restart), and restore my session using desktop.el, Emacs starts
> fontifying in the background.  At some point, sometimes more than
> once, I get this error:

>   Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

> Today I ran Emacs under a debugger, and caught this error.  The
> details are below, but in a nutshell, CC mode's fontification
> functions call re-search-forward with BOUND that is before point.  I
> hope the data below is enough to understand why that happens and fix
> it; if not, please tell what additional data is needed to diagnose the
> problem.

> Here're the C and Lisp backtraces from the error, and some relevant
> data that explains why the error happened:

Thanks for the bug report, and what you've worked out, so far.  I'm
afraid it's going to be several days before I get a chance to look at it
properly, with things in Real Life interfering significantly at the
moment.  I will look at it, though.

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello again, Eli.

On Sun, Oct 15, 2017 at 19:07:50 +0300, Eli Zaretskii wrote:
> This bug is bugging me for quite some time now, and my hopes for it to
> be resolved are now gone, so I finally sat down to debug it.

> I have jit-lock-stealth turned on in my sessions, so whenever I
> restart Emacs (e.g., when I build a new binary, or after a system
> restart), and restore my session using desktop.el, Emacs starts
> fontifying in the background.  At some point, sometimes more than
> once, I get this error:

>   Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

> Today I ran Emacs under a debugger, and caught this error.  The
> details are below, but in a nutshell, CC mode's fontification
> functions call re-search-forward with BOUND that is before point.  I
> hope the data below is enough to understand why that happens and fix
> it; if not, please tell what additional data is needed to diagnose the
> problem.

The details you've given me are enough to form a strong hypothesis.
Thanks.

> Here're the C and Lisp backtraces from the error, and some relevant
> data that explains why the error happened:

[ .... ]

>   Lisp Backtrace:
>   "re-search-forward" (0x8898d0)
>   "c-syntactic-re-search-forward" (0x889ed0)
>   "c-forward-declarator" (0x88a410)
>   "c-font-lock-declarators" (0x88a980)
>   "c-font-lock-single-decl" (0x88ae50)
>   0xad881a0 PVEC_COMPILED
>   "c-find-decl-spots" (0x88c040)
>   "c-font-lock-declarations" (0x88c410)
>   "font-lock-fontify-keywords-region" (0x88ca70)
>   "font-lock-default-fontify-region" (0x88ce60)
>   "c-font-lock-fontify-region" (0x88d230)
>   "font-lock-fontify-region" (0x88d578)
>   0x83d89a8 PVEC_COMPILED
>   "run-hook-wrapped" (0x88daf0)
>   "jit-lock--run-functions" (0x88dee0)
>   "jit-lock-fontify-now" (0x88e3e0)
>   "jit-lock-stealth-fontify" (0x88e9d0)
>   "apply" (0x88e9c8)
>   "timer-event-handler" (0x88edb8)
>   (gdb) p n
>   $1 = 1
>   (gdb) p lim
>   $2 = <optimized out>
>   (gdb) pp bound
>   123806
>   (gdb) p PT
>   $3 = 123811

> So point is 123811 and the BOUND argument of re-search-forward is
> 123806, too small.

What I think's happening is that c-forward-declarator has found a "["
which is before BOUND, but then sets point to the matching "]" which is
after BOUND.  It then calls c-syntactic-re-search-forward again,
resulting in the error.

In master's process.c, there is a "]" very close to 123811.

>   (gdb) up
>   #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000ad97598),
>       bound=..., noerror=..., count=...) at search.c:2271
>   2271      return search_command (regexp, bound, noerror, count, 1, 1, 0);
>   (gdb) pp regexp
>   "[;:,]\\|\\s)\\|\\(=\\|\\s(\\)"
>   (gdb) p current_buffer
>   $4 = (struct buffer *) 0xb362590
>   (gdb) pp current_buffer->name_
>   "process.c"

> These are the regexp argument to re-search-forward and the buffer
> which was being fontified.

> The problem happens in c-syntactic-re-search-forward in this snippet:

[ .... ]

> This is called from c-forward-declarator:

> 	 ;; Search syntactically to the end of the declarator (";",
> 	 ;; ",", a closing paren, eob etc) or to the beginning of an
> 	 ;; initializer or function prototype ("=" or "\\s\(").
> 	 ;; Note that square brackets are now not also treated as
> 	 ;; initializers, since this broke when there were also
> 	 ;; initializing brace lists.
> 	 (let (found)
> 	   (while
> 	       (and (progn
> 		      ;; In the next loop, we keep searching forward whilst
> 		      ;; we find ":"s which aren't single colons inside C++
> 		      ;; "for" statements.
> 		      (while
> 			  (and
> 			   (setq found
> 				 (c-syntactic-re-search-forward   <<<<<<<<<<<
> 				  "[;:,]\\|\\s)\\|\\(=\\|\\s(\\)"
> 				  limit t t))

As already suggested, I think the bug is that there aren't enough checks
that (< (point) limit) in this function.  I have added them in.

> It looks like c-syntactic-re-search-forward calls re-search-forward in
> a loop, but perhaps it fails to update the limit to be in sync with
> point that moves as the search proceeds?

A further problem is that c-font-lock-declarators is calling
c-forward-declarator with a limit; this is silly - if the end of a
declaration runs over a jit-lock chunk boundary, we still want to
fontify this declaration fully.  So I've changed the LIMIT argument in
the pertinent two calls to nil.  (There is a third call somewhere where
this LIMIT argument is the end of a macro, and it is absolutely needed).

> Let me know what other data I can provide to help fix this annoying
> problem.

I haven't reproduced the problem, but I admit I haven't tried all that
hard.  Could you please try out the patch below, and let me know if it
fixes the bug.

> In GNU Emacs 26.0.90 (build 1, i686-pc-mingw32)
>  of 2017-10-12 built on HOME-C4E4A596F7
> Windowing system distributor 'Microsoft Corp.', version 5.1.2600
> Recent messages:
> For information about GNU Emacs and the GNU system, type C-h C-a.

[ .... ]


diff --git a/lisp/progmodes/cc-engine.el b/lisp/progmodes/cc-engine.el
index 3792835752..07b9215046 100644
--- a/lisp/progmodes/cc-engine.el
+++ b/lisp/progmodes/cc-engine.el
@@ -8102,12 +8102,14 @@ c-forward-declarator
 	 ;; initializing brace lists.
 	 (let (found)
 	   (while
-	       (and (progn
+	       (and (< (point) limit)
+		    (progn
 		      ;; In the next loop, we keep searching forward whilst
 		      ;; we find ":"s which aren't single colons inside C++
 		      ;; "for" statements.
 		      (while
 			  (and
+			   (< (point) limit)
 			   (setq found
 				 (c-syntactic-re-search-forward
 				  "[;:,]\\|\\s)\\|\\(=\\|\\s(\\)"
@@ -8129,7 +8131,7 @@ c-forward-declarator
 		    (c-go-up-list-forward))
 	     (setq brackets-after-id t))
 	   (when found (backward-char))
-	   t))
+	   (<= (point) limit)))
 	(list id-start id-end brackets-after-id (match-beginning 1) decorated)
 
       (goto-char here)
diff --git a/lisp/progmodes/cc-fonts.el b/lisp/progmodes/cc-fonts.el
index 02b685d240..b8dbe3c26b 100644
--- a/lisp/progmodes/cc-fonts.el
+++ b/lisp/progmodes/cc-fonts.el
@@ -1062,7 +1062,7 @@ c-font-lock-declarators
     ;; The following `while' fontifies a single declarator id each time round.
     ;; It loops only when LIST is non-nil.
     (while
-	(and pos (setq decl-res (c-forward-declarator limit)))
+	(and pos (setq decl-res (c-forward-declarator)))
       (setq next-pos (point)
 	    id-start (car decl-res)
 	    id-face (if (and (eq (char-after) ?\()
@@ -1091,7 +1091,7 @@ c-font-lock-declarators
 					       (throw 'is-function nil))
 					      ((not (eq got-type 'maybe))
 					       (throw 'is-function t)))
-					     (c-forward-declarator limit t)
+					     (c-forward-declarator nil t)
 					     (eq (char-after) ?,))
 					 (forward-char)
 					 (c-forward-syntactic-ws))


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> Date: Sun, 22 Oct 2017 20:13:40 +0000
> Cc: 28850@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > So point is 123811 and the BOUND argument of re-search-forward is
> > 123806, too small.
> 
> What I think's happening is that c-forward-declarator has found a "["
> which is before BOUND, but then sets point to the matching "]" which is
> after BOUND.  It then calls c-syntactic-re-search-forward again,
> resulting in the error.
> 
> In master's process.c, there is a "]" very close to 123811.

For the record, here's the place where this happened, with the two
locations shown by "^":

      char namebuf[sizeof (ifq->ifr_name) + 1];
                                         ^    ^
The reason you don't see this in process.c you have is that the
version I used was edited wrt to what you have.

> I haven't reproduced the problem, but I admit I haven't tried all that
> hard.  Could you please try out the patch below, and let me know if it
> fixes the bug.

Thanks, this fixes a very large part of the problem, so I think you
should install this on the release branch.

It doesn't solve all of it, though, because I got that breakpoint hit
again.  This time it took much longer before that happened (a sign
that most of the problem is indeed solved), and the backtrace is
different.  Here's the C and the Lisp backtraces, followed by some
relevant values:

  Thread 1 hit Breakpoint 3, search_command (string=...,
      bound=make_number(2425), noerror=..., count=...,
      direction=direction@entry=1, RE=RE@entry=1, posix=posix@entry=false)
      at search.c:1046
  1046            error ("Invalid search bound (wrong side of point)");
  (gdb) bt
  #0  search_command (string=..., bound=make_number(2425), noerror=...,
      count=..., direction=direction@entry=1, RE=RE@entry=1,
      posix=posix@entry=false) at search.c:1046
  #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000adcb1f0),
      bound=..., noerror=..., count=...) at search.c:2271
  #2  0x0121f9f0 in funcall_subr (subr=<optimized out>,
      subr@entry=0x137fc48 <Sre_search_forward>, numargs=<optimized out>,
      numargs@entry=3, args=<optimized out>, args@entry=0x88c050) at eval.c:2849
  #3  0x0121ddc6 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88c048)
      at eval.c:2766
  #4  0x0129445c in exec_byte_code (bytestr=XIL(0x800000000adcb1c0),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #5  0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88c410) at eval.c:3049
  #6  0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88c408)
      at eval.c:2768
  #7  0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146ae20),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #8  0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88ca70) at eval.c:3049
  #9  0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88ca68)
      at eval.c:2768
  #10 0x0129445c in exec_byte_code (bytestr=XIL(0x8000000001469d78),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #11 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88ce60) at eval.c:3049
  #12 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88ce58)
      at eval.c:2768
  #13 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000ad9d5a8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #14 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88d230) at eval.c:3049
  #15 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88d228)
      at eval.c:2768
  #16 0x0129445c in exec_byte_code (bytestr=XIL(0x8000000001469798),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #17 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88d578) at eval.c:3049
  #18 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88d570)
      at eval.c:2768
  #19 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146d9a8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88daf8)
      at bytecode.c:629
  #20 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88daf8) at eval.c:2967
  #21 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88daf0)
      at eval.c:2768
  #22 0x0121dfff in run_hook_wrapped_funcall (nargs=2, args=0x88daf0)
      at eval.c:2493
  #23 0x0121b1e2 in run_hook_with_args (nargs=nargs@entry=2,
      args=args@entry=0x88daf0,
      funcall=funcall@entry=0x121dfcf <run_hook_wrapped_funcall>) at eval.c:2574
  #24 0x0121b38b in Frun_hook_wrapped (nargs=2, args=0x88daf0) at eval.c:2508
  #25 0x0121f8f8 in funcall_subr (
      subr=subr@entry=0x167d5c8 <Srun_hook_wrapped>, numargs=numargs@entry=2,
      args=args@entry=0x88daf0) at eval.c:2821
  #26 0x0121ddc6 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88dae8)
      at eval.c:2766
  #27 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146d938),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=2, args=<optimized out>, args@entry=0x88dee0)
      at bytecode.c:629
  #28 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88dee0) at eval.c:2967
  #29 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88ded8)
      at eval.c:2768
  #30 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146da00),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=2, args=<optimized out>, args@entry=0x88e3e0)
      at bytecode.c:629
  #31 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88e3e0) at eval.c:2967
  #32 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88e3d8)
      at eval.c:2768
  #33 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146dc90),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88e9d0)
      at bytecode.c:629
  #34 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88e9d0) at eval.c:2967
  #35 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88e9c8)
      at eval.c:2768
  #36 0x01220d8a in Fapply (nargs=2, args=0x88e9c8) at eval.c:2343
  #37 0x0121f8f8 in funcall_subr (subr=subr@entry=0x167d668 <Sapply>,
      numargs=numargs@entry=2, args=args@entry=0x88e9c8) at eval.c:2821
  #38 0x0121ddc6 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88e9c0)
      at eval.c:2766
  #39 0x0129445c in exec_byte_code (bytestr=XIL(0x80000000014770f8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88edb8)
      at bytecode.c:629
  #40 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88edb8) at eval.c:2967
  #41 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88edb0)
      at eval.c:2768
  #42 0x0121e088 in call1 (fn=XIL(0xf1f0), arg1=...) at eval.c:2617
  #43 0x0114a180 in timer_check_2 (timers=..., idle_timers=...)
      at keyboard.c:4462
  #44 0x01150644 in timer_check () at keyboard.c:4524
  #45 0x0115069a in readable_events (flags=flags@entry=1) at keyboard.c:3340
  #46 0x011594d2 in get_input_pending (flags=flags@entry=1) at keyboard.c:6824
  #47 0x011596b9 in detect_input_pending_run_timers (
      do_display=do_display@entry=true) at keyboard.c:9951
  #48 0x012a9223 in wait_reading_process_output (time_limit=<optimized out>,
      nsecs=<optimized out>, nsecs@entry=0, read_kbd=<optimized out>,
      read_kbd@entry=-1, do_display=<optimized out>, wait_for_cell=...,
      wait_proc=<optimized out>, wait_proc@entry=0x0,
      just_wait_proc=<optimized out>, just_wait_proc@entry=0) at process.c:5504
  #49 0x01159b56 in kbd_buffer_get_event (kbp=kbp@entry=0x88f47c,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3,
      end_time=end_time@entry=0x0) at keyboard.c:3831
  #50 0x0115aab4 in read_event_from_main_queue (end_time=end_time@entry=0x0,
      local_getcjmp=local_getcjmp@entry=0x88f670,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3) at keyboard.c:2151
  #51 0x0115ae38 in read_decoded_event_from_main_queue (
      end_time=end_time@entry=0x0, local_getcjmp=local_getcjmp@entry=0x88f670,
      prev_event=XIL(0), used_mouse_menu=used_mouse_menu@entry=0x88f7c3)
      at keyboard.c:2214
  #52 0x0115c9ff in read_char (commandflag=1, map=..., prev_event=...,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3,
      end_time=end_time@entry=0x0) at keyboard.c:2802
  #53 0x0115e6be in read_key_sequence (keybuf=keybuf@entry=0x88f870,
      bufsize=bufsize@entry=30, prompt=XIL(0x9b938800000000),
      dont_downcase_last=dont_downcase_last@entry=false,
      can_return_switch_frame=can_return_switch_frame@entry=true,
      fix_current_buffer=fix_current_buffer@entry=true,
      prevent_redisplay=prevent_redisplay@entry=false) at keyboard.c:9147
  #54 0x01161825 in command_loop_1 () at keyboard.c:1368
  #55 0x0121a98e in internal_condition_case (
      bfun=bfun@entry=0x1161517 <command_loop_1>, handlers=...,
      hfun=hfun@entry=0x114d246 <cmd_error>) at eval.c:1332
  #56 0x01142ce3 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110
  #57 0x0121a8f0 in internal_catch (tag=XIL(0xf538),
      func=func@entry=0x1142cbc <command_loop_2>, arg=...) at eval.c:1097
  #58 0x01142c8b in command_loop () at keyboard.c:1089
  #59 0x0114cba6 in recursive_edit_1 () at keyboard.c:695
  #60 0x0114cff7 in Frecursive_edit () at keyboard.c:766
  #61 0x01141ab3 in main (argc=<optimized out>, argv=<optimized out>)
      at emacs.c:1713

  Lisp Backtrace:
  "re-search-forward" (0x88c050)
  0xad7f328 PVEC_COMPILED
  "font-lock-fontify-keywords-region" (0x88ca70)
  "font-lock-default-fontify-region" (0x88ce60)
  "c-font-lock-fontify-region" (0x88d230)
  "font-lock-fontify-region" (0x88d578)
  0x13810840 PVEC_COMPILED
  "run-hook-wrapped" (0x88daf0)
  "jit-lock--run-functions" (0x88dee0)
  "jit-lock-fontify-now" (0x88e3e0)
  "jit-lock-stealth-fontify" (0x88e9d0)
  "apply" (0x88e9c8)
  "timer-event-handler" (0x88edb8)
  (gdb) pp current_buffer->name_
  "platform.h"
  (gdb) pp current_buffer->directory_
  "d:/utils/lz4-1.7.5/programs/"
  (gdb) p PT
  $1 = 2645
  (gdb) p bound
  $2 = make_number(2425)
  (gdb) up
  #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000adcb1f0),
      bound=..., noerror=..., count=...) at search.c:2271
  2271      return search_command (regexp, bound, noerror, count, 1, 1, 0);
  (gdb) pp regexp
  "\\(\\=\\|\\(\\=\\|[^\\]\\)[
  ]\\)\\s *#\\s *\\(\\(?:\\(?:el\\)?if\\)\\)\\([^[:alnum:]_$]\\|$\\)\\(\\\\\\(.\\|
  [
  ]\\)\\|[^
  ]\\)*"

As you see, point is at 2645, whereas BOUND is at 2425.  This happens
in the file platform.h from the lz4-1.7.5 distribution.  Here's the
relevant part of platform.h with the two locations shown (I added an
empty line for each "^" marker):

/* **************************************
*  Detect 64-bit OS
*  http://nadeausoftware.com/articles/2012/02/c_c_tip_how_detect_processor_type_using_compiler_predefined_macros
****************************************/
#if defined __ia64 || defined _M_IA64                                                                               /* Intel Itanium */ \
  || defined __powerpc64__ || defined __ppc64__ || defined __PPC64__                                                /* POWER 64-bit */  \
  || (defined __sparc && (defined __sparcv9 || defined __sparc_v9__ || defined __arch64__)) || defined __sparc64__  /* SPARC 64-bit */  \
  || defined __x86_64__s || defined _M_X64                                                                          /* x86 64-bit */    \
  || defined __arm64__ || defined __aarch64__ || defined __ARM64_ARCH_8__                                           /* ARM 64-bit */    \
  || (defined __mips  && (__mips == 64 || __mips == 4 || __mips == 3))                                              /* MIPS 64-bit */   \
                                                                      ^
  || defined _LP64 || defined __LP64__ /* NetBSD, OpenBSD */ || defined __64BIT__ /* AIX */ || defined _ADDR64 /* Cray */               \
  || (defined __SIZEOF_POINTER__ && __SIZEOF_POINTER__ == 8) /* gcc */
              ^
#  if !defined(__64BIT__)
#    define __64BIT__  1
#  endif
#endif

As you see, BOUND is after the closing paren, after "3))", and point
is at the beginning of "__SIZEOF_POINTER__" a couple of lines further.

To tell the truth, the Lisp backtrace puzzles me a bit, because the
only call to re-search-forward in font-lock-fontify-keywords-region is
protected by a condition that should have prevented this problem from
happening:

      (while (and (< (point) end)
		  (if (stringp matcher)
		      (re-search-forward matcher end t)
		    (funcall matcher end))

So maybe I'm missing something, or maybe the problematic call to
re-search-forward comes from some macro expansion I didn't identify.

Let me know if I can provide any more details for your analysis.

Thanks.

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Eli.

On Tue, Oct 24, 2017 at 17:46:08 +0300, Eli Zaretskii wrote:
> > Date: Sun, 22 Oct 2017 20:13:40 +0000
> > Cc: 28850@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > So point is 123811 and the BOUND argument of re-search-forward is
> > > 123806, too small.

> > What I think's happening is that c-forward-declarator has found a "["
> > which is before BOUND, but then sets point to the matching "]" which is
> > after BOUND.  It then calls c-syntactic-re-search-forward again,
> > resulting in the error.

> > In master's process.c, there is a "]" very close to 123811.

> For the record, here's the place where this happened, with the two
> locations shown by "^":

>       char namebuf[sizeof (ifq->ifr_name) + 1];
>                                          ^    ^
> The reason you don't see this in process.c you have is that the
> version I used was edited wrt to what you have.

:-).  It's exactly as I'd surmised.

> > I haven't reproduced the problem, but I admit I haven't tried all that
> > hard.  Could you please try out the patch below, and let me know if it
> > fixes the bug.

> Thanks, this fixes a very large part of the problem, so I think you
> should install this on the release branch.

I'll do that just as soon as I'm in the mood for routine work.  This
evening was right for debugging the rest of it.

> It doesn't solve all of it, though, because I got that breakpoint hit
> again.  This time it took much longer before that happened (a sign
> that most of the problem is indeed solved), and the backtrace is
> different.

This is an entirely separate bug.

> Here's the C and the Lisp backtraces, followed by some relevant
> values:

[ .... ].

>   Lisp Backtrace:
>   "re-search-forward" (0x88c050)
>   0xad7f328 PVEC_COMPILED
>   "font-lock-fontify-keywords-region" (0x88ca70)
>   "font-lock-default-fontify-region" (0x88ce60)
>   "c-font-lock-fontify-region" (0x88d230)
>   "font-lock-fontify-region" (0x88d578)
>   0x13810840 PVEC_COMPILED
>   "run-hook-wrapped" (0x88daf0)
>   "jit-lock--run-functions" (0x88dee0)
>   "jit-lock-fontify-now" (0x88e3e0)
>   "jit-lock-stealth-fontify" (0x88e9d0)
>   "apply" (0x88e9c8)
>   "timer-event-handler" (0x88edb8)
>   (gdb) pp current_buffer->name_
>   "platform.h"
>   (gdb) pp current_buffer->directory_
>   "d:/utils/lz4-1.7.5/programs/"
>   (gdb) p PT
>   $1 = 2645
>   (gdb) p bound
>   $2 = make_number(2425)
>   (gdb) up
>   #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000adcb1f0),
>       bound=..., noerror=..., count=...) at search.c:2271
>   2271      return search_command (regexp, bound, noerror, count, 1, 1, 0);
>   (gdb) pp regexp
>   "\\(\\=\\|\\(\\=\\|[^\\]\\)[
>   ]\\)\\s *#\\s *\\(\\(?:\\(?:el\\)?if\\)\\)\\([^[:alnum:]_$]\\|$\\)\\(\\\\\\(.\\|
>   [
>   ]\\)\\|[^
>   ]\\)*"

I've tracked down that regexp.  It matches "#if" or "#elif" inside
macros.  It's used in c-cpp-matchers in cc-fonts.el.

> As you see, point is at 2645, whereas BOUND is at 2425.  This happens
> in the file platform.h from the lz4-1.7.5 distribution.  Here's the
> relevant part of platform.h with the two locations shown (I added an
> empty line for each "^" marker):

The reason for this is that a generated lambda form with the argument
LIMIT (which would be the end of a jit-lock chunk or similar) internally
binds LIMIT to the end of the current macro.  Inside this binding, which
searches for "defined" repeatedly, we go forward to after the last
"defined", as indicated in your source excerpt below.  Unfortunately,
this point is beyond the original LIMIT supplied to the lambda, so in the
next re-search-forward, point is the wrong side of this original LIMIT.

This particular bit of CC Mode is "write only" code, and it could take
me some while to disentangle the stack of macros and function generators
which has produced it.  I think I'm going to extract this form and
rewrite it more by hand, making it simpler to debug in the future.

> /* **************************************
> *  Detect 64-bit OS
> *  http://nadeausoftware.com/articles/2012/02/c_c_tip_how_detect_processor_type_using_compiler_predefined_macros
> ****************************************/
> #if defined __ia64 || defined _M_IA64                                                                               /* Intel Itanium */ \
>   || defined __powerpc64__ || defined __ppc64__ || defined __PPC64__                                                /* POWER 64-bit */  \
>   || (defined __sparc && (defined __sparcv9 || defined __sparc_v9__ || defined __arch64__)) || defined __sparc64__  /* SPARC 64-bit */  \
>   || defined __x86_64__s || defined _M_X64                                                                          /* x86 64-bit */    \
>   || defined __arm64__ || defined __aarch64__ || defined __ARM64_ARCH_8__                                           /* ARM 64-bit */    \
>   || (defined __mips  && (__mips == 64 || __mips == 4 || __mips == 3))                                              /* MIPS 64-bit */   \
>                                                                       ^
>   || defined _LP64 || defined __LP64__ /* NetBSD, OpenBSD */ || defined __64BIT__ /* AIX */ || defined _ADDR64 /* Cray */               \
>   || (defined __SIZEOF_POINTER__ && __SIZEOF_POINTER__ == 8) /* gcc */
>               ^
> #  if !defined(__64BIT__)
> #    define __64BIT__  1
> #  endif
> #endif

> As you see, BOUND is after the closing paren, after "3))", and point
> is at the beginning of "__SIZEOF_POINTER__" a couple of lines further.

Yes.  BOUND would have been the end of a jit-lock-chunk.

> To tell the truth, the Lisp backtrace puzzles me a bit, because the
> only call to re-search-forward in font-lock-fontify-keywords-region is
> protected by a condition that should have prevented this problem from
> happening:

>       (while (and (< (point) end)
> 		  (if (stringp matcher)
> 		      (re-search-forward matcher end t)
> 		    (funcall matcher end))

> So maybe I'm missing something, or maybe the problematic call to
> re-search-forward comes from some macro expansion I didn't identify.

Indeed it does.  The expansion of the macro is in C Mode's
font-lock-keywords.

> Let me know if I can provide any more details for your analysis.

Will do, but I think you've given me enough to solve this.  Thanks!

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello again, Eli.

On Tue, Oct 24, 2017 at 20:33:12 +0000, Alan Mackenzie wrote:
> On Tue, Oct 24, 2017 at 17:46:08 +0300, Eli Zaretskii wrote:
> > > Date: Sun, 22 Oct 2017 20:13:40 +0000
> > > Cc: 28850@debbugs.gnu.org
> > > From: Alan Mackenzie <acm@muc.de>

[ .... ]

> > Thanks, this fixes a very large part of the problem, so I think you
> > should install this on the release branch.

DONE.

> > It doesn't solve all of it, though, because I got that breakpoint hit
> > again.  This time it took much longer before that happened (a sign
> > that most of the problem is indeed solved), and the backtrace is
> > different.

> This is an entirely separate bug.

> > Here's the C and the Lisp backtraces, followed by some relevant
> > values:

> [ .... ].

> >   Lisp Backtrace:
> >   "re-search-forward" (0x88c050)
> >   0xad7f328 PVEC_COMPILED
> >   "font-lock-fontify-keywords-region" (0x88ca70)
> >   "font-lock-default-fontify-region" (0x88ce60)
> >   "c-font-lock-fontify-region" (0x88d230)
> >   "font-lock-fontify-region" (0x88d578)
> >   0x13810840 PVEC_COMPILED
> >   "run-hook-wrapped" (0x88daf0)
> >   "jit-lock--run-functions" (0x88dee0)
> >   "jit-lock-fontify-now" (0x88e3e0)
> >   "jit-lock-stealth-fontify" (0x88e9d0)
> >   "apply" (0x88e9c8)
> >   "timer-event-handler" (0x88edb8)
> >   (gdb) pp current_buffer->name_
> >   "platform.h"
> >   (gdb) pp current_buffer->directory_
> >   "d:/utils/lz4-1.7.5/programs/"
> >   (gdb) p PT
> >   $1 = 2645
> >   (gdb) p bound
> >   $2 = make_number(2425)
> >   (gdb) up
> >   #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000adcb1f0),
> >       bound=..., noerror=..., count=...) at search.c:2271
> >   2271      return search_command (regexp, bound, noerror, count, 1, 1, 0);
> >   (gdb) pp regexp
> >   "\\(\\=\\|\\(\\=\\|[^\\]\\)[
> >   ]\\)\\s *#\\s *\\(\\(?:\\(?:el\\)?if\\)\\)\\([^[:alnum:]_$]\\|$\\)\\(\\\\\\(.\\|
> >   [
> >   ]\\)\\|[^
> >   ]\\)*"

> I've tracked down that regexp.  It matches "#if" or "#elif" inside
> macros.  It's used in c-cpp-matchers in cc-fonts.el.

> > As you see, point is at 2645, whereas BOUND is at 2425.  This happens
> > in the file platform.h from the lz4-1.7.5 distribution.  Here's the
> > relevant part of platform.h with the two locations shown (I added an
> > empty line for each "^" marker):

> The reason for this is that a generated lambda form with the argument
> LIMIT (which would be the end of a jit-lock chunk or similar) internally
> binds LIMIT to the end of the current macro.  Inside this binding, which
> searches for "defined" repeatedly, we go forward to after the last
> "defined", as indicated in your source excerpt below.  Unfortunately,
> this point is beyond the original LIMIT supplied to the lambda, so in the
> next re-search-forward, point is the wrong side of this original LIMIT.

> This particular bit of CC Mode is "write only" code, and it could take
> me some while to disentangle the stack of macros and function generators
> which has produced it.  I think I'm going to extract this form and
> rewrite it more by hand, making it simpler to debug in the future.

Actually, it wasn't that difficult to amend that form generator.  Would
you please try out the patch below, which should apply cleanly to
master.

> > /* **************************************
> > *  Detect 64-bit OS
> > *  http://nadeausoftware.com/articles/2012/02/c_c_tip_how_detect_processor_type_using_compiler_predefined_macros
> > ****************************************/
> > #if defined __ia64 || defined _M_IA64                                                                               /* Intel Itanium */ \
> >   || defined __powerpc64__ || defined __ppc64__ || defined __PPC64__                                                /* POWER 64-bit */  \
> >   || (defined __sparc && (defined __sparcv9 || defined __sparc_v9__ || defined __arch64__)) || defined __sparc64__  /* SPARC 64-bit */  \
> >   || defined __x86_64__s || defined _M_X64                                                                          /* x86 64-bit */    \
> >   || defined __arm64__ || defined __aarch64__ || defined __ARM64_ARCH_8__                                           /* ARM 64-bit */    \
> >   || (defined __mips  && (__mips == 64 || __mips == 4 || __mips == 3))                                              /* MIPS 64-bit */   \
> >                                                                       ^
> >   || defined _LP64 || defined __LP64__ /* NetBSD, OpenBSD */ || defined __64BIT__ /* AIX */ || defined _ADDR64 /* Cray */               \
> >   || (defined __SIZEOF_POINTER__ && __SIZEOF_POINTER__ == 8) /* gcc */
> >               ^
> > #  if !defined(__64BIT__)
> > #    define __64BIT__  1
> > #  endif
> > #endif

> > As you see, BOUND is after the closing paren, after "3))", and point
> > is at the beginning of "__SIZEOF_POINTER__" a couple of lines further.

> Yes.  BOUND would have been the end of a jit-lock-chunk.

[ .... ]


diff -r c2f277772ea2 cc-fonts.el
--- a/cc-fonts.el	Wed Oct 25 18:00:13 2017 +0000
+++ b/cc-fonts.el	Wed Oct 25 18:57:20 2017 +0000
@@ -286,12 +286,17 @@
 			      nil)))))
 	  res))))
 
-  (defun c-make-font-lock-search-form (regexp highlights)
+  (defun c-make-font-lock-search-form (regexp highlights &optional check-point)
     ;; Return a lisp form which will fontify every occurrence of REGEXP
     ;; (a regular expression, NOT a function) between POINT and `limit'
     ;; with HIGHLIGHTS, a list of highlighters as specified on page
-    ;; "Search-based Fontification" in the elisp manual.
-    `(while (re-search-forward ,regexp limit t)
+    ;; "Search-based Fontification" in the elisp manual.  If CHECK-POINT
+    ;; is non-nil, we will check (< (point) limit) in the main loop.
+    `(while
+	 ,(if check-point
+	      `(and (< (point) limit)
+		    (re-search-forward ,regexp limit t))
+	    `(re-search-forward ,regexp limit t))
        (unless (progn
 		 (goto-char (match-beginning 0))
 		 (c-skip-comments-and-strings limit))
@@ -470,7 +475,9 @@
 			,(c-make-font-lock-search-form
 			  regexp highlights)))))
 	     state-stanzas)
-	  ,(c-make-font-lock-search-form (car normal) (cdr normal))
+	  ;; In the next form, check that point hasn't been moved beyond
+	  ;; `limit' in any of the above stanzas.
+	  ,(c-make-font-lock-search-form (car normal) (cdr normal) t)
 	  nil))))
 
 ;  (eval-after-load "edebug" ; 2006-07-09: def-edebug-spec is now in subr.el.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> Date: Wed, 25 Oct 2017 19:11:37 +0000
> Cc: 28850@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Actually, it wasn't that difficult to amend that form generator.  Would
> you please try out the patch below, which should apply cleanly to
> master.

I think you've solved the problem, because I let Emacs run idle for 10
hours, and it didn't hit this error even once.

Thanks.  Please push to the release branch.

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Eli.

On Thu, Oct 26, 2017 at 19:44:28 +0300, Eli Zaretskii wrote:
> > Date: Wed, 25 Oct 2017 19:11:37 +0000
> > Cc: 28850@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > Actually, it wasn't that difficult to amend that form generator.  Would
> > you please try out the patch below, which should apply cleanly to
> > master.

> I think you've solved the problem, ....

It was definitely a joint effort.  :-)

> .... because I let Emacs run idle for 10 hours, and it didn't hit this
> error even once.

> Thanks.  Please push to the release branch.

DONE.  I'm closing the bug with this post.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Basil L. Contovounesios]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Wed, 25 Oct 2017 19:11:37 +0000
>> Cc: 28850@debbugs.gnu.org
>> From: Alan Mackenzie <acm@muc.de>
>> 
>> Actually, it wasn't that difficult to amend that form generator.  Would
>> you please try out the patch below, which should apply cleanly to
>> master.
>
> I think you've solved the problem, because I let Emacs run idle for 10
> hours, and it didn't hit this error even once.

It seems to have returned in some way.  I can't reproduce this on Emacs
26, but on latest master, the following steps:

0. emacs -Q
1. (progn (setq debug-on-error t)
          (setq jit-lock-stealth-nice nil)
          (setq jit-lock-stealth-time 0)
          (find-function #'next-property-change))
2. C-x C-e

almost immediately lead to the following backtrace:

Debugger entered--Lisp error: (error "Invalid search bound (wrong side of point)")
  search-forward-regexp("\\<\\(\\(?:enum\\)\\)\\>[^][{};/#=]*{" 1673 t)
  c-font-lock-enum-body(1673)
  font-lock-fontify-keywords-region(1123 1673 nil)
  font-lock-default-fontify-region(1123 1673 nil)
  c-font-lock-fontify-region(1173 1673 nil)
  font-lock-fontify-region(1173 1673)
  #f(compiled-function (fun) #<bytecode 0x1565bb8a9581>)(font-lock-fontify-region)
  run-hook-wrapped(#f(compiled-function (fun) #<bytecode 0x1565bb8a9581>) font-lock-fontify-region)
  jit-lock--run-functions(1173 1673)
  jit-lock-fontify-now(1173 1673)
  jit-lock-stealth-fontify(t)
  apply(jit-lock-stealth-fontify t)
  timer-event-handler([t 0 0 974323 nil jit-lock-stealth-fontify (t) idle 261000])

I'm not sure if this says anything, but when the *Backtrace* buffer is
displayed, the textprop.c buffer is marked as modified.  Could this be
related to the before/after change machinery?

A similar error I occasionally see, but have not yet figured out how to
reproduce:

  Error during redisplay: (jit-lock-function 19569)
    signaled (error "Invalid search bound (wrong side of point)")

Thanks,

-- 
Basil

In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-04-30 built on thunk
Repository revision: f478082f9ff22ff41fbd9616ebea75757f9a0311
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Debian GNU/Linux buster/sid

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O2 -march=native' --config-cache
 --prefix=/home/blc/.local --with-mailutils --with-x-toolkit=lucid
 --with-modules --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT
LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS
LIBSYSTEMD JSON PDUMPER LCMS2 GMP

Package: CC Mode 5.33.2 (C/*l)
Buffer Style: GNU
c-emacs-features: (pps-extended-state col-0-paren posix-char-classes gen-string-delim gen-comment-delim syntax-properties 1-bit)

current state:
==============
(setq
 c-basic-offset 2
 c-comment-only-line-offset '(0 . 0)
 c-indent-comment-alist '((anchored-comment column . 0) (end-block space . 1)
			  (cpp-end-block space . 2))
 c-indent-comments-syntactically-p nil
 c-block-comment-prefix ""
 c-comment-prefix-regexp '((pike-mode . "//+!?\\|\\**") (awk-mode . "#+")
			   (other . "//+\\|\\**"))
 c-doc-comment-style '((java-mode . javadoc) (pike-mode . autodoc)
		       (c-mode . gtkdoc))
 c-cleanup-list '(scope-operator)
 c-hanging-braces-alist '((substatement-open before after) (arglist-cont-nonempty))
 c-hanging-colons-alist nil
 c-hanging-semi&comma-criteria '(c-semi&comma-inside-parenlist)
 c-backslash-column 48
 c-backslash-max-column 72
 c-special-indent-hook '(c-gnu-impose-minimum)
 c-label-minimum-indentation 1
 c-offsets-alist '((inexpr-class . +)
		   (inexpr-statement . +)
		   (lambda-intro-cont . +)
		   (inlambda . c-lineup-inexpr-block)
		   (template-args-cont c-lineup-template-args +)
		   (incomposition . +)
		   (inmodule . +)
		   (innamespace . +)
		   (inextern-lang . +)
		   (composition-close . 0)
		   (module-close . 0)
		   (namespace-close . 0)
		   (extern-lang-close . 0)
		   (composition-open . 0)
		   (module-open . 0)
		   (namespace-open . 0)
		   (extern-lang-open . 0)
		   (objc-method-call-cont
		    c-lineup-ObjC-method-call-colons
		    c-lineup-ObjC-method-call
		    +
		    )
		   (objc-method-args-cont . c-lineup-ObjC-method-args)
		   (objc-method-intro . [0])
		   (friend . 0)
		   (cpp-define-intro c-lineup-cpp-define +)
		   (cpp-macro-cont . +)
		   (cpp-macro . [0])
		   (inclass . +)
		   (stream-op . c-lineup-streamop)
		   (arglist-cont-nonempty c-lineup-gcc-asm-reg c-lineup-arglist)
		   (arglist-cont c-lineup-gcc-asm-reg 0)
		   (comment-intro c-lineup-knr-region-comment c-lineup-comment)
		   (catch-clause . 0)
		   (else-clause . 0)
		   (do-while-closure . 0)
		   (access-label . -)
		   (case-label . 0)
		   (substatement . +)
		   (statement-case-intro . +)
		   (statement . 0)
		   (brace-entry-open . 0)
		   (brace-list-entry . 0)
		   (brace-list-close . 0)
		   (block-close . 0)
		   (block-open . 0)
		   (inher-cont . c-lineup-multi-inher)
		   (inher-intro . +)
		   (member-init-cont . c-lineup-multi-inher)
		   (member-init-intro . +)
		   (annotation-var-cont . +)
		   (annotation-top-cont . 0)
		   (topmost-intro . 0)
		   (knr-argdecl . 0)
		   (func-decl-cont . +)
		   (inline-close . 0)
		   (class-close . 0)
		   (class-open . 0)
		   (defun-block-intro . +)
		   (defun-close . 0)
		   (defun-open . 0)
		   (c . c-lineup-C-comments)
		   (string . c-lineup-dont-change)
		   (topmost-intro-cont
		    first
		    c-lineup-topmost-intro-cont
		    c-lineup-gnu-DEFUN-intro-cont
		    )
		   (brace-list-intro
		    first
		    c-lineup-2nd-brace-entry-in-arglist
		    c-lineup-class-decl-init-+
		    +
		    )
		   (brace-list-open . +)
		   (inline-open . 0)
		   (arglist-close . c-lineup-arglist)
		   (arglist-intro . c-lineup-arglist-intro-after-paren)
		   (statement-cont . +)
		   (statement-case-open . +)
		   (label . 0)
		   (substatement-label . 0)
		   (substatement-open . +)
		   (knr-argdecl-intro . 5)
		   (statement-block-intro . +)
		   )
 c-buffer-is-cc-mode 'c-mode
 c-tab-always-indent t
 c-syntactic-indentation t
 c-syntactic-indentation-in-macros t
 c-ignore-auto-fill '(string cpp code)
 c-auto-align-backslashes t
 c-backspace-function 'backward-delete-char-untabify
 c-delete-function 'delete-char
 c-electric-pound-behavior nil
 c-default-style '((java-mode . "java") (awk-mode . "awk") (other . "gnu"))
 c-enable-xemacs-performance-kludge-p nil
 c-old-style-variable-behavior nil
 defun-prompt-regexp nil
 tab-width 8
 comment-column 32
 parse-sexp-ignore-comments t
 parse-sexp-lookup-properties t
 auto-fill-function nil
 comment-multi-line t
 comment-start-skip "\\(//+\\|/\\*+\\)\\s *"
 fill-prefix nil
 fill-column 70
 paragraph-start "[ 	]*\\(//+\\|\\**\\)[ 	]*$\\|^\f"
 adaptive-fill-mode t
 adaptive-fill-regexp "[ 	]*\\(//+\\|\\**\\)[ 	]*\\([ 	]*\\([-–!|#%;>*·•‣⁃◦]+[ 	]*\\)*\\)"
 )

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Basil.

On Tue, Apr 30, 2019 at 02:51:03 +0100, Basil L. Contovounesios wrote:
> Eli Zaretskii <eliz@gnu.org> writes:

> >> Date: Wed, 25 Oct 2017 19:11:37 +0000
> >> Cc: 28850@debbugs.gnu.org
> >> From: Alan Mackenzie <acm@muc.de>

> >> Actually, it wasn't that difficult to amend that form generator.  Would
> >> you please try out the patch below, which should apply cleanly to
> >> master.

> > I think you've solved the problem, because I let Emacs run idle for 10
> > hours, and it didn't hit this error even once.

> It seems to have returned in some way.  I can't reproduce this on Emacs
> 26, but on latest master, the following steps:

> 0. emacs -Q
> 1. (progn (setq debug-on-error t)
>           (setq jit-lock-stealth-nice nil)
>           (setq jit-lock-stealth-time 0)
>           (find-function #'next-property-change))
> 2. C-x C-e

> almost immediately lead to the following backtrace:

> Debugger entered--Lisp error: (error "Invalid search bound (wrong side of point)")
>   search-forward-regexp("\\<\\(\\(?:enum\\)\\)\\>[^][{};/#=]*{" 1673 t)
>   c-font-lock-enum-body(1673)
>   font-lock-fontify-keywords-region(1123 1673 nil)
>   font-lock-default-fontify-region(1123 1673 nil)
>   c-font-lock-fontify-region(1173 1673 nil)
>   font-lock-fontify-region(1173 1673)
>   #f(compiled-function (fun) #<bytecode 0x1565bb8a9581>)(font-lock-fontify-region)
>   run-hook-wrapped(#f(compiled-function (fun) #<bytecode 0x1565bb8a9581>) font-lock-fontify-region)
>   jit-lock--run-functions(1173 1673)
>   jit-lock-fontify-now(1173 1673)
>   jit-lock-stealth-fontify(t)
>   apply(jit-lock-stealth-fontify t)
>   timer-event-handler([t 0 0 974323 nil jit-lock-stealth-fontify (t) idle 261000])

Yes, I see this too, on master.  However, I don't see it on Emacs 26.2,
even while running an up to date CC Mode.  So I think it's likely to be
the breaking of some (possibly implicit) interface requirement with CC Mode.

> I'm not sure if this says anything, but when the *Backtrace* buffer is
> displayed, the textprop.c buffer is marked as modified.  Could this be
> related to the before/after change machinery?

It could.  There's a macro in CC Mode, c-tentative-buffer-changes, which
executes a ,@body, then undoes its changes when the result of ,@body is
nil.  Possibly the exception happened there.

> A similar error I occasionally see, but have not yet figured out how to
> reproduce:

>   Error during redisplay: (jit-lock-function 19569)
>     signaled (error "Invalid search bound (wrong side of point)")

Maybe that's the same bug.  :-)

I'll look into this problem with find-function.

> Thanks,

> -- 
> Basil

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello again, Basil.

On Tue, Apr 30, 2019 at 02:51:03 +0100, Basil L. Contovounesios wrote:
> Eli Zaretskii <eliz@gnu.org> writes:

> >> Date: Wed, 25 Oct 2017 19:11:37 +0000
> >> Cc: 28850@debbugs.gnu.org
> >> From: Alan Mackenzie <acm@muc.de>

> >> Actually, it wasn't that difficult to amend that form generator.  Would
> >> you please try out the patch below, which should apply cleanly to
> >> master.

> > I think you've solved the problem, because I let Emacs run idle for 10
> > hours, and it didn't hit this error even once.

> It seems to have returned in some way.  I can't reproduce this on Emacs
> 26, but on latest master, the following steps:

> 0. emacs -Q
> 1. (progn (setq debug-on-error t)
>           (setq jit-lock-stealth-nice nil)
>           (setq jit-lock-stealth-time 0)
>           (find-function #'next-property-change))
> 2. C-x C-e

> almost immediately lead to the following backtrace:

> Debugger entered--Lisp error: (error "Invalid search bound (wrong side of point)")
>   search-forward-regexp("\\<\\(\\(?:enum\\)\\)\\>[^][{};/#=]*{" 1673 t)
>   c-font-lock-enum-body(1673)
>   font-lock-fontify-keywords-region(1123 1673 nil)
>   font-lock-default-fontify-region(1123 1673 nil)
>   c-font-lock-fontify-region(1173 1673 nil)
>   font-lock-fontify-region(1173 1673)
>   #f(compiled-function (fun) #<bytecode 0x1565bb8a9581>)(font-lock-fontify-region)
>   run-hook-wrapped(#f(compiled-function (fun) #<bytecode 0x1565bb8a9581>) font-lock-fontify-region)
>   jit-lock--run-functions(1173 1673)
>   jit-lock-fontify-now(1173 1673)
>   jit-lock-stealth-fontify(t)
>   apply(jit-lock-stealth-fontify t)
>   timer-event-handler([t 0 0 974323 nil jit-lock-stealth-fontify (t) idle 261000])

The cause of this is a (search-forward-regexp <regexp> limit t) in a loop, where
the code fails to check (< (point) limit) at the start.  In textprop.c in
master, it so happens that the last successful iteration of the loop
leaves point beyond limit.  So we get the exception.

This is easy to correct, but first I'm going to check for all the other
places where the same mistake occurs.  Expect a patch soon!

[ .... ]

> Thanks,

> -- 
> Basil

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Basil L. Contovounesios]

Alan Mackenzie <acm@muc.de> writes:

> The cause of this is a (search-forward-regexp <regexp> limit t) in a loop, where
> the code fails to check (< (point) limit) at the start.  In textprop.c in
> master, it so happens that the last successful iteration of the loop
> leaves point beyond limit.  So we get the exception.
>
> This is easy to correct, but first I'm going to check for all the other
> places where the same mistake occurs.  Expect a patch soon!

How exciting!  Thanks for looking into this so quickly.

-- 
Basil

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello Basil.

On Tue, Apr 30, 2019 at 13:57:16 +0100, Basil L. Contovounesios wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > The cause of this is a (search-forward-regexp <regexp> limit t) in a loop, where
> > the code fails to check (< (point) limit) at the start.  In textprop.c in
> > master, it so happens that the last successful iteration of the loop
> > leaves point beyond limit.  So we get the exception.

> > This is easy to correct, but first I'm going to check for all the other
> > places where the same mistake occurs.  Expect a patch soon!

> How exciting!  Thanks for looking into this so quickly.

I've just committed a patch to all the usual places (including the
savannah master) which I'm confident will have fixed the bug.  Feel free
to test this (and tell me what isn't quite fixed ;-).

I'm just thinking, maybe this fix should have gone into the emacs-26
branch.  Not sure about that.

> -- 
> Basil

-- 
Alan Mackenzie (Nuremberg, Geramny).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Basil L. Contovounesios]

Alan Mackenzie <acm@muc.de> writes:

> Hello Basil.
>
> On Tue, Apr 30, 2019 at 13:57:16 +0100, Basil L. Contovounesios wrote:
>> Alan Mackenzie <acm@muc.de> writes:
>
>> > The cause of this is a (search-forward-regexp <regexp> limit t) in a loop, where
>> > the code fails to check (< (point) limit) at the start.  In textprop.c in
>> > master, it so happens that the last successful iteration of the loop
>> > leaves point beyond limit.  So we get the exception.
>
>> > This is easy to correct, but first I'm going to check for all the other
>> > places where the same mistake occurs.  Expect a patch soon!
>
>> How exciting!  Thanks for looking into this so quickly.
>
> I've just committed a patch to all the usual places (including the
> savannah master) which I'm confident will have fixed the bug.  Feel free
> to test this (and tell me what isn't quite fixed ;-).

Thanks, I've had no problems so far.  I definitely can't reproduce the
error in the OP any more, so you can close the report again as far as
I'm concerned.

> I'm just thinking, maybe this fix should have gone into the emacs-26
> branch.  Not sure about that.

The patch looks unproblematic enough to me, but that is, of course,
Eli's call.

Thanks,

-- 
Basil

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Cc: Eli Zaretskii <eliz@gnu.org>,  <28850@debbugs.gnu.org>
> Date: Tue, 30 Apr 2019 14:44:27 +0100
> 
> > I'm just thinking, maybe this fix should have gone into the emacs-26
> > branch.  Not sure about that.
> 
> The patch looks unproblematic enough to me, but that is, of course,
> Eli's call.

Btw, I'm not sure I understand the need for calling (point) in that
patch.  Both functions return the value of point, and they both return
LIMIT if the fail to find a match, so just testing the return value
against LIMIT should be enough, no?

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Eli.

On Tue, Apr 30, 2019 at 18:35:15 +0300, Eli Zaretskii wrote:
> > From: "Basil L. Contovounesios" <contovob@tcd.ie>
> > Cc: Eli Zaretskii <eliz@gnu.org>,  <28850@debbugs.gnu.org>
> > Date: Tue, 30 Apr 2019 14:44:27 +0100
 
> > > I'm just thinking, maybe this fix should have gone into the emacs-26
> > > branch.  Not sure about that.

> > The patch looks unproblematic enough to me, but that is, of course,
> > Eli's call.

> Btw, I'm not sure I understand the need for calling (point) in that
> patch.  Both functions return the value of point, and they both return
> LIMIT if they fail to find a match, so just testing the return value
> against LIMIT should be enough, no?

What happens is the search-forward-regexp finds a match entirely before
LIMIT, but the body of the loop processes the entire construct
introduced by that match and leaves point at the end of the construct.
This can be after LIMIT.  Hence the check on (point) before the
search-forward-regexp.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Basil.

On Tue, Apr 30, 2019 at 14:44:27 +0100, Basil L. Contovounesios wrote:
> Alan Mackenzie <acm@muc.de> writes:

[ .... ]

> > I've just committed a patch to all the usual places (including the
> > savannah master) which I'm confident will have fixed the bug.  Feel free
> > to test this (and tell me what isn't quite fixed ;-).

> Thanks, I've had no problems so far.  I definitely can't reproduce the
> error in the OP any more, so you can close the report again as far as
> I'm concerned.

OK, that's great.  The bug which Eli reported and investigated has been
fixed, too, so I'm closing this bug report (again).

> -- 
> Basil

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Basil L. Contovounesios]

Alan Mackenzie <acm@muc.de> writes:

> On Tue, Apr 30, 2019 at 14:44:27 +0100, Basil L. Contovounesios wrote:
>> Alan Mackenzie <acm@muc.de> writes:
>
> [ .... ]
>
>> > I've just committed a patch to all the usual places (including the
>> > savannah master) which I'm confident will have fixed the bug.  Feel free
>> > to test this (and tell me what isn't quite fixed ;-).
>
>> Thanks, I've had no problems so far.  I definitely can't reproduce the
>> error in the OP any more, so you can close the report again as far as
>> I'm concerned.
>
> OK, that's great.  The bug which Eli reported and investigated has been
> fixed, too, so I'm closing this bug report (again).

Thanks again to both of ye!

-- 
Basil

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> Date: Tue, 30 Apr 2019 13:32:03 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, 28850@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> I'm just thinking, maybe this fix should have gone into the emacs-26
> branch.  Not sure about that.

Didn't you say that the problem you fixed didn't exist in Emacs 26.2?

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Eli.

On Tue, Apr 30, 2019 at 18:30:37 +0300, Eli Zaretskii wrote:
> > Date: Tue, 30 Apr 2019 13:32:03 +0000
> > Cc: Eli Zaretskii <eliz@gnu.org>, 28850@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > I'm just thinking, maybe this fix should have gone into the emacs-26
> > branch.  Not sure about that.

> Didn't you say that the problem you fixed didn't exist in Emacs 26.2?

If I did, what I really meant was I hadn't seen it in 26.2.

What triggers the bug is the first enum in textprop.c straddling two
jit-lock chunks.  This seems not to happen in 26.2's textprop.c.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Cc: Eli Zaretskii <eliz@gnu.org>,  <28850@debbugs.gnu.org>
> Date: Tue, 30 Apr 2019 13:57:16 +0100
> 
> Alan Mackenzie <acm@muc.de> writes:
> 
> > The cause of this is a (search-forward-regexp <regexp> limit t) in a loop, where
> > the code fails to check (< (point) limit) at the start.  In textprop.c in
> > master, it so happens that the last successful iteration of the loop
> > leaves point beyond limit.  So we get the exception.
> >
> > This is easy to correct, but first I'm going to check for all the other
> > places where the same mistake occurs.  Expect a patch soon!
> 
> How exciting!  Thanks for looking into this so quickly.

Seconded.

FWIW, I think I see something similar in Emacs 26.2, I will try to
catch it one of these days.

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> Date: Tue, 30 Apr 2019 18:26:26 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: acm@muc.de, 28850@debbugs.gnu.org
> 
> FWIW, I think I see something similar in Emacs 26.2, I will try to
> catch it one of these days.

Done.  The error messages are slightly different: wrong-type-argument
number-or-marker-p nil.

It was very hard to debug, but eventually I succeeded to catch a
backtrace:

  Thread 1 hit Breakpoint 5, wrong_type_argument (predicate=XIL(0x9f90),
      value=XIL(0)) at data.c:146
  146     {
  #0  wrong_type_argument (predicate=XIL(0x9f90), value=XIL(0)) at data.c:146
  #1  0x01127519 in CHECK_TYPE (x=XIL(0), predicate=<optimized out>, ok=0)
      at lisp.h:625
  #2  Fadd1 (number=XIL(0)) at data.c:3125
  #3  0x011429ba in eval_sub (form=XIL(0xc000000018bb6b60)) at eval.c:2241
  #4  0x01142768 in eval_sub (form=XIL(0xc000000018bb6b40)) at eval.c:2229
  #5  0x01142ee0 in Fprogn (body=<optimized out>) at eval.c:459
  #6  0x0113af02 in Fsave_excursion (args=XIL(0xc000000018bb6af0))
      at editfns.c:1050
  #7  0x01142a9a in eval_sub (form=XIL(0xc000000018bb6b20)) at eval.c:2193
  #8  0x0114300b in Fprogn (body=<optimized out>) at eval.c:459
  #9  Fcond (args=<optimized out>) at eval.c:439
  #10 0x01142a9a in eval_sub (form=XIL(0xc000000018bb6950)) at eval.c:2193
  #11 0x01146b24 in Fsetq (args=XIL(0xc000000018bb6930)) at eval.c:517
  #12 0x01142a9a in eval_sub (form=XIL(0xc000000018bb6940)) at eval.c:2193
  #13 0x01147612 in Fprogn (body=<optimized out>) at eval.c:459
  #14 Flet (args=<optimized out>) at eval.c:973
  #15 0x01142a9a in eval_sub (form=XIL(0xc000000018bb66d0)) at eval.c:2193
  #16 0x01142f53 in Fprogn (body=<optimized out>) at eval.c:459
  #17 Fif (args=XIL(0xc000000018b9ff50)) at eval.c:415
  #18 0x01142a9a in eval_sub (form=XIL(0xc000000018b9ffa0)) at eval.c:2193
  #19 0x01142ee0 in Fprogn (body=<optimized out>) at eval.c:459
  #20 0x01141574 in internal_catch (tag=XIL(0x809f040),
      func=func@entry=0x1142eb2 <Fprogn>, arg=XIL(0xc000000018ba0490))
      at eval.c:1101
  #21 0x0114782a in Fcatch (args=XIL(0xc000000018ba04e0)) at eval.c:1078
  #22 0x01142a9a in eval_sub (form=XIL(0xc000000018ba04f0)) at eval.c:2193
  #23 0x011477e1 in Fwhile (args=XIL(0xc000000018b1c810)) at eval.c:989
  #24 0x01142a9a in eval_sub (form=XIL(0xc000000018b1c840)) at eval.c:2193
  #25 0x01142f53 in Fprogn (body=<optimized out>) at eval.c:459
  #26 Fif (args=XIL(0xc000000018b7a4b0)) at eval.c:415
  #27 0x01142a9a in eval_sub (form=XIL(0xc000000018b7a4c0)) at eval.c:2193
  #28 0x01142ee0 in Fprogn (body=<optimized out>) at eval.c:459
  #29 0x0113dc1a in Fsave_restriction (body=XIL(0xc000000018bb5160))
      at editfns.c:3990
  #30 0x01142a9a in eval_sub (form=XIL(0xc000000018bb5170)) at eval.c:2193
  #31 0x01147612 in Fprogn (body=<optimized out>) at eval.c:459
  #32 Flet (args=<optimized out>) at eval.c:973
  #33 0x01142a9a in eval_sub (form=XIL(0xc000000018625d50)) at eval.c:2193
  #34 0x0114345d in Fprogn (body=<optimized out>) at eval.c:459
  #35 funcall_lambda (fun=XIL(0xc000000018625d40), nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88bbf0) at eval.c:3052
  #36 0x0114215a in apply_lambda (fun=<optimized out>, args=<optimized out>,
      count=<optimized out>, count@entry=84) at eval.c:2913
  #37 0x0114263a in eval_sub (form=XIL(0xc00000001c7bd2b0)) at eval.c:2316
  #38 0x01146b24 in Fsetq (args=XIL(0xc00000001c7bd2c0)) at eval.c:517
  #39 0x01142a9a in eval_sub (form=XIL(0xc00000001c7bd2d0)) at eval.c:2193
  #40 0x01142768 in eval_sub (form=XIL(0xc00000001c7bd2e0)) at eval.c:2229
  #41 0x01142768 in eval_sub (form=XIL(0xc00000001c7bd2f0)) at eval.c:2229
  #42 0x01142e9e in Fand (args=<optimized out>) at eval.c:393
  #43 0x01142a9a in eval_sub (form=XIL(0xc00000001c7c5550)) at eval.c:2193
  #44 0x011477e1 in Fwhile (args=XIL(0xc00000001c7c5380)) at eval.c:989
  #45 0x01142a9a in eval_sub (form=XIL(0xc00000001c7c5390)) at eval.c:2193
  #46 0x01147612 in Fprogn (body=<optimized out>) at eval.c:459
  #47 Flet (args=<optimized out>) at eval.c:973
  #48 0x01142a9a in eval_sub (form=XIL(0xc00000001c7c52c0)) at eval.c:2193
  #49 0x01147233 in Fprogn (body=<optimized out>) at eval.c:459
  #50 FletX (args=XIL(0xc00000001c7c3ed0)) at eval.c:904
  #51 0x01142a9a in eval_sub (form=XIL(0xc00000001c7c3ec0)) at eval.c:2193
  #52 0x01142ee0 in Fprogn (body=<optimized out>) at eval.c:459
  #53 0x01141574 in internal_catch (tag=XIL(0xfeb19468),
      func=func@entry=0x1142eb2 <Fprogn>, arg=XIL(0xc00000001c7c3e90))
      at eval.c:1101
  #54 0x0114782a in Fcatch (args=XIL(0xc00000001c7c3ea0)) at eval.c:1078
  #55 0x01142a9a in eval_sub (form=XIL(0xc00000001c7c3eb0)) at eval.c:2193
  #56 0x0114345d in Fprogn (body=<optimized out>) at eval.c:459
  #57 funcall_lambda (fun=XIL(0xc00000001c7c3e80), nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88c810) at eval.c:3052
  #58 0x011437c9 in Ffuncall (nargs=2, args=args@entry=0x88c808) at eval.c:2790
  #59 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=0,
      args=<optimized out>, args@entry=0x0) at bytecode.c:630
  #60 0x01143291 in funcall_lambda (fun=XIL(0xa00000000adfab48),
      nargs=nargs@entry=1, arg_vector=arg_vector@entry=0x88cb00) at eval.c:3059
  #61 0x011437c9 in Ffuncall (nargs=2, args=args@entry=0x88caf8) at eval.c:2790
  #62 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=0,
      args=<optimized out>, args@entry=0x0) at bytecode.c:630
  #63 0x01143291 in funcall_lambda (fun=XIL(0xa000000001324700),
      nargs=nargs@entry=3, arg_vector=arg_vector@entry=0x88d0c0) at eval.c:3059
  #64 0x011437c9 in Ffuncall (nargs=4, args=args@entry=0x88d0b8) at eval.c:2790
  #65 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=0,
      args=<optimized out>, args@entry=0x0) at bytecode.c:630
  #66 0x01143291 in funcall_lambda (fun=XIL(0xa000000001323638),
      nargs=nargs@entry=3, arg_vector=arg_vector@entry=0x88d410) at eval.c:3059
  #67 0x011437c9 in Ffuncall (nargs=4, args=args@entry=0x88d408) at eval.c:2790
  #68 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=0,
      args=<optimized out>, args@entry=0x0) at bytecode.c:630
  #69 0x01143291 in funcall_lambda (fun=XIL(0xa00000000ae74d20),
      nargs=nargs@entry=3, arg_vector=arg_vector@entry=0x88d740) at eval.c:3059
  #70 0x011437c9 in Ffuncall (nargs=4, args=args@entry=0x88d738) at eval.c:2790
  #71 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=0,
      args=<optimized out>, args@entry=0x0) at bytecode.c:630
  #72 0x01143291 in funcall_lambda (fun=XIL(0xa000000001323038),
      nargs=nargs@entry=2, arg_vector=arg_vector@entry=0x88d9e8) at eval.c:3059
  #73 0x011437c9 in Ffuncall (nargs=3, args=args@entry=0x88d9e0) at eval.c:2790
  #74 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=1,
      args=<optimized out>, args@entry=0x88de08) at bytecode.c:630
  #75 0x011433ec in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88de08) at eval.c:2977
  #76 0x011437c9 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88de00)
      at eval.c:2790
  #77 0x0114393e in run_hook_wrapped_funcall (nargs=nargs@entry=2,
      args=args@entry=0x88de00) at eval.c:2503
  #78 0x01141a36 in run_hook_with_args (nargs=2, args=0x88de00,
      funcall=0x114390e <run_hook_wrapped_funcall>) at eval.c:2584
  #79 0x0114389b in Ffuncall (nargs=3, args=args@entry=0x88ddf8) at eval.c:2776
  #80 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=2,
      args=<optimized out>, args@entry=0x88e150) at bytecode.c:630
  #81 0x011433ec in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88e150) at eval.c:2977
  #82 0x011437c9 in Ffuncall (nargs=3, args=args@entry=0x88e148) at eval.c:2790
  #83 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=2,
      args=<optimized out>, args@entry=0x88e5b0) at bytecode.c:630
  #84 0x011433ec in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88e5b0) at eval.c:2977
  #85 0x011437c9 in Ffuncall (nargs=3, args=args@entry=0x88e5a8) at eval.c:2790
  #86 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=1,
      args=<optimized out>, args@entry=0x88ea60) at bytecode.c:630
  #87 0x011433ec in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88ea60) at eval.c:2977
  #88 0x011437c9 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88ea58)
      at eval.c:2790
  #89 0x01145d67 in Fapply (nargs=2, args=0x88ea58) at eval.c:2353
  #90 0x0114389b in Ffuncall (nargs=3, args=args@entry=0x88ea50) at eval.c:2776
  #91 0x01189680 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=1,
      args=<optimized out>, args@entry=0x88eda8) at bytecode.c:630
  #92 0x011433ec in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88eda8) at eval.c:2977
  #93 0x011437c9 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88eda0)
      at eval.c:2790
  #94 0x011439c7 in call1 (fn=XIL(0xcfc0), arg1=XIL(0xa00000000ae3af70))
      at eval.c:2627
  #95 0x010bfe7c in timer_check_2 (idle_timers=<optimized out>,
      timers=<optimized out>) at keyboard.c:4472
  #96 timer_check () at keyboard.c:4534
  #97 0x010c04a1 in readable_events (flags=flags@entry=1) at keyboard.c:3349
  #98 0x010c0e98 in get_input_pending (flags=flags@entry=1) at keyboard.c:6841
  #99 0x010c3a53 in detect_input_pending_run_timers (
      do_display=do_display@entry=true) at keyboard.c:9961
  #100 0x01195175 in wait_reading_process_output (time_limit=<optimized out>,
      nsecs=nsecs@entry=0, read_kbd=-1, do_display=do_display@entry=true,
      wait_for_cell=XIL(0), wait_proc=wait_proc@entry=0x0,
      just_wait_proc=just_wait_proc@entry=0) at process.c:5531
  #101 0x0100a69e in sit_for (timeout=make_number(22),
      reading=reading@entry=true, display_option=display_option@entry=1)
      at dispnew.c:5805
  #102 0x010c6c8c in read_char (commandflag=<optimized out>,
      commandflag@entry=1, map=<optimized out>, prev_event=<optimized out>,
      used_mouse_menu=<optimized out>, used_mouse_menu@entry=0x88f743,
      end_time=<optimized out>, end_time@entry=0x0) at keyboard.c:2723
  #103 0x010c8303 in read_key_sequence (keybuf=keybuf@entry=0x88f850,
      prompt=<optimized out>,
      dont_downcase_last=dont_downcase_last@entry=false,
      can_return_switch_frame=can_return_switch_frame@entry=true,
      fix_current_buffer=fix_current_buffer@entry=true,
      prevent_redisplay=prevent_redisplay@entry=false, bufsize=30)
      at keyboard.c:9157
  #104 0x010ca3e4 in command_loop_1 () at keyboard.c:1368
  #105 0x011415d2 in internal_condition_case (
      bfun=bfun@entry=0x10ca155 <command_loop_1>, handlers=XIL(0x4e90),
      hfun=hfun@entry=0x10bf040 <cmd_error>) at eval.c:1336
  #106 0x010b82a3 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110
  #107 0x01141574 in internal_catch (tag=XIL(0xd290),
      func=func@entry=0x10b827c <command_loop_2>, arg=XIL(0)) at eval.c:1101
  #108 0x010b824b in command_loop () at keyboard.c:1089
  #109 0x010bec14 in recursive_edit_1 () at keyboard.c:695
  #110 0x010bef02 in Frecursive_edit () at keyboard.c:766
  #111 0x01233073 in main (argc=<optimized out>, argv=<optimized out>)
      at emacs.c:1722

  Lisp Backtrace:
  "1+" (0x88acd0)
  "goto-char" (0x88adb8)
  "save-excursion" (0x88aee8)
  "cond" (0x88aff8)
  "setq" (0x88b138)
  "let" (0x88b2b8)
  "if" (0x88b3d8)
  "catch" (0x88b548)
  "while" (0x88b668)
  "if" (0x88b788)
  "save-restriction" (0x88b8b8)
  "let" (0x88baf8)
  "c-beginning-of-statement-1" (0x88bbf0)
  "setq" (0x88be48)
  "eq" (0x88bf38)
  "not" (0x88c028)
  "and" (0x88c138)
  "while" (0x88c258)
  "let" (0x88c3d8)
  "let*" (0x88c528)
  "catch" (0x88c698)
  "c-beginning-of-decl-1" (0x88c810)
  0xadfab48 PVEC_COMPILED
  "font-lock-fontify-keywords-region" (0x88d0c0)
  "font-lock-default-fontify-region" (0x88d410)
  "c-font-lock-fontify-region" (0x88d740)
  "font-lock-fontify-region" (0x88d9e8)
  0x18b47060 PVEC_COMPILED
  "run-hook-wrapped" (0x88de00)
  "jit-lock--run-functions" (0x88e150)
  "jit-lock-fontify-now" (0x88e5b0)
  "jit-lock-stealth-fontify" (0x88ea60)
  "apply" (0x88ea58)
  "timer-event-handler" (0x88eda8)

This comes from the following code fragment:

  (defun c-beginning-of-statement-1 (&optional lim ignore-labels
					       noerror comma-delim)
  [...]
			   ;; Just gone back over some paren block?
			   ((looking-at "\\s(")
			    (save-excursion
			      (goto-char (1+ (c-down-list-backward
					      before-sws-pos)))
			      (c-crosses-statement-barrier-p
			       (point) maybe-after-boundary-pos)))

c-down-list-backward is documented to be able to return nil, so
passing the result to 1+ is unsafe.

I cannot claim a good understanding of the code, but the following
ad-hoc patch fixes the problem for me:

--- lisp/progmodes/cc-engine.el~0	2019-01-07 16:26:06.000000000 +0200
+++ lisp/progmodes/cc-engine.el	2019-05-01 14:43:35.823456200 +0300
@@ -1130,10 +1130,12 @@
 			 ;; Just gone back over some paren block?
 			 ((looking-at "\\s(")
 			  (save-excursion
-			    (goto-char (1+ (c-down-list-backward
-					    before-sws-pos)))
-			    (c-crosses-statement-barrier-p
-			     (point) maybe-after-boundary-pos)))
+			    (let ((pos1 (c-down-list-backward
+					 before-sws-pos)))
+			      (when (number-or-marker-p pos1)
+				(goto-char (1+ pos1))
+				(c-crosses-statement-barrier-p
+				 (point) maybe-after-boundary-pos)))))
 			 ;; Just gone back over an ordinary symbol of some sort?
 			 (t (c-crosses-statement-barrier-p
 			     (point) maybe-after-boundary-pos))))

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Eli.

Sorry it's taken me a little time to answer you.  I was rather busy with
another bug.

On Wed, May 01, 2019 at 21:49:07 +0300, Eli Zaretskii wrote:
> > Date: Tue, 30 Apr 2019 18:26:26 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: acm@muc.de, 28850@debbugs.gnu.org
> > 
> > FWIW, I think I see something similar in Emacs 26.2, I will try to
> > catch it one of these days.

> Done.  The error messages are slightly different: wrong-type-argument
> number-or-marker-p nil.

> It was very hard to debug, but eventually I succeeded to catch a
> backtrace:

>   Thread 1 hit Breakpoint 5, wrong_type_argument (predicate=XIL(0x9f90),
>       value=XIL(0)) at data.c:146
>   146     {
>   #0  wrong_type_argument (predicate=XIL(0x9f90), value=XIL(0)) at data.c:146
>   #1  0x01127519 in CHECK_TYPE (x=XIL(0), predicate=<optimized out>, ok=0)
>       at lisp.h:625

[ .... ]


>   Lisp Backtrace:
>   "1+" (0x88acd0)
>   "goto-char" (0x88adb8)
>   "save-excursion" (0x88aee8)
>   "cond" (0x88aff8)
>   "setq" (0x88b138)
>   "let" (0x88b2b8)
>   "if" (0x88b3d8)
>   "catch" (0x88b548)
>   "while" (0x88b668)
>   "if" (0x88b788)
>   "save-restriction" (0x88b8b8)
>   "let" (0x88baf8)
>   "c-beginning-of-statement-1" (0x88bbf0)
>   "setq" (0x88be48)
>   "eq" (0x88bf38)
>   "not" (0x88c028)
>   "and" (0x88c138)
>   "while" (0x88c258)
>   "let" (0x88c3d8)
>   "let*" (0x88c528)
>   "catch" (0x88c698)
>   "c-beginning-of-decl-1" (0x88c810)
>   0xadfab48 PVEC_COMPILED
>   "font-lock-fontify-keywords-region" (0x88d0c0)
>   "font-lock-default-fontify-region" (0x88d410)
>   "c-font-lock-fontify-region" (0x88d740)
>   "font-lock-fontify-region" (0x88d9e8)
>   0x18b47060 PVEC_COMPILED
>   "run-hook-wrapped" (0x88de00)
>   "jit-lock--run-functions" (0x88e150)
>   "jit-lock-fontify-now" (0x88e5b0)
>   "jit-lock-stealth-fontify" (0x88ea60)
>   "apply" (0x88ea58)
>   "timer-event-handler" (0x88eda8)

> This comes from the following code fragment:

>   (defun c-beginning-of-statement-1 (&optional lim ignore-labels
> 					       noerror comma-delim)
>   [...]
> 			   ;; Just gone back over some paren block?
> 			   ((looking-at "\\s(")
> 			    (save-excursion
> 			      (goto-char (1+ (c-down-list-backward
> 					      before-sws-pos)))
> 			      (c-crosses-statement-barrier-p
> 			       (point) maybe-after-boundary-pos)))

> c-down-list-backward is documented to be able to return nil, so
> passing the result to 1+ is unsafe.

The movement function which brought point to that "\\s(" was
c-backward-sexp, i.e. (goto-char (scan-sexps (point) -1)), so the nil
that you got from c-down-list-backward "can't possibly happen".  :-(

> I cannot claim a good understanding of the code, ....

Long ago, that function, before it was rewritten by Martin Stjernholm,
carried the comment "if you think you understand this function you are
probably mistaken." ;-).

> .... but the following ad-hoc patch fixes the problem for me:

> --- lisp/progmodes/cc-engine.el~0	2019-01-07 16:26:06.000000000 +0200
> +++ lisp/progmodes/cc-engine.el	2019-05-01 14:43:35.823456200 +0300
> @@ -1130,10 +1130,12 @@
>  			 ;; Just gone back over some paren block?
>  			 ((looking-at "\\s(")
>  			  (save-excursion
> -			    (goto-char (1+ (c-down-list-backward
> -					    before-sws-pos)))
> -			    (c-crosses-statement-barrier-p
> -			     (point) maybe-after-boundary-pos)))
> +			    (let ((pos1 (c-down-list-backward
> +					 before-sws-pos)))
> +			      (when (number-or-marker-p pos1)
> +				(goto-char (1+ pos1))
> +				(c-crosses-statement-barrier-p
> +				 (point) maybe-after-boundary-pos)))))
>  			 ;; Just gone back over an ordinary symbol of some sort?
>  			 (t (c-crosses-statement-barrier-p
>  			     (point) maybe-after-boundary-pos))))

However, that nil clearly did happen, so I'll be spending some time
working out how it could have happened, and amending
c-beginning-of-statement-1 accordingly, whether with your ad-hoc patch
or otherwise.

Thanks for taking so much time and trouble to get that stack trace.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> Date: Sat, 4 May 2019 12:41:02 +0000
> Cc: contovob@tcd.ie, 28850@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> However, that nil clearly did happen, so I'll be spending some time
> working out how it could have happened, and amending
> c-beginning-of-statement-1 accordingly, whether with your ad-hoc patch
> or otherwise.

Thanks in advance.  Looking forward to seeing the fix in Emacs near
me.

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Eli.

On Sat, May 04, 2019 at 16:36:36 +0300, Eli Zaretskii wrote:
> > Date: Sat, 4 May 2019 12:41:02 +0000
> > Cc: contovob@tcd.ie, 28850@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>
> > 
> > However, that nil clearly did happen, so I'll be spending some time
> > working out how it could have happened, and amending
> > c-beginning-of-statement-1 accordingly, whether with your ad-hoc patch
> > or otherwise.

> Thanks in advance.  Looking forward to seeing the fix in Emacs near
> me.

I have a hypothesis and a patch.

The code at this point in c-beginning-of-statement-1 is in a loop, where
it goes back a sexp at a time, checking for end of statement after each
going back.  (Note, the most immediate `while' isn't this loop.)

If that sexp was a paren block, the code checks for a statement boundary
between just before the terminating paren and the starting point for the
sexp movement.  However, having gone back over this paren block, it
would be a waste of time to step forward over it again, so the function
notes the starting point in the variable before-sws-pos ("sws" =
"syntactic whitespace").  This before-sws-pos is the argument to the
c-down-list-backward which shouldn't return nil.

This goes wrong if there's a macro between the sexp starting point and
the closing paren.  The c-down-list-backward then moves into the macro
(if there's a paren there), and we have nonsense.

That's the theory.  The fix, which is now obvious, is to (setq
before-sws-pos ...) after moving back over a macro.  Perhaps I should
check the result of c-down-list-backward too, but that's to be done
after checking the current fix.

I can't actually test this myself, so would you please try out the patch
below in your test setup, and let me know whether it fixes this nasty
bug.

Thanks!


diff -r 13a9cf53cd4d cc-engine.el
--- a/cc-engine.el	Thu May 02 20:41:32 2019 +0000
+++ b/cc-engine.el	Sun May 05 08:40:14 2019 +0000
@@ -1148,6 +1148,9 @@
 			 ;; Have we moved into a macro?
 			 ((and (not macro-start)
 			       (c-beginning-of-macro))
+			  (save-excursion
+			    (c-backward-syntactic-ws)
+			    (setq before-sws-pos (point)))
 			  ;; Have we crossed a statement boundary?  If not,
 			  ;; keep going back until we find one or a "real" sexp.
 			  (and


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Eli Zaretskii]

> Date: Sun, 5 May 2019 09:06:22 +0000
> Cc: contovob@tcd.ie, 28850@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> That's the theory.  The fix, which is now obvious, is to (setq
> before-sws-pos ...) after moving back over a macro.  Perhaps I should
> check the result of c-down-list-backward too, but that's to be done
> after checking the current fix.
> 
> I can't actually test this myself, so would you please try out the patch
> below in your test setup, and let me know whether it fixes this nasty
> bug.

With this patch, the problem is gone, so I guess you hit the nail on
the head.  Thanks.

(In terms of testing, I think you can visit textprop.c, then turn on
jit-lock-stealth, and let Emacs run for a while.  Without the patch,
you should see the error message pretty soon.)

bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

[Alan Mackenzie]

Hello, Eli.

On Mon, May 06, 2019 at 18:35:28 +0300, Eli Zaretskii wrote:
> > Date: Sun, 5 May 2019 09:06:22 +0000
> > Cc: contovob@tcd.ie, 28850@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > That's the theory.  The fix, which is now obvious, is to (setq
> > before-sws-pos ...) after moving back over a macro.  Perhaps I should
> > check the result of c-down-list-backward too, but that's to be done
> > after checking the current fix.

> > I can't actually test this myself, so would you please try out the patch
> > below in your test setup, and let me know whether it fixes this nasty
> > bug.

> With this patch, the problem is gone, so I guess you hit the nail on
> the head.  Thanks.

Thank you!  I've committed that patch.

> (In terms of testing, I think you can visit textprop.c, then turn on
> jit-lock-stealth, and let Emacs run for a while.  Without the patch,
> you should see the error message pretty soon.)

Are you sure you mean textprop.c?  With jit-lock-stealth-mode enabled, I
couldn't get the error on textprop.c.  That file only has seven CPP
constructs, all right near the start of the buffer, and none of them
seem to have the closing brace/bracket/parenthesis in the right place to
trigger the error.

Anyway, it's probably not that important.  There was definitely a bug in
CC Mode at that place in c-beginning-of-statement-1, and there was
definitely a bug in CC Mode which threw an exception in your stealthy
fontification.  The patch appears to have fixed both.

I'll close bug #28850 (again), seeing as how Basil C. has confirmed that
"his" bug is fixed, too.

-- 
Alan Mackenzie (Nuremberg, Germany).