bug#48447: 27.2.50; Font-lock error in temporary bibtex-mode buffer

bug#48447: 27.2.50; Font-lock error in temporary bibtex-mode buffer

[Basil L. Contovounesios]

Severity: minor

0. emacs -Q
1. C-h e
2. Insert:

(with-temp-buffer
  (insert "\
@article{key
  author={Author},
  title={Title},
  journal={Journal},
  year=2021
}\n")
  (goto-char (point-min))
  (bibtex-mode)
  (bibtex-set-dialect nil 'local)
  (bibtex-skip-to-valid-entry))

3. C-x C-e

This puts the following in *Messages*:

  Error during redisplay: (internal--syntax-propertize 25) signaled
  (error "Font-lock trying to use keywords before setting them up")

I can't reproduce this in Emacs versions 24.5.1, 25.3.1, or 26.3.

I can reproduce it on master with and without the call to
bibtex-set-dialect (in older versions omitting bibtex-set-dialect leads
to an error because bibtex-entry-head is nil instead of a string).

None of the commits touching bibtex.el between emacs-26..emacs-27 strike
me as obvious causes, so perhaps it follows a change in the font-lock
machinery?

Is this use of bibtex-skip-to-valid-entry wrong?  If so, why, and what
should be used instead?  Either way, is that error expected?

Thanks,

-- 
Basil

In GNU Emacs 27.2.50 (build 2, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw3d scroll bars)
 of 2021-05-09 built on tia
Repository revision: 4c3abb3dd105e075bf1cf55e3fe8b5ec2ac8e6cc
Repository branch: wt/emacs-27
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux bullseye/sid

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark set
Error during redisplay: (internal--syntax-propertize 25) signaled (error "Font-lock trying to use keywords before setting them up")
(1 . 85)
Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O0 -g3 -ggdb -gdwarf-4'
 --config-cache --prefix=/home/blc/.local --program-suffix=-27
 --enable-checking=yes,glyphs --enable-check-lisp-object-type
 --with-x-toolkit=lucid --with-file-notification=yes --with-x
 --with-cairo'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT
LIBOTF ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS
LIBSYSTEMD JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml mml-sec password-cache epa derived epg epg-config
gnus-util rmail rmail-loaddefs text-property-search time-date subr-x seq
byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs cl-lib
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils bibtex
easymenu tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 51935 9915)
 (symbols 48 6377 1)
 (strings 32 18852 1763)
 (string-bytes 1 575619)
 (vectors 16 9831)
 (vector-slots 8 131141 11094)
 (floats 8 26 46)
 (intervals 56 236 0)
 (buffers 1000 11))

bug#48447: 27.2.50; Font-lock error in temporary bibtex-mode buffer

[Roland Winkler]

On Sat May 15 2021 Basil L. Contovounesios wrote:
> Severity: minor
[snip] 
> I can't reproduce this in Emacs versions 24.5.1, 25.3.1, or 26.3.

Thanks, I'll take a look into this.  It may be that the code is
getting a little rusty.

bug#48447: 27.2.50; Font-lock error in temporary bibtex-mode buffer

[Lars Ingebrigtsen]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> This puts the following in *Messages*:
>
>   Error during redisplay: (internal--syntax-propertize 25) signaled
>   (error "Font-lock trying to use keywords before setting them up")

I've now fixed this in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#48447: 27.2.50; Font-lock error in temporary bibtex-mode buffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Lars Ingebrigtsen [2022-07-13 13:24:12] wrote:

> "Basil L. Contovounesios" <contovob@tcd.ie> writes:
>
>> This puts the following in *Messages*:
>>
>>   Error during redisplay: (internal--syntax-propertize 25) signaled
>>   (error "Font-lock trying to use keywords before setting them up")
>
> I've now fixed this in Emacs 29.
[...]
> @@ -3659,7 +3659,11 @@ if that value is non-nil.
>                                          ?\s)))))
>      (if (and buffer-file-name enable-local-variables)
>          (add-hook 'hack-local-variables-hook fun nil t)
> -      (funcall fun))))
> +      (funcall fun)))
> +  ;; We may be using the mode programmatically to extract data, and we
> +  ;; then need this to be set up first so that sexp-based movement
> +  ;; commands don't bug out.
> +  (font-lock-set-defaults))

Hmm... this looks like a workaround rather than a fix of the core
problem because I can't see any use of "font-lock internals" in
bibtex.el which would justify such a need.

I suspect that the problem is linked to
`syntax-propertize-via-font-lock`, so I think the fix should go there.


        Stefan

bug#48447: 27.2.50; Font-lock error in temporary bibtex-mode buffer

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Hmm... this looks like a workaround rather than a fix of the core
> problem because I can't see any use of "font-lock internals" in
> bibtex.el which would justify such a need.
>
> I suspect that the problem is linked to
> `syntax-propertize-via-font-lock`, so I think the fix should go there.

Possibly.  The backtrace is

Debugger entered--Lisp error: (error "Font-lock trying to use keywords before setting th...")
  signal(error ("Font-lock trying to use keywords before setting th..."))
  error("Font-lock trying to use keywords before setting th...")
  font-lock-compile-keywords((("^[ \11]*\\(@\\)Comment\\>" 1 '(11))) t)
  font-lock-fontify-syntactic-keywords-region(1 86)
  #f(compiled-function (start end) #<bytecode 0xabaf7b9213c487b>)(1 86)
  syntax-propertize(86)
  internal--syntax-propertize(25)
  scan-sexps(24 1)
  forward-sexp(1)
  bibtex-parse-field-string()
  bibtex-parse-field-text()
  bibtex-parse-association(bibtex-parse-field-name bibtex-parse-field-text)


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#48447: 27.2.50; Font-lock error in temporary bibtex-mode buffer

[Roland Winkler]

On Wed, Jul 13 2022, Stefan Monnier wrote:
> Hmm... this looks like a workaround rather than a fix of the core
> problem because I can't see any use of "font-lock internals" in
> bibtex.el which would justify such a need.
>
> I suspect that the problem is linked to
> `syntax-propertize-via-font-lock`, so I think the fix should go there.

I am not sure I can follow.  Are you saying the proper solution to this
bug likely needs to go outside bibtex.el but into something like
syntax-propertize-via-font-lock?  That would put this bug far beyond my
emacs hacking skills.

bug#48447: 27.2.50; Font-lock error in temporary bibtex-mode buffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Roland Winkler [2022-07-13 14:32:13] wrote:
> On Wed, Jul 13 2022, Stefan Monnier wrote:
>> Hmm... this looks like a workaround rather than a fix of the core
>> problem because I can't see any use of "font-lock internals" in
>> bibtex.el which would justify such a need.
>>
>> I suspect that the problem is linked to
>> `syntax-propertize-via-font-lock`, so I think the fix should go there.
>
> I am not sure I can follow.  Are you saying the proper solution to this
> bug likely needs to go outside bibtex.el but into something like
> syntax-propertize-via-font-lock?

Yup.

> That would put this bug far beyond my emacs hacking skills.

The upside is that it's "not your problem" any more :-)


        Stefan