bug#48097: 28.0.50; C code not being correctly fontified in features/native-comp

bug#48097: 28.0.50; C code not being correctly fontified in features/native-comp

[Alex Bennée]

This happens with a lot of C files but for a concrete reproducible
example from my development repository:

  https://github.com/stsquad/qemu/blob/review/qtest-accel-check-v4/tests/qtest/arm-cpu-features.c

Running:

  emacs -Q tests/qtest/arm-cpu-features.c

And witness the lack of fontification from half way through assert_error
although resp_has_props seems to recover for the function prototype and
initial variable. When I run:

 M-x font-lock-debug-fontify

The fontification fixes itself although I get a bunch of errors in
Messages:

  Error during redisplay: (jit-lock-function 2666) signaled (wrong-type-argument number-or-marker-p nil)
  Error during redisplay: (jit-lock-function 3125) signaled (wrong-type-argument number-or-marker-p nil)

In GNU Emacs 28.0.50 (build 4, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw3d scroll bars)
 of 2021-04-24 built on zen
Repository revision: fa65c044f2ebe666467166075c1507a8d0e1347f
Repository branch: feature/native-comp
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux 10 (buster)

Configured using:
 'configure --with-x-toolkit=lucid --prefix=/home/alex/src/emacs/install
 --with-modules --with-imagemagick --with-native-compilation'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2
M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP
SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XAW3D XDBE XIM XPM LUCID ZLIB

Important settings:
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Messages

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug comp comp-cstr warnings rx cl-extra
help-mode message rmc puny dired dired-loaddefs rfc822 mml mml-sec epa
derived epg epg-config gnus-util rmail rmail-loaddefs auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json map
text-property-search seq byte-opt gv bytecomp byte-compile cconv
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils time-date subr-x vc-git diff-mode easy-mmode vc-dispatcher
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs cl-loaddefs cl-lib iso-transl tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
loaddefs faces cus-face macroexp files window text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
x multi-tty make-network-process nativecomp emacs)

Memory information:
((conses 16 131404 9363)
 (symbols 48 10101 1)
 (strings 32 29919 2259)
 (string-bytes 1 1072854)
 (vectors 16 18858)
 (vector-slots 8 322162 14264)
 (floats 8 31 33)
 (intervals 56 2086 0)
 (buffers 992 13))

-- 
Alex Bennée
QEMU/KVM Hacker for Linaro

bug#48097:

[Alex Bennée]

[-- Attachment #1: Type: text/plain, Size: 233 bytes --]

I suspect this might be a duplicate of #48061 - re-building with the
current master the bug has gone away and problems with c-mode cause by
strange code would certainly explain it.

-- 
Alex Bennée
KVM/QEMU Hacker for Linaro

[-- Attachment #2: Type: text/html, Size: 361 bytes --]

bug#48097:

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Alex Bennée <alex.bennee@linaro.org> writes:

> I suspect this might be a duplicate of #48061 - re-building with the current master the bug has gone away and problems
> with c-mode cause by strange code would certainly explain it.

Hi Alex,

okay shall we close this then?

  Andrea

bug#48097:

[Alex Bennée]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

Yes, please proceed.

On Thu, 29 Apr 2021 at 13:17, Andrea Corallo <akrl@sdf.org> wrote:

> Alex Bennée <alex.bennee@linaro.org> writes:
>
> > I suspect this might be a duplicate of #48061 - re-building with the
> current master the bug has gone away and problems
> > with c-mode cause by strange code would certainly explain it.
>
> Hi Alex,
>
> okay shall we close this then?
>
>   Andrea
>


-- 
Alex Bennée
KVM/QEMU Hacker for Linaro

[-- Attachment #2: Type: text/html, Size: 888 bytes --]

bug#48097:

[Eli Zaretskii]

> Cc: 48097@debbugs.gnu.org
> Date: Thu, 29 Apr 2021 12:17:43 +0000
> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Alex Bennée <alex.bennee@linaro.org> writes:
> 
> > I suspect this might be a duplicate of #48061 - re-building with the current master the bug has gone away and problems
> > with c-mode cause by strange code would certainly explain it.
> 
> Hi Alex,
> 
> okay shall we close this then?

Please don't close yet, I still see the problem.

It is strange that Alex says bug#48061 should solve this: the fix for
that bug changed comp-cstr.el, and I have that natively-compiled after
that change.  Do I need to make sure other files are recompiled, for
example the CC mode files?

bug#48097:

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 48097@debbugs.gnu.org
>> Date: Thu, 29 Apr 2021 12:17:43 +0000
>> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> Alex Bennée <alex.bennee@linaro.org> writes:
>> 
>> > I suspect this might be a duplicate of #48061 - re-building with the current master the bug has gone away and problems
>> > with c-mode cause by strange code would certainly explain it.
>> 
>> Hi Alex,
>> 
>> okay shall we close this then?
>
> Please don't close yet, I still see the problem.
>
> It is strange that Alex says bug#48061 should solve this: the fix for
> that bug changed comp-cstr.el, and I have that natively-compiled after
> that change.  Do I need to make sure other files are recompiled, for
> example the CC mode files?

Yes, all possibly miscompiled files should be recompiled so at least
"cc-engine.el" were we know for sure `c-determine-limit-no-macro' was
miscompiled.  If the eln corresponding to "cc-engine.el" is in your
"~/.emacs.d/eln-cache" it's probably still the broken one, in this case
should be sufficient remove it and restart Emacs.

  Andrea

bug#48097:

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: alex.bennee@linaro.org, 48097@debbugs.gnu.org
> Date: Thu, 29 Apr 2021 14:26:50 +0000
> 
> > Please don't close yet, I still see the problem.
> >
> > It is strange that Alex says bug#48061 should solve this: the fix for
> > that bug changed comp-cstr.el, and I have that natively-compiled after
> > that change.  Do I need to make sure other files are recompiled, for
> > example the CC mode files?
> 
> Yes, all possibly miscompiled files should be recompiled so at least
> "cc-engine.el" were we know for sure `c-determine-limit-no-macro' was
> miscompiled.  If the eln corresponding to "cc-engine.el" is in your
> "~/.emacs.d/eln-cache" it's probably still the broken one, in this case
> should be sufficient remove it and restart Emacs.

OK, after recompiling the CC mode files, the problem is indeed gone.
Closing.

bug#48097:

[Alex Bennée]

[-- Attachment #1: Type: text/plain, Size: 1246 bytes --]

During my rebuild I did a distclean, wiped the old install directory and
also cleared the eln-cache just to make sure everything was pristine. The
function in question is reported as natively compiled:

c-determine-limit-no-macro is a native compiled Lisp function in
‘cc-engine.el’.

On Thu, 29 Apr 2021 at 14:12, Eli Zaretskii <eliz@gnu.org> wrote:

> > Cc: 48097@debbugs.gnu.org
> > Date: Thu, 29 Apr 2021 12:17:43 +0000
> > From:  Andrea Corallo via "Bug reports for GNU Emacs,
> >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> >
> > Alex Bennée <alex.bennee@linaro.org> writes:
> >
> > > I suspect this might be a duplicate of #48061 - re-building with the
> current master the bug has gone away and problems
> > > with c-mode cause by strange code would certainly explain it.
> >
> > Hi Alex,
> >
> > okay shall we close this then?
>
> Please don't close yet, I still see the problem.
>
> It is strange that Alex says bug#48061 should solve this: the fix for
> that bug changed comp-cstr.el, and I have that natively-compiled after
> that change.  Do I need to make sure other files are recompiled, for
> example the CC mode files?
>


-- 
Alex Bennée
KVM/QEMU Hacker for Linaro

[-- Attachment #2: Type: text/html, Size: 1906 bytes --]

bug#48097:

[Eli Zaretskii]

> From: Alex Bennée <alex.bennee@linaro.org>
> Date: Thu, 29 Apr 2021 11:34:39 +0100
> 
> I suspect this might be a duplicate of #48061 - re-building with the current master the bug has gone away and
> problems with c-mode cause by strange code would certainly explain it.

I still see the problem with the latest master branch, when
natively-compiled Lisp files are used.  I don't see the problem when
using byte-compiled Lisp.