bug#37137: Setting font-lock-case-fold-search to t causes hangs on certain types in c-mode

bug#37137: Setting font-lock-case-fold-search to t causes hangs on certain types in c-mode

[Zachary Napier]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

I have in my .emacs file only these two lines:

    (custom-set-variables
     '(font-lock-keywords-case-fold-search t))

Then if I visit an empty or nonexistent C file and just type "LONG"
(in all caps or with at least one letter being capital), Emacs will
hang after typing the final G but before the G appears on
screen. Typing C-g a couple of times gets Emacs unstuck until I
continue to type. The same behavior also happens if I type "SHORT".

This happens in both Emacs 26.1 in text mode on a Linux machine and in
Emacs 25.3 in GUI mode on Windows.

After messing with GDB a bit, I am pretty sure the hang is at the
while loop in c-forward-type at cc-engine.el:7657 and its getting
stuck because looking-at matches LONG because its case insensitive,
but the c-forward-keyword-clause doesn't move forward because the
first thing it does is try to find "LONG" in c-keywords-obarray via
`intern-soft` and immediately returns because it doesn't find it
there. Here is the while loop:

    (while (progn
             (setq safe-pos (point))
             (looking-at c-opt-type-component-key))
      (when (and c-record-type-identifiers
                 (looking-at c-primitive-type-key))
        (c-record-type-id (cons (match-beginning 1)
                                (match-end 1))))
      (c-forward-keyword-clause 1))

[-- Attachment #2: Type: text/html, Size: 1537 bytes --]

bug#37137: Setting font-lock-case-fold-search to t causes hangs on certain types in c-mode

[Alan Mackenzie]

Hello, Zachary.

In article <mailman.18.1566429065.1922.bug-gnu-emacs@gnu.org> you wrote:
> [-- text/plain, encoding 7bit, charset: UTF-8, 31 lines --]

> I have in my .emacs file only these two lines:

>     (custom-set-variables
>      '(font-lock-keywords-case-fold-search t))

May I ask why you're making this setting?

I think font-lock-keywords-case-fold-search is really intended to be set
by major modes, rather than as a user customisation.

> Then if I visit an empty or nonexistent C file and just type "LONG"
> (in all caps or with at least one letter being capital), Emacs will
> hang after typing the final G but before the G appears on
> screen. Typing C-g a couple of times gets Emacs unstuck until I
> continue to type. The same behavior also happens if I type "SHORT".

Yes.  The keywords in all the CC Mode languages are case sensitive.

Almost exactly the same bug came up in 2013, when the solution was to
bind case-fold-search to nil at every "entry point" to CC Mode.  I missed
font-lock-keywords-case-fold-search at the time.

> This happens in both Emacs 26.1 in text mode on a Linux machine and in
> Emacs 25.3 in GUI mode on Windows.

> After messing with GDB a bit, I am pretty sure the hang is at the
> while loop in c-forward-type at cc-engine.el:7657 and its getting
> stuck because looking-at matches LONG because its case insensitive,
> but the c-forward-keyword-clause doesn't move forward because the
> first thing it does is try to find "LONG" in c-keywords-obarray via
> `intern-soft` and immediately returns because it doesn't find it
> there. Here is the while loop:

>     (while (progn
>              (setq safe-pos (point))
>              (looking-at c-opt-type-component-key))
>       (when (and c-record-type-identifiers
>                  (looking-at c-primitive-type-key))
>         (c-record-type-id (cons (match-beginning 1)
>                                 (match-end 1))))
>       (c-forward-keyword-clause 1))

That's an excellent piece of debugging.  :-)

I think the best fix for the bug would be for CC Mode to set
font-lock-keywords-case-fold-search explicitly to nil (buffer locally) at
C Mode (etc.) initialisation.  This would undo the effect of your
custom-set-variables form in C Mode buffers.  Would this be a problem?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#37137: Setting font-lock-case-fold-search to t causes hangs on certain types in c-mode

[Stefan Monnier]

> I have in my .emacs file only these two lines:
>
>     (custom-set-variables
>      '(font-lock-keywords-case-fold-search t))

How did you end up with that?
font-lock-keywords-case-fold-search is not a customize variable, so you
should not set it with custom-set-variables.

> Then if I visit an empty or nonexistent C file and just type "LONG"
> (in all caps or with at least one letter being capital), Emacs will
> hang after typing the final G but before the G appears on screen

Clearly not a desirable behavior, indeed, but the above var setting is
clearly a pilot error.  Of course

    (setq-default font-lock-keywords-case-fold-search t)

would give the same result, but I'd also argue that it'd be
a pilot error.


        Stefan

bug#37137: Setting font-lock-case-fold-search to t causes hangs on certain types in c-mode

[Zachary Napier]

[-- Attachment #1: Type: text/plain, Size: 1554 bytes --]

Hello Alen and Stefan,

> May I ask why you're making this setting?

> How did you end up with that?
> font-lock-keywords-case-fold-search is not a customize variable, so you
> should not set it with custom-set-variables.

I inherited this .emacs file from someone else who had this
set. I'm guessing it used to be customizable? If it really isn't
proper to customize it this way thats fine. I can disable it with
no real issues. I can just change the regexps I am using in
font-lock-add-keywords. I assumed that it could be customized
because if I did a search the customize option for it showed up,
but maybe improperly setting it with custom-set-variables just
adds it to Customize.

On Sun, Aug 25, 2019 at 11:12 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > I have in my .emacs file only these two lines:
> >
> >     (custom-set-variables
> >      '(font-lock-keywords-case-fold-search t))
>
> How did you end up with that?
> font-lock-keywords-case-fold-search is not a customize variable, so you
> should not set it with custom-set-variables.
>
> > Then if I visit an empty or nonexistent C file and just type "LONG"
> > (in all caps or with at least one letter being capital), Emacs will
> > hang after typing the final G but before the G appears on screen
>
> Clearly not a desirable behavior, indeed, but the above var setting is
> clearly a pilot error.  Of course
>
>     (setq-default font-lock-keywords-case-fold-search t)
>
> would give the same result, but I'd also argue that it'd be
> a pilot error.
>
>
>         Stefan
>
>

[-- Attachment #2: Type: text/html, Size: 2376 bytes --]

bug#37137: Setting font-lock-case-fold-search to t causes hangs on certain types in c-mode

[Stefan Monnier]

> I inherited this .emacs file from someone else who had this
> set.  I'm guessing it used to be customizable?

AFAICT it's never been custom-izable, and it really would be weird for
it to be.  I think it's more likely that this custom-set-variables was
hand-edited.

> I can just change the regexps I am using in
> font-lock-add-keywords.

That's a much safer approach.  You can also do

    (add-hook '<foo>-mode-hook
      (lambda ()
        (setq-local font-lock-case-fold-search t)))

for the specific mode where you need it: it may break the font-locking
of that mode, but if it does, at least the breakage will be limited to
that mode.


        Stefan

bug#37137: Setting font-lock-case-fold-search to t causes hangs on certain types in c-mode

[Alan Mackenzie]

Hello, Stefan and Zachary.

On Mon, Sep 02, 2019 at 08:50:12 -0400, Stefan Monnier wrote:
> > I inherited this .emacs file from someone else who had this
> > set.  I'm guessing it used to be customizable?

> AFAICT it's never been custom-izable, and it really would be weird for
> it to be.  I think it's more likely that this custom-set-variables was
> hand-edited.

That variable couldn't possibly have been customisable.  It's default
value being other than nil breaks font locking, in that it spikes its
active binding to t, regardless of the value supplied in (nth 2
font-lock-defaults).

> > I can just change the regexps I am using in
> > font-lock-add-keywords.

> That's a much safer approach.  You can also do

>     (add-hook '<foo>-mode-hook
>       (lambda ()
>         (setq-local font-lock-case-fold-search t)))

> for the specific mode where you need it: it may break the font-locking
> of that mode, but if it does, at least the breakage will be limited to
> that mode.

No.  font-lock-keywords-case-fold-search is purely a font-lock internal
variable.  Any buffer local value of it created outside of font-lock
gets ignored.  Setting its default value to anything other than nil will
break font-locking, as described above.

There is neither need nor sense in setting
font-lock-keywords-case-fold-search.  Instead, set element 2 of
font-lock-defaults to the desired setting.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).