bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Achim Gratz]

Publishing from Worg (the community webpages for Org) runs in batch mode
and uses cc-mode for syntax highlighting for some source code examples.
While testing the publishing process with Emacs 24 I've come across an
apparent regression in cc-mode: it tries to use the initial value for
c-standard-font-lock-fontify-region-function, which happens to be nil to
do a funcall and errors out since the function slot of that variable is
void.  I've worked around the error by adding this to the init file:

--8<---------------cut here---------------start------------->8---
;; to have things work correctly in batch-mode
(require 'font-lock)
(require 'cc-mode)
(c-after-font-lock-init)
--8<---------------cut here---------------end--------------->8---

I cannot tell if that works correctly in all cases or why this is
necessary, but simply by switching to Emacs 23 it all works correctly
without it.

The publishing setup, if required for reproduction of the bug, is
described in more detail here:

http://thread.gmane.org/gmane.emacs.orgmode/71405/focus=71466


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Glenn Morris]

Minimal example:

./src/emacs -Q -batch --eval \
  '(progn (find-file "./src/emacs.c") (font-lock-fontify-buffer))'

Ie, a use of font-lock that does not first call font-lock-mode-hook.

This was already fixed once:
http://lists.gnu.org/archive/html/emacs-diffs/2012-02/msg00114.html

but reverted:
http://lists.gnu.org/archive/html/emacs-diffs/2012-02/msg00348.html

There was some kerfluffle about it:
http://lists.gnu.org/archive/html/emacs-devel/2012-02/msg00380.html


To me, c-standard-font-lock-fontify-region-function seems pointless in
Emacs, where font-lock is preloaded since 22.1.

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Glenn Morris]

Glenn Morris wrote:

> To me, c-standard-font-lock-fontify-region-function seems pointless in
> Emacs, where font-lock is preloaded since 22.1.

Even it is wasn't preloaded, I still don't see anything wrong with the
following. cc-mode only changes the buffer-local value.

*** lisp/progmodes/cc-mode.el	2013-04-15 16:10:24 +0000
--- lisp/progmodes/cc-mode.el	2013-05-01 03:34:59 +0000
***************
*** 1160,1168 ****
    ;; `c-set-fl-decl-start' for the detailed functionality.
    (cons (c-set-fl-decl-start beg) end))
  
- (defvar c-standard-font-lock-fontify-region-function nil
-   "Standard value of `font-lock-fontify-region-function'")
- 
  (defun c-font-lock-fontify-region (beg end &optional verbose)
    ;; Effectively advice around `font-lock-fontify-region' which extends the
    ;; region (BEG END), for example, to avoid context fontification chopping
--- 1160,1165 ----
***************
*** 1187,1193 ****
  		  (setq new-region (funcall fn new-beg new-end))
  		  (setq new-beg (car new-region)  new-end (cdr new-region)))
  		c-before-context-fontification-functions))))
!     (funcall c-standard-font-lock-fontify-region-function
  	     new-beg new-end verbose)))
  
  (defun c-after-font-lock-init ()
--- 1184,1190 ----
  		  (setq new-region (funcall fn new-beg new-end))
  		  (setq new-beg (car new-region)  new-end (cdr new-region)))
  		c-before-context-fontification-functions))))
!     (funcall (default-value 'font-lock-fontify-region-function)
  	     new-beg new-end verbose)))
  
  (defun c-after-font-lock-init ()
***************
*** 1195,1203 ****
    ;; function will get executed before the font-lock one.  Amongst other
    ;; things.
    (remove-hook 'after-change-functions 'c-after-change t)
!   (add-hook 'after-change-functions 'c-after-change nil t)
!   (setq c-standard-font-lock-fontify-region-function
! 	(default-value 'font-lock-fontify-region-function)))
  
  (defun c-font-lock-init ()
    "Set up the font-lock variables for using the font-lock support in CC Mode.
--- 1192,1198 ----
    ;; function will get executed before the font-lock one.  Amongst other
    ;; things.
    (remove-hook 'after-change-functions 'c-after-change t)
!   (add-hook 'after-change-functions 'c-after-change nil t))
  
  (defun c-font-lock-init ()
    "Set up the font-lock variables for using the font-lock support in CC Mode.

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Achim Gratz]

Glenn Morris writes:
> Minimal example:
>
> ./src/emacs -Q -batch --eval \
>   '(progn (find-file "./src/emacs.c") (font-lock-fontify-buffer))'
>
> Ie, a use of font-lock that does not first call font-lock-mode-hook.

Why is that hook skipped?  And what would be the correct workaround?

> This was already fixed once:
> http://lists.gnu.org/archive/html/emacs-diffs/2012-02/msg00114.html
>
> but reverted:
> http://lists.gnu.org/archive/html/emacs-diffs/2012-02/msg00348.html
>
> There was some kerfluffle about it:
> http://lists.gnu.org/archive/html/emacs-devel/2012-02/msg00380.html

I see… maybe a consensus might be reached this time.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Alan Mackenzie]

Achim Gratz <Stromeko@nexgo.de> wrote:

> Publishing from Worg (the community webpages for Org) runs in batch mode
> and uses cc-mode for syntax highlighting for some source code examples.
> While testing the publishing process with Emacs?24 I've come across an
> apparent regression in cc-mode: it tries to use the initial value for
> c-standard-font-lock-fontify-region-function, which happens to be nil to
> do a funcall and errors out since the function slot of that variable is
> void.  I've worked around the error by adding this to the init file:

> --8<---------------cut here---------------start------------->8---
> ;; to have things work correctly in batch-mode
> (require 'font-lock)
> (require 'cc-mode)
> (c-after-font-lock-init)
> --8<---------------cut here---------------end--------------->8---

> I cannot tell if that works correctly in all cases or why this is
> necessary, but simply by switching to Emacs?23 it all works correctly
> without it.

Yes, the c-after-font-lock-init call will initialise everything properly,
though it's more a workaround than a solution.

Font Lock Mode is a minor mode, and part of its initialisation is calling
font-lock-mode-hook.  (font-lock-mode-hook is here c-after-font-lock-init.)
If you run font-lock-fontify-buffer (etc.) without fully initialising Font
Lock Mode, you're liable to run into bugs.  If that were all there were
to it, I'd have nothing more to say, but sadly it's not so simple.

Font Lock Mode is, by default disabled in batch mode.  If you enable it,
this also enables jit-lock-mode (which is surely wrong), which prevents
any fontification actually taking place, since the buffer is never
displayed on the screen to trigger the fontification.

I think the following sequence of commands would fontify the buffer
properly in batch mode:
    (c-mode)
    (setq font-lock-support-mode nil) ; disable jit-lock-mode
    (font-lock-mode 1)
.

However, you'll probably prefer to carry on with using Font Lock Mode
uninitialised.  ;-)  I think Glenn's patch achieves this painlessly.

> Regards,
> Achim.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Achim Gratz]

Alan Mackenzie writes:
> Yes, the c-after-font-lock-init call will initialise everything properly,
> though it's more a workaround than a solution.

So what would be the solution, then?

> Font Lock Mode is a minor mode, and part of its initialisation is calling
> font-lock-mode-hook.  (font-lock-mode-hook is here c-after-font-lock-init.)
> If you run font-lock-fontify-buffer (etc.) without fully initialising Font
> Lock Mode, you're liable to run into bugs.  If that were all there were
> to it, I'd have nothing more to say, but sadly it's not so simple.

Well, Org requires font-lock, it switches font-lock off and on when it
changes font-lock settings and it uses it for fontification with
whatever major mode the content wants to have.  If there is a code-path
that enters cc-mode in uninitialized state, then shouldn't cc-mode check
for nil instead of crashing Emacs by blindly assuming it can funcall the
contents of that variable?

> Font Lock Mode is, by default disabled in batch mode.  If you enable it,
> this also enables jit-lock-mode (which is surely wrong), which prevents
> any fontification actually taking place, since the buffer is never
> displayed on the screen to trigger the fontification.

Since Org can use the results of the fontification, some of it must
happen anyway?  Besides, even if I stop jit-lock-mode from being
used (by adding it to font-lock-inhibit-thing-lock), cc-mode still
throws an error.

> I think the following sequence of commands would fontify the buffer
> properly in batch mode:
>     (c-mode)
>     (setq font-lock-support-mode nil) ; disable jit-lock-mode
>     (font-lock-mode 1)

I've tried to put these (without switching to c-mode since selecting the
mode is done elsewhere) into various places and still get the same error
from cc-mode.

> However, you'll probably prefer to carry on with using Font Lock Mode
> uninitialised.  ;-)  I think Glenn's patch achieves this painlessly.

I'd prefer if fontlock-fontify-* would work in batch mode without
workarounds.  I'll have no problem sticking (when noninteractive ...)
into the intialization or Org itself if that's the ticket.  I still
don't have a clue where that should be done.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Alan Mackenzie]

Achim Gratz <Stromeko@nexgo.de> wrote:
> Alan Mackenzie writes:
>> Yes, the c-after-font-lock-init call will initialise everything properly,
>> though it's more a workaround than a solution.

> So what would be the solution, then?

>> Font Lock Mode is a minor mode, and part of its initialisation is calling
>> font-lock-mode-hook.  (font-lock-mode-hook is here c-after-font-lock-init.)
>> If you run font-lock-fontify-buffer (etc.) without fully initialising Font
>> Lock Mode, you're liable to run into bugs.  If that were all there were
>> to it, I'd have nothing more to say, but sadly it's not so simple.

> Well, Org requires font-lock, it switches font-lock off and on when it
> changes font-lock settings and it uses it for fontification with
> whatever major mode the content wants to have.

OK.  Could you possibly give the exact sequence of Font Lock and CC Mode
calls which lead to the error?

> If there is a code-path that enters cc-mode in uninitialized state, then
> shouldn't cc-mode check for nil instead of crashing Emacs by blindly
> assuming it can funcall the contents of that variable?

The variable is (or should be) initialised by font-lock-mode-hook calling
c-after-font-lock-init.  This should happen when font-lock-mode is called.
The question is why the switching on of Font Lock Mode here fails to do
this.

>> Font Lock Mode is, by default disabled in batch mode.  If you enable it,
>> this also enables jit-lock-mode (which is surely wrong), which prevents
>> any fontification actually taking place, since the buffer is never
>> displayed on the screen to trigger the fontification.

> Since Org can use the results of the fontification, some of it must
> happen anyway?  Besides, even if I stop jit-lock-mode from being
> used (by adding it to font-lock-inhibit-thing-lock), cc-mode still
> throws an error.

>> I think the following sequence of commands would fontify the buffer
>> properly in batch mode:
>>     (c-mode)
>>     (setq font-lock-support-mode nil) ; disable jit-lock-mode
>>     (font-lock-mode 1)

> I've tried to put these (without switching to c-mode since selecting the
> mode is done elsewhere) into various places and still get the same error
> from cc-mode.

OK.

>> However, you'll probably prefer to carry on with using Font Lock Mode
>> uninitialised.  ;-)  I think Glenn's patch achieves this painlessly.

> I'd prefer if fontlock-fontify-* would work in batch mode without
> workarounds.  I'll have no problem sticking (when noninteractive ...)
> into the intialization or Org itself if that's the ticket.  I still
> don't have a clue where that should be done.

I'd like to track this down, too.

> Regards,
> Achim.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Glenn Morris]

Alan Mackenzie wrote:

> However, you'll probably prefer to carry on with using Font Lock Mode
> uninitialised.  ;-)  I think Glenn's patch achieves this painlessly.

That patch seems like TRT to me anyway.

What problem is c-standard-font-lock-fontify-region-function supposed to
solve? Someone changing the default value of
font-lock-fontify-region-function in between font-lock-mode-hook being
run and font-lock-fontify-region being called?
That shouldn't happen, only the buffer-local value should ever be
changed. It's not a user-variable.
(I actually think you could safely just call
font-lock-default-fontify-region .)

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Achim Gratz]

Alan Mackenzie writes:
> OK.  Could you possibly give the exact sequence of Font Lock and CC Mode
> calls which lead to the error?

If you can tell me how to obtain a trace in batch mode?  Or if you
prefer to do this yourself, here's the call that makes the problem:

emacs -batch -Q -l worgtest-local-init.el -l worgtest-init.el \
 org-hacks.org -f toggle-debug-on-error -f org-html-export-to-html

You need a local clone of Worg (or at least these three files), a recent
Org mode and deactivate the workaround in worgtest-init.el of course.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Alan Mackenzie]

Hi, Glenn.

On Wed, May 01, 2013 at 01:59:49PM -0400, Glenn Morris wrote:
> Alan Mackenzie wrote:

> > However, you'll probably prefer to carry on with using Font Lock Mode
> > uninitialised.  ;-)  I think Glenn's patch achieves this painlessly.

> That patch seems like TRT to me anyway.

Yes.  Give me another hour or two to consider, then I'll probably commit
it.

> What problem is c-standard-font-lock-fontify-region-function supposed to
> solve?

font-lock-mode not being loaded at the time CC Mode gets loaded, in
particular in XEmacs.  [I really don't want two alternative bits of code
for this level of indirectness.]

> Someone changing the default value of
> font-lock-fontify-region-function in between font-lock-mode-hook being
> run and font-lock-fontify-region being called?  That shouldn't happen,
> only the buffer-local value should ever be changed. It's not a
> user-variable.

Anybody changing the default value of font-l-f-r-function had better
know what she's doing.  ;-)

> (I actually think you could safely just call
> font-lock-default-fontify-region .)

But somebody, somewhere, sometime _might_ change the default value of
font-lock-fontify-region-function.  Unlikely, but it's possible.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Glenn Morris]

Alan Mackenzie wrote:

> Yes.  Give me another hour or two to consider, then I'll probably commit
> it.

Sure, no rush! :)

>> What problem is c-standard-font-lock-fontify-region-function supposed to
>> solve?
>
> font-lock-mode not being loaded at the time CC Mode gets loaded, in
> particular in XEmacs.

But the _only_ place that uses
c-standard-font-lock-fontify-region-function is
c-font-lock-fontify-region, which you set as the value for
font-lock-fontify-region-function, which is only called from font-lock.
It doesn't matter whether or font-lock is loaded when cc-mode is loaded.
It will be loaded before c-font-lock-fontify-region is called.

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Alan Mackenzie]

Hi, Achim.

On Wed, May 01, 2013 at 08:15:03PM +0200, Achim Gratz wrote:
> Alan Mackenzie writes:
> > OK.  Could you possibly give the exact sequence of Font Lock and CC Mode
> > calls which lead to the error?

> If you can tell me how to obtain a trace in batch mode?

:-).  I thought you knew the software.  Never mind.

I've installed Glenn's patch into the Emacs trunk.  Could you please try
it out, and see if it solves your problem.

> Regards,
> Achim.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Achim Gratz]

Alan Mackenzie writes:
>> If you can tell me how to obtain a trace in batch mode?
>
> :-).  I thought you knew the software.  Never mind.

I do, but I don't know what are the differences in noninteractive mode.
Stepping through the code in interactive mode is possible, but I
wouldn't be able to tell if it goes through the same motions.

> I've installed Glenn's patch into the Emacs trunk.  Could you please try
> it out, and see if it solves your problem.

I'll give it a whirl over the weekend (I hope).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Achim Gratz]

Alan Mackenzie writes:
> I've installed Glenn's patch into the Emacs trunk.  Could you please
> try it out, and see if it solves your problem.

I've built a new Emacs from trunk today and the fix works as intended.
Thanks to you and Glenn.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

bug#14325: 24.3; cc-mode does not initialize correctly w/ -batch

[Alan Mackenzie]

Bug fixed.

-- 
Alan Mackenzie (Nuremberg, Germany).