bug#16160: [PATCH] define-derived-mode clobbers syntax tables

bug#16160: [PATCH] define-derived-mode clobbers syntax tables

[Daniel Colascione]

Repro:

1) Visit shell script with comments. See comments fontified properly.
2) M-x find-library sh-script RET
3) M-x eval-buffer
4) Return to shell script buffer
5) C-x C-v RET
6) Observe that comments are no longer fontified as comments.

The define-derived-mode macro in sh-script.el emits code that clobbers 
sh-mode-syntax-table because we don't have a :syntax-table argument. I 
think the following patch changes the code to the expected behavior.

~/edev/trunk
$ bzr diff
=== modified file 'lisp/emacs-lisp/derived.el'
--- lisp/emacs-lisp/derived.el	2013-05-27 16:12:52 +0000
+++ lisp/emacs-lisp/derived.el	2013-12-16 11:09:41 +0000
@@ -206,11 +206,11 @@
         ,(if declare-syntax
  	    `(progn
  	       (unless (boundp ',syntax)
-		 (put ',syntax 'definition-name ',child))
-	       (defvar ,syntax (make-syntax-table))
-	       (unless (get ',syntax 'variable-documentation)
-		 (put ',syntax 'variable-documentation
-		      (purecopy ,(format "Syntax table for `%s'." child))))))
+		 (put ',syntax 'definition-name ',child)
+                 (defvar ,syntax (make-syntax-table))
+                 (unless (get ',syntax 'variable-documentation)
+                   (put ',syntax 'variable-documentation
+                        (purecopy ,(format "Syntax table for `%s'." 
child)))))))
         ,(if declare-abbrev
  	    `(progn
  	       (put ',abbrev 'definition-name ',child)


That is, we shouldn't touch the syntax table variable at all unless the 
variable is unbound.

bug#16160: [PATCH] define-derived-mode clobbers syntax tables

[Stefan Monnier]

> 1) Visit shell script with comments. See comments fontified properly.
> 2) M-x find-library sh-script RET
> 3) M-x eval-buffer
> 4) Return to shell script buffer
> 5) C-x C-v RET
> 6) Observe that comments are no longer fontified as comments.

I can't reproduce it, starting from "emacs -Q".

> -		 (put ',syntax 'definition-name ',child))
> -	       (defvar ,syntax (make-syntax-table))
> -	       (unless (get ',syntax 'variable-documentation)
> -		 (put ',syntax 'variable-documentation
> -		      (purecopy ,(format "Syntax table for `%s'." child))))))
> +		 (put ',syntax 'definition-name ',child)
> +                 (defvar ,syntax (make-syntax-table))
> +                 (unless (get ',syntax 'variable-documentation)
> +                   (put ',syntax 'variable-documentation
> +                        (purecopy ,(format "Syntax table for `%s'." child)))))))
> That is, we shouldn't touch the syntax table variable at all unless the
> variable is unbound.

But that shouldn't make much difference since none of the code you
changed should affect the var's value when it is already bound.
I'm not necessarily opposed to the change (haven't dug enough to try and
remember why it's written this way), but I'd first like to understand
why it fixes the problem you see.


        Stefan

bug#16160: [PATCH] define-derived-mode clobbers syntax tables

[Daniel Colascione]

On 12/16/2013 11:17 AM, Stefan Monnier wrote:
>> 1) Visit shell script with comments. See comments fontified properly.
>> 2) M-x find-library sh-script RET
>> 3) M-x eval-buffer
>> 4) Return to shell script buffer
>> 5) C-x C-v RET
>> 6) Observe that comments are no longer fontified as comments.
>
> I can't reproduce it, starting from "emacs -Q".

I can't repro it either now, with or without -Q. That was odd. I know 
what I saw, and I know that my patch resolved the problem in that 
instance of Emacs, but I have no idea how defvar was clobbering that 
variable. I was able to repro it over an dover again though. I can only 
conclude that Emacs was punishing me for my many sins.

If I see it again, I'll try to hook Emacs up to a debugger. In the 
meantime, let's avoid changing fundamental infrastructure before a 
feature freeze.

bug#16160: [PATCH] define-derived-mode clobbers syntax tables

[Stefan Monnier]

> I can only conclude that Emacs was punishing me for my many sins.

I believe He may forgive you, if you fix a few bugs for penance.


        Stefan

bug#16160: [PATCH] define-derived-mode clobbers syntax tables

[Daniel Colascione]

Actually...

On 12/16/2013 06:17 PM, Daniel Colascione wrote:
> On 12/16/2013 11:17 AM, Stefan Monnier wrote:
>>> 1) Visit shell script with comments. See comments fontified properly.
>>> 2) M-x find-library sh-script RET
>>> 3) M-x eval-buffer
>>> 4) Return to shell script buffer
>>> 5) C-x C-v RET
>>> 6) Observe that comments are no longer fontified as comments.

The repro steps are incorrect. Try these:

1) emacs -Q
2) visit a shell script
3) M-x find-library sh-script RET
4) C-s define-derived-mode RET
5) C-M-x
6) switch to your shell script buffer
7) C-x C-v
8) Observe that the shell script is fontified incorrectly; the syntax 
table is now clobbered.

bug#16160: [PATCH] define-derived-mode clobbers syntax tables

[Lars Ingebrigtsen]

Daniel Colascione <dancol@dancol.org> writes:

> The repro steps are incorrect. Try these:
>
> 1) emacs -Q
> 2) visit a shell script
> 3) M-x find-library sh-script RET
> 4) C-s define-derived-mode RET
> 5) C-M-x
> 6) switch to your shell script buffer
> 7) C-x C-v
> 8) Observe that the shell script is fontified incorrectly; the syntax
> table is now clobbered.

I can confirm that this bug still exists.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#16160: [PATCH] define-derived-mode clobbers syntax tables

[Glenn Morris]

Version: 26.1

>> The repro steps are incorrect. Try these:
>>
>> 1) emacs -Q
>> 2) visit a shell script
>> 3) M-x find-library sh-script RET
>> 4) C-s define-derived-mode RET
>> 5) C-M-x
>> 6) switch to your shell script buffer
>> 7) C-x C-v
>> 8) Observe that the shell script is fontified incorrectly; the syntax
>> table is now clobbered.

Fixed in 8db7b65 along the suggested lines.

So AIUI the issue is that define-derived-mode expands to 

(defvar ,syntax (make-syntax-table))

Normally this has no effect if ,syntax is already bound.
But M-x eval-defun resets defvars to their default settings...
Hence if you selectively re-evaluate only the mode definition from
sh-script.el, you get the reported problem.

So I changed it to explicitly check if ,syntax is bound.

This will however now make things go wrong in the opposite way for
someone who has been playing around with customizing a syntax table that
was actually defined by define-derived-mode and wants to reset it by
re-evaluating the derived mode definition. But AFAICS there's no way to
fix both scenarios, and the second seems less likely (?) to me than the
one reported here.