bug#11739: 24.1; font-lock (highlighting) bug with dollar+quote sequence ('...$')

bug#11739: 24.1; font-lock (highlighting) bug with dollar+quote sequence ('...$')

[Vincent Lefevre]

Open the following file with "emacs -Q <filename>":

------------------------------------------------------------
#!/usr/bin/env perl
# -*- mode: cperl -*-

$foo = 'blah $ '
  and print "$foo\n";  # OK

$foo = 'blah $'
  and print "$foo\n";  # Incorrectly highlighted
------------------------------------------------------------

The first "and print ..." line is correctly highlighted, but not the
second one. It seems that CPerl is confused by a $ at the end of a
single-quote string literal.

This is a regression: GNU Emacs 23.4.1 from Debian does not have this
problem.


In GNU Emacs 24.1.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10)
 of 2012-06-15 on ypig
Windowing system distributor `The X.Org Foundation', version 11.0.11201902
Configured using:
 `configure '--prefix=/usr/local/emacs-24.1' '--enable-asserts''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: POSIX
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: en_DK
  value of $LANG: POSIX
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  display-time-mode: t
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<escape> x r e p o <tab> r t - b u <tab> <return>

Recent messages:
Loading /home/vlefevre/share/emacs/site-lisp/mutteditor.el (source)...done
Loading time...done
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr warnings emacsbug message format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils help-mode easymenu view time cus-start
cus-load paren cc-styles cc-align cc-engine cc-vars cc-defs regexp-opt
time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image fringe lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

bug#11739: 24.1; font-lock (highlighting) bug with dollar+quote sequence ('...$')

[Vincent Lefevre]

On 2012-06-19 12:23:51 +0200, Vincent Lefevre wrote:
> Open the following file with "emacs -Q <filename>":
> 
> ------------------------------------------------------------
> #!/usr/bin/env perl
> # -*- mode: cperl -*-
> 
> $foo = 'blah $ '
>   and print "$foo\n";  # OK
> 
> $foo = 'blah $'
>   and print "$foo\n";  # Incorrectly highlighted
> ------------------------------------------------------------
> 
> The first "and print ..." line is correctly highlighted, but not the
> second one. It seems that CPerl is confused by a $ at the end of a
> single-quote string literal.

The problem is due to this change in cperl-mode.el:

@@ -1832,7 +1832,13 @@
   (make-local-variable 'cperl-syntax-state)
   (setq cperl-syntax-state nil)                ; reset syntaxification cache
   (if cperl-use-syntax-table-text-property
-      (progn
+      (if (boundp 'syntax-propertize-function)
+          (progn
+            ;; Reset syntaxification cache.
+            (set (make-local-variable 'cperl-syntax-done-to) nil)
+            (set (make-local-variable 'syntax-propertize-function)
+                 (lambda (start end)
+                   (goto-char start) (cperl-fontify-syntaxically end))))
        (make-local-variable 'parse-sexp-lookup-properties)
        ;; Do not introduce variable if not needed, we check it!
        (set 'parse-sexp-lookup-properties t)

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#11739: 24.1; font-lock (highlighting) bug with dollar+quote sequence ('...$')

[Stefan Monnier]

> The first "and print ..." line is correctly highlighted, but not the
> second one. It seems that CPerl is confused by a $ at the end of a
> single-quote string literal.

Actually, cperl recognizes these just fine, but there was some
miscommunication between cperl and syntax-propertize so one ended up
removing something expecting the other would refresh it, but without
telling it.
It should be fixed now on the emacs-24 branch.


        Stefan


--- lisp/progmodes/cperl-mode.el	2012-01-19 07:21:25 +0000
+++ lisp/progmodes/cperl-mode.el	2012-06-21 16:41:01 +0000
@@ -8951,11 +8951,12 @@
       (setq cperl-syntax-done-to (min cperl-syntax-done-to beg))))
 
 (defun cperl-update-syntaxification (from to)
-  (if (and cperl-use-syntax-table-text-property
-	   cperl-syntaxify-by-font-lock
+  (cond
+   ((not cperl-use-syntax-table-text-property) nil)
+   ((fboundp 'syntax-propertize) (syntax-propertize to))
+   ((and cperl-syntaxify-by-font-lock
 	   (or (null cperl-syntax-done-to)
 	       (< cperl-syntax-done-to to)))
-      (progn
 	(save-excursion
 	  (goto-char from)
 	  (cperl-fontify-syntaxically to)))))