bug#12373: 24.1; cperl-mode syntax highlighting

bug#12373: 24.1; cperl-mode syntax highlighting

[Stefan Gottschalk]

Hello,

This is a bug report for cperl-mode major mode.  I'm sorry, I do not
know how to contact the author of this major mode.

First of all, I think cperl-mode is wonderful, so please don't think I'm
ungrateful for it.  

I'm having a problem with syntax highlighting, in which single-quoted
strings ending with a $ are misinterpreted.  In such cases, I suspect
that the ' is treated as if it belonged to the $ (as if it were part of the
special variable $' instead of the quote terminating the string).

This causes all subsequent text to be classified as "in the string".

Here's a real world sample input

            $fileType_filesButNot = join( '|', map { quotemeta($_).'$' } @{$fileType->{filesButNot}} );

In order to make the syntax highlighter happy, I have to do something like,

            $fileType_filesButNot = join( '|', map { quotemeta($_)."\$" } @{$fileType->{filesButNot}} );

which I think is less clear, and also draws comments from my team members during code reviews:

  Q: why did you make this change?
  A: because my syntax highlighter didn't handle it the other way
  Q: have you considered using a better editor, such as Eclipse?
  A: that was uncalled for, you $%#@!...

And, of course, it goes downhill from there.  

So, in order to preserve the peace at the office, I was hoping to find a
workaround for this specific problem of syntax highlighting '$'.

I hunted around the net for solutions of this, but I can't even find
anyone discussing it.

Much thanks for any feedback or suggestions.

-stefan








In GNU Emacs 24.1.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2012-06-12 on l-c4u5build64
Windowing system distributor `The X.Org Foundation', version 11.0.60900000
Configured using:
 `configure '--prefix=/home/utils/emacs-24.1''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: CPerl

Minor modes in effect:
  which-function-mode: t
  show-paren-mode: t
  display-time-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x C-s C-x C-v <return> <next> <next> <next> <next> 
<next> <next> <next> <prior> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <down> <down> <down> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <right> <right> <left> <left> 
<right> <right> <left> <left> <right> <right> <left> 
<left> <right> <right> <left> <left> <right> <right> 
<down-mouse-1> <mouse-1> <down-mouse-1> <mouse-1> C-x 
b C-g <up> <up> <up> <up> <up> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <C-right> <C-right> <C-right> 
<C-right> <C-right> <C-right> <C-right> <C-left> <C-left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> " " <left> \ $ <right> C-d C-d C-d <down> <down> 
<down> <down> <down> <down> <down> <left> <left> " 
" <left> <right> C-d C-d C-d <left> \ $ C-a C-x C-s 
<up> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> C-s ' $ ' C-s C-s 
C-s C-s <up> <up> <up> <up> C-s $ ' C-s C-s C-s C-s 
C-s C-s C-s C-s C-s C-s C-s C-s C-a <up> <up> <up> 
<up> <up> <up> <up> <up> <down> <down> <down-mouse-1> 
<mouse-1> C-s " $ <up> <up> C-s " \ $ " C-s C-a C-SPC 
<down> ESC w <down-mouse-1> <mouse-1> ESC x r p o r 
t - e m c s <backspace> <backspace> a c s - b u g <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> C-a C-k r e p o r t - e m a c 
s - b <tab> <return>

Recent messages:
byte-code: End of buffer [8 times]
Buffer is read-only: #<buffer convert_or_validate_maxwell_tree.pl>
Saving file /home/scratch.sgottschalk_maxwell-info/testgen_maxwell_conversion_2/dev/inf/testgen/mainline/bin/scripts/admin/convert_or_validate_maxwell_tree.pl...
Wrote /home/scratch.sgottschalk_maxwell-info/testgen_maxwell_conversion_2/dev/inf/testgen/mainline/bin/scripts/admin/convert_or_validate_maxwell_tree.pl
Error during redisplay: (wrong-type-argument arrayp nil) [3 times]
Quit
Saving file /home/scratch.sgottschalk_maxwell-info/testgen_maxwell_conversion_2/dev/inf/testgen/mainline/bin/scripts/admin/convert_or_validate_maxwell_tree.pl...
Wrote /home/scratch.sgottschalk_maxwell-info/testgen_maxwell_conversion_2/dev/inf/testgen/mainline/bin/scripts/admin/convert_or_validate_maxwell_tree.pl
Mark saved where search started [4 times]
Mark set

Load-path shadows:
/home/sgottschalk/emacs/replace hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/replace
/home/sgottschalk/emacs/remember/remember hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/textmodes/remember
/home/sgottschalk/emacs/verilog-mode hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/progmodes/verilog-mode
~/emacs/org-6.28e/lisp/org-xoxo hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-xoxo
~/emacs/org-6.28e/lisp/org-wl hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-wl
~/emacs/org-6.28e/lisp/org-w3m hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-w3m
~/emacs/org-6.28e/lisp/org-vm hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-vm
~/emacs/org-6.28e/lisp/org-timer hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-timer
~/emacs/org-6.28e/lisp/org-table hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-table
~/emacs/org-6.28e/lisp/org-src hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-src
~/emacs/org-6.28e/lisp/org-rmail hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-rmail
~/emacs/org-6.28e/lisp/org-remember hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-remember
~/emacs/org-6.28e/lisp/org-publish hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-publish
~/emacs/org-6.28e/lisp/org-protocol hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-protocol
~/emacs/org-6.28e/lisp/org-plot hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-plot
~/emacs/org-6.28e/lisp/org-mouse hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-mouse
~/emacs/org-6.28e/lisp/org-mhe hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-mhe
~/emacs/org-6.28e/lisp/org-mew hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-mew
~/emacs/org-6.28e/lisp/org-macs hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-macs
~/emacs/org-6.28e/lisp/org-mac-message hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-mac-message
~/emacs/org-6.28e/lisp/org-list hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-list
~/emacs/org-6.28e/lisp/org-latex hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-latex
~/emacs/org-6.28e/lisp/org-jsinfo hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-jsinfo
~/emacs/org-6.28e/lisp/org-irc hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-irc
~/emacs/org-6.28e/lisp/org-install hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-install
~/emacs/org-6.28e/lisp/org-inlinetask hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-inlinetask
~/emacs/org-6.28e/lisp/org-info hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-info
~/emacs/org-6.28e/lisp/org-id hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-id
~/emacs/org-6.28e/lisp/org-icalendar hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-icalendar
~/emacs/org-6.28e/lisp/org-html hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-html
~/emacs/org-6.28e/lisp/org-gnus hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-gnus
~/emacs/org-6.28e/lisp/org-footnote hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-footnote
~/emacs/org-6.28e/lisp/org-feed hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-feed
~/emacs/org-6.28e/lisp/org-faces hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-faces
~/emacs/org-6.28e/lisp/org-exp hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-exp
~/emacs/org-6.28e/lisp/org-exp-blocks hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-exp-blocks
~/emacs/org-6.28e/lisp/org-docbook hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-docbook
~/emacs/org-6.28e/lisp/org-compat hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-compat
~/emacs/org-6.28e/lisp/org-colview hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-colview
~/emacs/org-6.28e/lisp/org-clock hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-clock
~/emacs/org-6.28e/lisp/org-bibtex hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-bibtex
~/emacs/org-6.28e/lisp/org-bbdb hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-bbdb
~/emacs/org-6.28e/lisp/org-attach hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-attach
~/emacs/org-6.28e/lisp/org-ascii hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-ascii
~/emacs/org-6.28e/lisp/org-archive hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-archive
~/emacs/org-6.28e/lisp/org-agenda hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org-agenda
~/emacs/org-6.28e/lisp/org hides /home/utils/emacs-24.1/share/emacs/24.1/lisp/org/org

Features:
(shadow sort mail-extr emacsbug sendmail mule-util newcomment ediff-merg
ediff-diff ediff-wind ediff-help ediff-util ediff-mult ediff-init ediff
cperl-mode add-log perl-mode help-mode view multi-isearch ffap edmacro
kmacro org-remember org org-footnote org-src org-list org-faces
org-compat org-macs noutline outline easy-mmode cal-menu calendar
cal-loaddefs gnus-sum nnoo gnus-group gnus-undo nnmail mail-source
gnus-start gnus-spec gnus-int gnus-range message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus gnus-ems
nnheader gnus-util mail-utils mm-util mail-prsvr wid-edit org-install
perltidy remember which-func imenu paren cus-start cus-load xcscope ring
gtags lineno cl derived p4 linum cc-mode cc-fonts easymenu cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs regexp-opt
time time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win
x-dnd tool-bar dnd fontset image fringe lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process dynamic-setting font-render-setting x-toolkit x
multi-tty emacs)

bug#12373: 24.1; cperl-mode syntax highlighting

[Stefan Monnier]

> Here's a real world sample input

>             $fileType_filesButNot = join( '|', map { quotemeta($_).'$' } @{$fileType->{filesButNot}} );

Hmm... handling of $' in Perl is tricky, so I wouldn't be surprised if
it still has bugs, but when I try a Perl file (using cperl-mode) with
just the above line in it, it is displayed correctly both by Emacs-23
and Emacs-24.  Here's the command I used to test:

 % emacs24 -Q --eval "(defalias 'cperl-mode 'perl-mode)" test/indent/perl.perl

And here's the perl.perl file:

   #!/usr/bin/perl

   $fileType_filesButNot           # bug#12373
       = join( '|', map { quotemeta($_).'$' } @{$fileType->{filesButNot}} );

Using `perl-mode' instead of `cperl-mode' also seems to work fine.
Can you send us another test case that exhibits the problem?


        Stefan

bug#12373: 24.1; cperl-mode syntax highlighting

[Stefan Gottschalk]

Hi, Stefan.  

Thanks for looking at this.
I tried your repro - and it works fine for me too.

My apologies!  
I will work up a better repro - or be happy that I can't make it happen!

-stefan

-----Original Message-----
From: Stefan Monnier [mailto:monnier@iro.umontreal.ca] 
Sent: Tuesday, October 30, 2012 10:03 PM
To: Stefan Gottschalk
Cc: 12373@debbugs.gnu.org
Subject: Re: bug#12373: 24.1; cperl-mode syntax highlighting

> Here's a real world sample input

>             $fileType_filesButNot = join( '|', map { quotemeta($_).'$' 
> } @{$fileType->{filesButNot}} );

Hmm... handling of $' in Perl is tricky, so I wouldn't be surprised if it still has bugs, but when I try a Perl file (using cperl-mode) with just the above line in it, it is displayed correctly both by Emacs-23 and Emacs-24.  Here's the command I used to test:

 % emacs24 -Q --eval "(defalias 'cperl-mode 'perl-mode)" test/indent/perl.perl

And here's the perl.perl file:

   #!/usr/bin/perl

   $fileType_filesButNot           # bug#12373
       = join( '|', map { quotemeta($_).'$' } @{$fileType->{filesButNot}} );

Using `perl-mode' instead of `cperl-mode' also seems to work fine.
Can you send us another test case that exhibits the problem?


        Stefan