Font-lock with newcomment.el

Font-lock with newcomment.el

[David C Sterratt]

I'm writing a mode for the HOC language used by the neuron simulator
(see http://www.anc.ed.ac.uk/~dcs/progs/neuron/ for details of the
mode).  

The mode was written under XEmacs, and now I'm trying to make it work
with gnu emacs.  The mode has C++/C style comments.  Everything seems
to work apart from the // form of comment syntax highlighting.  Under
XEmacs, the nrnhoc-mode definition appended at the end seemed
sufficient to get the font-locking of //-style comments working (I've
commented out values of comment-start and comment-start-skip that I
also tried with gnu emacs, following their values in c++ mode.)

I notice that comment-start and friends are defined in newcomment.el
in gnu emacs as opposed to simple.el in xemacs.  But it's not clear
from these files how the variables comment-start &c set up the
font-locking.  I'd be very grateful if someone could tell me whether
there some extra code I need to get font-locking working in gnu emacs.

Best wishes,

David



(defun nrnhoc-mode ()
  "NRNHOC-mode is a major mode for editing Neuron HOC dot-hoc files.
\\<nrnhoc-mode-map>

Variables:
  `nrnhoc-indent-level'		       Level to indent blocks.
  `nrnhoc-comment-column'		     The goal comment column
  `fill-column'			           Column used in auto-fill.
  `nrnhoc-return-function'  	     Customize RET handling with this function
  `nrnhoc-closing-brace-function' Customize } handling with this function

All Key Bindings:
\\{nrnhoc-mode-map}"
  (interactive)
  (kill-all-local-variables)
  (use-local-map nrnhoc-mode-map)
  (setq major-mode 'nrnhoc-mode)
  (setq mode-name "Hoc")
  (make-local-variable 'indent-line-function)
  (setq indent-line-function 'nrnhoc-indent-line)
  (make-local-variable 'comment-start-skip)
  (setq comment-start-skip "//\\s-+")
  ;(setq comment-start-skip "/\\*+ *\\|//+ *")
  (make-local-variable 'comment-start)
  (setq comment-start "//")
  ;(setq comment-start "// ")
  (make-local-variable 'comment-column)
  (setq comment-column nrnhoc-comment-column)
  (make-local-variable 'font-lock-defaults)
  (setq font-lock-defaults '((nrnhoc-font-lock-keywords)
                             nil ; do not do string/comment highlighting
                             nil ; keywords are case sensitive.
                             ;; This puts _ as a word constituent,
                             ;; simplifying our keywords significantly
                             ((?_ . "w")
                              (?\n . "> b")
                              (?/ . ". 1456")
                              (?* . ". 23")
                              (?\^m . "> b")
                              )))
  (run-hooks 'nrnhoc-mode-hook)
  )

Re: Font-lock with newcomment.el

[lawrence mitchell]

David C Sterratt wrote:

[...]

>   (setq font-lock-defaults '((nrnhoc-font-lock-keywords)
>                              nil ; do not do string/comment highlighting
>                              nil ; keywords are case sensitive.
>                              ;; This puts _ as a word constituent,
>                              ;; simplifying our keywords significantly
>                              ((?_ . "w")
>                               (?\n . "> b")
>                               (?/ . ". 1456")
                                         ^^^^
Emacs has a different set of characters to mean the same thing
as XEmacs, try using ". 124b" instead of ". 1456".


-- 
lawrence mitchell <wence@gmx.li>

Re: Font-lock with newcomment.el

[Stefan Monnier]

>>>>> "David" == David C Sterratt <david.c.sterratt@ed.ac.uk> writes:
> in gnu emacs as opposed to simple.el in xemacs.  But it's not clear
> from these files how the variables comment-start &c set up the
> font-locking.

It's not clear simply because it's not the case.
Font-lock uses syntax-tables rather than comment-start and comment-end.

Your problem is that XEmacs uses different settings for
"two-char comment-markers of style b":

>                               (?\n . "> b")
>                               (?/ . ". 1456")
>                               (?* . ". 23")

The `56' above is XEmacs specific.  In Emacs, you'd do
use (?/ . ". 124b") instead.  I think the Emacs syntax is accepted
by XEmacs as well.


        Stefan

Re: Font-lock with newcomment.el

[David C Sterratt]

>>>>> Stefan Monnier writes:

>>>>> "David" == David C Sterratt <david.c.sterratt@ed.ac.uk> writes:
 >> in gnu emacs as opposed to simple.el in xemacs.  But it's not
 >> clear from these files how the variables comment-start &c set up
 >> the font-locking.

 > It's not clear simply because it's not the case.  Font-lock uses
 > syntax-tables rather than comment-start and comment-end.

 > Your problem is that XEmacs uses different settings for "two-char
 > comment-markers of style b":

 >> (?\n . "> b") (?/ . ". 1456") (?* . ". 23")

 > The `56' above is XEmacs specific.  In Emacs, you'd do use (?/
 > . ". 124b") instead.  I think the Emacs syntax is accepted by
 > XEmacs as well.

Thanks very much to you and Lawrence for your speedy assistance.  The
above syntax does work for emacs, but not for XEmacs 21.4.  The
slightly ugly construct below does the trick for both.

 (cond ( 
         (string-match "XEmacs" (emacs-version))
         (setq font-lock-defaults '((nrnhoc-font-lock-keywords)
                             nil ; do not do string/comment highlighting
                             nil ; keywords are case sensitive.
                             ;; This puts _ as a word constituent,
                             ;; simplifying our keywords significantly
                             ((?_ . "w")
                              (?\n . "> b")
                              (?/ . ". 1456")
                              (?* . ". 23")
                              (?\^m . "> b")))))
        (t
         (setq font-lock-defaults '((nrnhoc-font-lock-keywords)
                             nil ; do not do string/comment highlighting
                             nil ; keywords are case sensitive.
                             ;; This puts _ as a word constituent,
                             ;; simplifying our keywords significantly
                             ((?_ . "w")
                              (?\n . "> b")
                              (?/ . ". 124b")
                              (?* . ". 23")
                              (?\^m . "> b"))))))


David.

Re: Font-lock with newcomment.el

[lawrence mitchell]

David C Sterratt wrote:

[...] Having to repeat font-lock-defaults with one minor change.

How about:
(setq font-lock-defaults `((nnrnhoc-font-lock-keywords)
                           nil
                           nil
                           ((?_ . "w")
                            (?\n . "> b")
                            (?/ . ,(if (string-match "XEmacs"
                                                     (emacs-version))
                                       ". 1456"
                                       ". 124b"))
                            (?* . ". 23")
                            (?\^m . "> b"))))

That way, you only have a different bit for the XEmacs/Emacs
difference, rather than having to repeat the entire setq twice.

-- 
lawrence mitchell <wence@gmx.li>

Re: Font-lock with newcomment.el

[Stefan Monnier]

> (setq font-lock-defaults `((nnrnhoc-font-lock-keywords)
>                            nil
>                            nil
>                            ((?_ . "w")
>                             (?\n . "> b")
>                             (?/ . ,(if (string-match "XEmacs"
>                                                      (emacs-version))
>                                        ". 1456"
>                                        ". 124b"))
>                             (?* . ". 23")
>                             (?\^m . "> b"))))

BTW, setting your syntax-table this way is probably not a good idea because
it will only affect font-locking whereas I suspect that you might want the
rest of Emacs to know about those comments as well (newcomment.el for
example uses the syntax-table (in place of and in addition to
comment-start/end)) so it's better to properly setup your major mode's
syntax-table.

Oh, and David, regarding your mode: do a C-h f looking-at, that will
save you a bunch of buffer-substrings.


        Stefan

Re: Font-lock with newcomment.el

[David C Sterratt]

>>>>> Stefan Monnier writes:

 >> (setq font-lock-defaults `((nnrnhoc-font-lock-keywords) nil nil
 >> ((?_ . "w") (?\n . "> b") (?/ . ,(if (string-match "XEmacs"
 >> (emacs-version)) ". 1456" ". 124b")) (?* . ". 23") (?\^m . ">
 >> b"))))

 > BTW, setting your syntax-table this way is probably not a good idea
 > because it will only affect font-locking whereas I suspect that you
 > might want the rest of Emacs to know about those comments as well
 > (newcomment.el for example uses the syntax-table (in place of and
 > in addition to comment-start/end)) so it's better to properly setup
 > your major mode's syntax-table.

Thanks for the advice.  The new version looks like this.  
 
  (setq font-lock-defaults '((nrnhoc-font-lock-keywords)
                             nil ; do not do string/comment highlighting
                             nil ; keywords are case sensitive.
                             nil
                             ))
  (modify-syntax-entry ?_  "w")
  (modify-syntax-entry ?\n  "> b")
  (modify-syntax-entry ?/  (
                            if (string-match "XEmacs" (emacs-version))
                               ". 1456"
                             ". 124b"))
  (modify-syntax-entry ?*  ". 23")
  (modify-syntax-entry ?\^m  "> b")

Much prettier.  And it seems to work too.
   
 > Oh, and David, regarding your mode: do a C-h f looking-at, that
 > will save you a bunch of buffer-substrings.

Thanks for looking at the code.  From the looking-at documentation,
it seems like it only matches regexps after the point, whereas in the
current way of doing indentation I need to match symbols anywhere in
the line, in most cases at least.  So it's not clear to me how
looking-at will help, unless I rethink the indentation code.  
 
David

Re: Font-lock with newcomment.el

[Kevin Rodgers]

David C Sterratt wrote:

>>>>>>Stefan Monnier writes:
>  > Oh, and David, regarding your mode: do a C-h f looking-at, that
>  > will save you a bunch of buffer-substrings.
> 
> Thanks for looking at the code.  From the looking-at documentation,
> it seems like it only matches regexps after the point, whereas in the
> current way of doing indentation I need to match symbols anywhere in
> the line, in most cases at least.  So it's not clear to me how
> looking-at will help, unless I rethink the indentation code.  

I don't have your code in front of me, but it sounds like you're using
buffer-substring to extract the current line and then string-match to
recognize a pattern in it.  Since looking-at only matches its REGEXP argument
immediately after point, you can search for any pattern on the current line
with (concat ".*" REGEXP), assuming you're at the beginning of the line -- if
not, just wrap it in (save-excursion (beginning-of-line) ...).

-- 
<a href="mailto:&lt;kevin.rodgers&#64;ihs.com&gt;">Kevin Rodgers</a>

Re: Font-lock with newcomment.el

[David C Sterratt]

>>>>> Kevin Rodgers writes:

 > David C Sterratt wrote:
 >>>>>>> Stefan Monnier writes:
 >> > Oh, and David, regarding your mode: do a C-h f looking-at, that
 >> > will save you a bunch of buffer-substrings.
 >> Thanks for looking at the code.  From the looking-at
 >> documentation, it seems like it only matches regexps after the
 >> point, whereas in the current way of doing indentation I need to
 >> match symbols anywhere in the line, in most cases at least.  So
 >> it's not clear to me how looking-at will help, unless I rethink
 >> the indentation code.

 > I don't have your code in front of me, but it sounds like you're
 > using buffer-substring to extract the current line and then
 > string-match to recognize a pattern in it.  Since looking-at only
 > matches its REGEXP argument immediately after point, you can search
 > for any pattern on the current line with (concat ".*" REGEXP),
 > assuming you're at the beginning of the line -- if not, just wrap
 > it in (save-excursion (beginning-of-line) ...).

Thanks for the advice.  I may have to save it for a  rainy day, as
I've got too much to do at the moment, but when I'm feeling like a
bit more elisp hacking I'll take a look at the indentation code
again.

David.

Re: Font-lock with newcomment.el

[Stefan Monnier]

>   (setq font-lock-defaults '((nrnhoc-font-lock-keywords)
>                              nil ; do not do string/comment highlighting
>                              nil ; keywords are case sensitive.
>                              nil
>                              ))
>   (modify-syntax-entry ?_  "w")
>   (modify-syntax-entry ?\n  "> b")
>   (modify-syntax-entry ?/  (
>                             if (string-match "XEmacs" (emacs-version))
>                                ". 1456"
>                              ". 124b"))
>   (modify-syntax-entry ?*  ". 23")
>   (modify-syntax-entry ?\^m  "> b")

Look at the SampleMode code in the www.emacswiki.org.  It shows the
canonical way to setup a mode's syntax-table.  It's generally better
to leave this stuff outside of the major-mode function itself.

>> Oh, and David, regarding your mode: do a C-h f looking-at, that
>> will save you a bunch of buffer-substrings.

> Thanks for looking at the code.  From the looking-at documentation,
> it seems like it only matches regexps after the point, whereas in the
> current way of doing indentation I need to match symbols anywhere in
> the line, in most cases at least.  So it's not clear to me how
> looking-at will help, unless I rethink the indentation code.  

Then use re-search-forward (with a (line-end-position) limit if necessary).
Also, rather than using regexp-matching to find { and } and comment-markers,
it's a lot more robust to use sexp-based operations, such as
(forward-sexp -1) and/or (up-list -1).  If you (buffer-locally) set
parse-sexp-ignore-comments (which you should) they will automatically skip
comments (i.e. they will properly handle the case where you have a { inside
a comment).  Indentation in Emacs is still a black art, sadly, but
most of the indentation functions I've written have at their core a loop
that looks like

    (let ((outside-pos nil)
          (start pos (point)))
      (condition-case err
           (while
               (progn
                 (forward-sexp -1)
                 ;; Are we at the beginning of the line?
                 (save-excursion (skip-chars-backward " \t") (not (bolp)))))
         (scan-error
          (setq outside-pos (nth 2 err))))
      ;; Now we're either at the beginning of a previous line at the same
      ;; parenthesis level (in which case `outside-pos' is nil) or we're
      ;; looking at the first element after the open paren in which the
      ;; line we're coming from is enclosed (in which case `outside-pos'
      ;; is the position of that paren).
      (if (or (null outside-pos) (< (point) start))
          ;; We're at the start of a previous line at the same paren level.
          (current-indentation)
        ;; We're the first thing after the open paren:
        ;; indent relative to open-paren.
        (goto-char outside-pos)
        (+ basic-indent (current-indentation))))


        Stefan

PS: You might also like to use `regexp-opt' to build the regexps matching
a list of keywords.  Makes the code more readable.

Re: Font-lock with newcomment.el

[David C Sterratt]

>>>>> lawrence mitchell writes:

 > David C Sterratt wrote: [...] Having to repeat font-lock-defaults
 > with one minor change.

 > How about: (setq font-lock-defaults `((nnrnhoc-font-lock-keywords)
 >                            nil nil ((?_ . "w")
 >                             (?\n . "> b") (?/ . ,(if (string-match
 >                             "XEmacs"
 >                                                      (emacs-version))
 >                                        ". 1456" ". 124b"))
 >                             (?* . ". 23") (?\^m . "> b"))))
 
 > That way, you only have a different bit for the XEmacs/Emacs
 > difference, rather than having to repeat the entire setq twice.

I thought something like that should be possible, and tried similar
things, though with a cond around the cons cell rather the cdr.  Your
scheme looks very nice, but Emacs tells me

Debugger entered--Lisp error: (invalid-read-syntax ". in wrong context")
  eval-buffer(#<buffer  *load*> nil "nrnhoc" nil t)
  ;;; Reading at buffer position 9577
  load-with-code-conversion("/home/sterratt/datastore/share/emacs/site-lisp/nrnhoc.el" "nrnhoc" nil nil)
  nrnhoc-mode()

when I try it with the comma in.  I thought the comma might be a typo,
so left it out, but instead I got:

Debugger entered--Lisp error: (wrong-type-argument stringp (if (string-match "XEmacs" (emacs-version)) ". 1456" ". 124b"))
  modify-syntax-entry(47 (if (string-match "XEmacs" (emacs-version)) ". 1456" ". 124b") #^[t nil (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (2097164) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (0) (1) (7) (1) (2) (2) (3) (1) (4 . 41) (5 . 40) (3) (3) (1) (3) (1) (3) ...])
  font-lock-set-defaults()
  font-lock-mode-internal(t)

Thanks for you interest.  

David.