regexp font-lock highlighting

regexp font-lock highlighting

[martin rudalics]

The recent modification of `lisp-font-lock-keywords-2' to highlight
subexpressions of regexps has two minor bugs:

(1) If you attempt to write the regexp to match the string "\\)" as
     "\\\\\\\\)" the last three chars of that regexp are highlighted with
     `font-lock-comment-face'.

(2) If the region enclosed by the arguments START and END of
     `font-lock-fontify-keywords-region' contains one of "\\(", "\\|",
     "\\)" within a comment, doc-string, or key definition, all
     subsequent occurrences within a normal string are _not_ highlighted.
     `font-lock-fontify-keywords-region' goes to START when it evaluates
     your lambda, decides that the expression should not get highlighted
     since it has the wrong face, and wrongly concludes that no such
     expression exists up to END.

The following lambda should avoid these problems:

        ((lambda (bound)
           (catch 'found
             (while (re-search-forward "\\(\\\\\\\\\\)\\(?:\\(\\\\\\\\\\)\\|\\([(|)]\\)\\(\\?:\\)?\\)" bound t)
               (unless (match-beginning 2)
                 (let ((face (get-text-property (1- (point)) 'face)))
                   (when (or (and (listp face)
                                  (memq 'font-lock-string-face face))
                             (eq 'font-lock-string-face face))
                     (throw 'found t)))))))
         ;; Should we introduce a lowlight face for this?
         ;; Ideally that would retain the color, dimmed.
         (1 'font-lock-comment-face prepend)
         (3 'bold prepend)
         (4 font-lock-type-face prepend t))



Moreover I don't think that anything is "broken" in the following:

        ;; Underline innermost grouping, so that you can more easily see what
        ;; belongs together.  2005-05-12: Font-lock can go into an
        ;; unbreakable endless loop on this -- something's broken.
        ;;("[\\][\\][(]\\(?:\\?:\\)?\\(\\(?:[^\\\"]+\\|[\\]\\(?:[^\\]\\|[\\][^(]\\)\\)+?\\)[\\][\\][)]"
	 ;;1 'underline prepend)

I believe that `font-lock-fontify-keywords-region' starts backtracking
and this can take hours in more complicated cases.  Anyway, regexps are
not suited to handle this.  If you are willing to pay for two additional
buffer-local variables such as

(defvar regexp-left-paren nil
   "Position of innermost unmatched \"\\\\(\".
The value of this variable is valid iff `regexp-left-paren-end' equals the upper
bound of the region `font-lock-fontify-keywords-region' currently investigates.")
(make-variable-buffer-local 'regexp-left-paren)

(defvar regexp-left-paren-end 0
   "Buffer position indicating whether the value of `regexp-left-paren' is valid.
If the value of this variable equals the value of the upper bound of the region
investigated by `font-lock-fontify-keywords-region' the current value of
`regexp-left-paren' is valid.")
(make-variable-buffer-local 'regexp-left-paren-end)

the following modification of the above lambda expression should handle
this problem:

        ((lambda (bound)
           (catch 'found
             (while (re-search-forward
                     "\\(\\\\\\\\\\)\\(?:\\(\\\\\\\\\\)\\|\\(\\((\\)\\|\\(|\\)\\|\\()\\)\\)\\)" bound t)
               (when (match-beginning 3)
                 (let ((face (get-text-property (1- (point)) 'face))
                       match-data-length)
                   (when (or (and (listp face)
                                  (memq 'font-lock-string-face face))
                             (eq 'font-lock-string-face face))
                     (cond
                      ((match-beginning 4) ; \\(
                       (setq regexp-left-paren (match-end 4))
                       (setq regexp-left-paren-end bound)
                       (set-match-data
                        (append (butlast (match-data) 2)
                                (list (point-min-marker) (point-min-marker)))))
                      ((match-beginning 5) ; \\|
                       (set-match-data
                        (append (butlast (match-data) 4)
                                (list (point-min-marker) (point-min-marker)))))
                      ((match-beginning 6) ; \\)
                       (set-match-data
                        (append (butlast (match-data) 6)
                                (if (= regexp-left-paren-end bound)
                                    (list (copy-marker regexp-left-paren) (match-beginning 6))
                                  (list (point-min-marker) (point-min-marker)))))
                       (setq regexp-left-paren nil)
                       (setq regexp-left-paren-end 0)))
                     (throw 'found t)))))))
         ;; Should we introduce a lowlight face for this?
         ;; Ideally that would retain the color, dimmed.
         (1 'font-lock-comment-face prepend)
         (3 'bold prepend)
         (4 'underline prepend))

I have tried this on some elisp files which had the original solution
choke and did not encounter any problems.  Note that I removed the
"\\(\\?:\\)?" since I find it distracting to put yet another face here.
If you believe that you _really_ need it you will have to reinsert it,
but in that case you have to modify match-data cropping as well.  (I do
have to modify match-data since redisplay wants some valid buffer
positions for highlighting.)



Finally, I would use three distinct font-lock faces for regexps:

- One face for highlighting the "\\"s which by default should inherit
   from `font-lock-string-face' with a dimmed foreground - I'm using
   Green4 for strings and PaleGreen3 for the "\\"s.  Anyone who doesn't
   like the highlighting could revert to `font-lock-string-face'.

- One face for highlighting the "(", "|" and ")" in these expressions.
   I find `bold' good here but again would leave it to the user whether
   she wants to turn off highlighting this.  Moreover, such a face could
   allow paren-highlighting to _never_ match a paren with that face with
   a paren with another face.  Consequently, paren-matching could finally
   provide more trustable information within regular expressions.

- One face for highlighting the innermost grouping.  Basically,
   `underline' is not bad here but appears a bit noisy in multiline
   expressions or things like

   (concat "\\("
           some-string
           "\\)")

   I'm using a background which is slightly darker than the default
   background and gives regular expressions a very distinguished
   appearance.  Anyway, users should be allowed to turn highlighting off
   by using the default face.

Re: regexp font-lock highlighting

[Daniel Brockman]

martin rudalics <rudalics@gmx.at> writes:

> The recent modification of `lisp-font-lock-keywords-2' to highlight
> subexpressions of regexps has two minor bugs:

[...]

> Finally, I would use three distinct font-lock faces for regexps:

FWIW, I wholeheartedly agree.  

> - One face for highlighting the "\\"s which by default should
>   inherit from `font-lock-string-face' with a dimmed foreground -
>   I'm using Green4 for strings and PaleGreen3 for the "\\"s.

That's exactly what I would like to do.

>   Anyone who doesn't like the highlighting could revert to
>   `font-lock-string-face'.

> - One face for highlighting the "(", "|" and ")" in these
>   expressions.  I find `bold' good here but again would leave it to
>   the user whether she wants to turn off highlighting this.
>   Moreover, such a face could allow paren-highlighting to _never_
>   match a paren with that face with a paren with another face.
>   Consequently, paren-matching could finally provide more trustable
>   information within regular expressions.

That would be awesome.

I'm sorry, I don't have anything to add.  I just felt the need to
support this suggestion.  I have been thinking along the same lines,
only not as well as Martin has. :-)

-- 
Daniel Brockman <daniel@brockman.se>

Re: regexp font-lock highlighting

[Richard Stallman]

    Moreover I don't think that anything is "broken" in the following:

	    ;; Underline innermost grouping, so that you can more easily see what
	    ;; belongs together.  2005-05-12: Font-lock can go into an
	    ;; unbreakable endless loop on this -- something's broken.
	    ;;("[\\][\\][(]\\(?:\\?:\\)?\\(\\(?:[^\\\"]+\\|[\\]\\(?:[^\\]\\|[\\][^(]\\)\\)+?\\)[\\][\\][)]"
	     ;;1 'underline prepend)

    I believe that `font-lock-fontify-keywords-region' starts backtracking
    and this can take hours in more complicated cases.

That sounds like "broken" to me.  Whether it takes hours, or forever,
either way Font Lock mode will be very unpleasant to use.

      Anyway, regexps are
    not suited to handle this.  If you are willing to pay for two additional
    buffer-local variables such as

I believe you, and it is ok to add local variables for this purpose.
(Their names should start with `font-lock-' so that people understand
what they are for.  Or perhaps with `lisp-fl-' since they are
specifically for Lisp mode.)

Your code is large enough that we need papers to install it.
I will send you some instructions in another message.
Meanwhile, I will revert the change that introduced this code.
The idea was good, but the current code is not good.

    Finally, I would use three distinct font-lock faces for regexps:

Those suggestions look good to me.

Re: regexp font-lock highlighting

[martin rudalics]

 >     Moreover I don't think that anything is "broken" in the following:
 >
 > 	    ;; Underline innermost grouping, so that you can more easily see what
 > 	    ;; belongs together.  2005-05-12: Font-lock can go into an
 > 	    ;; unbreakable endless loop on this -- something's broken.
 > 	    ;;("[\\][\\][(]\\(?:\\?:\\)?\\(\\(?:[^\\\"]+\\|[\\]\\(?:[^\\]\\|[\\][^(]\\)\\)+?\\)[\\][\\][)]"
 > 	     ;;1 'underline prepend)
 >
 >     I believe that `font-lock-fontify-keywords-region' starts backtracking
 >     and this can take hours in more complicated cases.
 >
 > That sounds like "broken" to me.  Whether it takes hours, or forever,
 > either way Font Lock mode will be very unpleasant to use.
 >
 >       Anyway, regexps are
 >     not suited to handle this.  If you are willing to pay for two additional
 >     buffer-local variables such as
 >
 > I believe you, and it is ok to add local variables for this purpose.
 > (Their names should start with `font-lock-' so that people understand
 > what they are for.  Or perhaps with `lisp-fl-' since they are
 > specifically for Lisp mode.)
 >

Note that I only fixed the endless looping behavior here.  My code is
not suitable when you use a support mode like jit-lock and change the
delimiters of regexp groups.  You would have to refontify text manually
which is not very desirable.  I'll try to handle this in a more
user-friendly way.

Re: regexp font-lock highlighting

[Richard Stallman]

    Note that I only fixed the endless looping behavior here.  My code is
    not suitable when you use a support mode like jit-lock and change the
    delimiters of regexp groups.

I don't understand that last sentence.  What does it mean to "change
the delimiters of regexp groups"?  Could you please show us a concrete
test case that still has a problem--and describe the problem exactly?

Re: regexp font-lock highlighting

[martin rudalics]

 >     Note that I only fixed the endless looping behavior here.  My code is
 >     not suitable when you use a support mode like jit-lock and change the
 >     delimiters of regexp groups.
 >
 > I don't understand that last sentence.  What does it mean to "change
 > the delimiters of regexp groups"?  Could you please show us a concrete
 > test case that still has a problem--and describe the problem exactly?
 >

Sorry, I couldn't understand that sentence either.  It's not so easy to
show a concrete test case since the problem depends on the particular
settings of font-lock and jit-lock.  Consider the following stretch from
`tex-mode.el'.

   (setq paragraph-start
	(concat "[ \t]*\\(\\$\\$\\|"
		"\\\\[][]\\|"
		"\\\\" (regexp-opt (append
				    (mapcar 'car latex-section-alist)
				    '("begin" "label" "end"
				      "item" "bibitem" "newline" "noindent"
				      "newpage" "footnote" "marginpar"
				      "parbox" "caption")) t)
		"\\>\\|\\\\[a-z]*" (regexp-opt '("space" "skip" "page") t)
		"\\>\\)"))

Here the innermost group delimited by "\\(" and "\\)" stretches from the
second to the last line and should get painted accordingly.  If you now
delete the "\\)" the painting should disappear.  If you next undo the
change, the painting generally doesn't reappear since font-lock
refontifies from the line preceding the one where the change occurred -
with `font-lock-lines-before' at its default value - only.  The reason
for this is documented in the Elisp manual:


    *Warning:* Do not design an element of `font-lock-keywords' to match
text which spans lines; this does not work reliably.  While
`font-lock-fontify-buffer' handles multi-line patterns correctly,
updating when you edit the buffer does not, since it considers text one
line at a time.  If you have patterns that typically only span one line
but can occasionally span two or three, such as `<title>...</title>',
you can ask Font Lock to be more careful by setting
`font-lock-multiline' to `t'.  But it still will not work in all cases.


I could avoid painting regexp groups that span multiple lines, but this
would deprive me from highlighting many interesting things.  I could set
`font-lock-lines-before' to a higher value but this would slow down
fontification in general and still not handle all cases.  Anyway, I
believe that I found a solution that does the right thing in most cases
and will send it to you in the next days.

Re: regexp font-lock highlighting

[Richard Stallman]

    I could avoid painting regexp groups that span multiple lines, but this
    would deprive me from highlighting many interesting things.  I could set
    `font-lock-lines-before' to a higher value but this would slow down
    fontification in general and still not handle all cases.

The current code seems simply to highlight all the \\( and \\)
constructs.  That seems to be reliable, and sufficient for me at least
to match them up.

So I am not sure this ought to be changed further.
Maybe we should leave well enough alone.

							      Anyway, I
    believe that I found a solution that does the right thing in most cases
    and will send it to you in the next days.

Could you dscribe in words what it does?

Re: regexp font-lock highlighting

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 8160 bytes --]

Richard Stallman wrote:
 >     believe that I found a solution that does the right thing in most cases
 >     and will send it to you in the next days.
 >
 > Could you dscribe in words what it does?

Attached find a file called `lisp-font-lock-regexp.el' which contains
all changes I propose.  You may try to load it, make the face
definitions meet your requirements, and look whether it works.
Syntax-highlighting and decoration for lisp-font-lock-keywords-2 must be
activated.  Eventually someone would have to decide on appropriate
names and defaults for faces.

I have set regexp highlighting to the minimum level 1.  If this were
incorporated in font-lock.el, the standard level should be 0 - which
means no regexp highlighting and thus no obtrusiveness.  Emacs would
behave as before the introduction of regexp highlighting a couple of
weeks ago.  Level 1 does regexp highlighting as introduced recently with
some minor bug fixes.

Levels 2 and 3 should do something that was proposed in font-lock.el but
commented out due to problems with an "unbreakable endless loop".  Level
2 does this for regexp groups on a single line only.  Level 3 should
handle regexp groups spanning several lines as well.  By no means the
default level should equal 3 as will become evident from remarks below.

The variable `lisp-font-lock-regexp' can be used to set the default
level.  Individual buffer settings can be achieved by using the command
`lisp-font-lock-regexp'.


Levels 2 and 3 use the syntax-table property to remove parenthesis
syntax from unescaped parentheses and escaped brackets within regexp
groups.  I added syntax-table to `font-lock-extra-managed-props' since I
don't want font-lock to perform the extra syntactic fontification pass.
This idea is non-standard and could be defeated by anyone who removed
syntax-table from that list - so far no one seems to use syntax-table
properties in elisp-mode.

With that property paren-matching/blinking and forward/backward-sexp
should work "as intended" within parenthetical groups.  You may have
noticed my simple-minded posting on emacs-pretest-bug about forward-sexp
not being able to handle unescaped semicolons within strings.  I
resolved the problem by setting the syntax-table property of `;' to
punctuation within regexp groups.  For a similar reason I reset the
escape syntax property of single backslashes preceding parentheses and
brackets.

I do not treat special characters "as ordinary ones if they are in
contexts where their special meanings make no sense".  Hence,
subexpressions like

\\(\\[[^]]*]\\)* in `reftex-extract-bib-entries-from-thebibliography'

\\(\\[[^\\]]*\\]\\)? in `reftex-all-used-citation-keys'

\\`\\(\\\\[sS]?.\\|\\[\\^?]?[^]]*]\\|[^\\]\\) in `gnus-score-regexp-bad-p'

\\(\[[0-9]+\] \\)* in `gud-jdb-marker-filter'

do contain mismatches.

With level 3 highlighting I'm using the font-lock-multiline property.
Apparently this property is used by `smerge.el' too.  Consequently, I
cannot simply reset the variable `font-lock-multiline' to nil when I
switch to a lower level.  I believe that this variable - and the
variable `parse-sexp-lookup-properties' as well - should be handled in a
way similar to hooks or `buffer-invisibility-spec'.  Anyone who wants to
set these variables should create or append its name to a corresponding
list and remove its name to eventually reset the variable.  Routines
checking the value of the variable would not be affected by this
convention.  Likely font-lock-multiline, syntax-table and
`lisp-font-lock-regexp' prefixed properties should be added to
`yank-excluded-properties' too.


I've been experimenting a bit with level 3 highlighting.  With a 200MHz
PC the results are negative: Fontifiying a buffer is moderatly slow,
modifying text is hardly supportable.  With a 1GHz PC I did not
encounter substantial difficulties with one exception - fontifying
`cperl-init-faces' took a couple of seconds.  I tried to look at bit
closer what's going on.

When I scrolled down through `cperl.el' and looked at what font-lock is
doing I found out that the range from position 168761 to 172839 gets
fontified no less than _seven_ times in sequence: Apparently `xdisp.c' -
encountering an unfontified object at a position START - asks
`jit-lock-function' to fontify from position START.  jit-lock-function
now calls `jit-lock-fontify-now' to fontify from START to (+ START
jit-lock-chunk-size).  The latter sets the fontified property for this
region to t.  `font-lock-default-fontify-region' detects that there is a
font-lock-multiline pattern, fontifies the entire region from beginning
to end of the pattern - the 168761 to 172839 region above - but does not
set the fontified property for this region.

I simply inserted `(put-text-property beg end 'fontified t)' in the text
of `font-lock-default-fontify-region' right before it calls
`font-lock-unfontify-region' and the problem disappeared.

When I change some text within a font-lock-multiline pattern of
`cperl-init-faces' font-lock refontifies the entire area twice which can
take a couple of seconds.  What happens here?  The first refontification
is triggered by redisplay which encounters an unfontified thing it
should display (the thing was unfontified by `jit-lock-after-change'
previously).  The second refontification is eventually triggered by
`jit-lock-context-fontify' which unfontifies everything from
`jit-lock-context-unfontify-pos' until point-max.  However, the second
refontification is useless because font-lock-default-fontify-region
already took care of the font-lock multiline pattern.  Moreover, the
second fontification usualy occurs right after the first has finished
_before_ I am able to enter the next character.

I could resolve this by having font-lock-default-fontify-region
fontify a region iff it has not fontified exactly that region already
since the last modification of the buffer.  But font-lock-multiline
patterns do not seem suited for handling this problem anyway.  Patterns
spanning more than a couple of lines - your mileage may vary - will
delay redisplay because inserting one single character triggers
refontification of the _entire_ pattern.  It should be possible to
resolve this problem by using the `jit-lock-defer-multiline' property.
However, the latter is broken.

Suppose I used jit-lock-defer-multiline instead of font-lock-multiline
for my pattern.  Inserting a character now will not delay redisplay
anymore since font-lock-default-fontify-region does not cater for
jit-lock-defer-multiline.  Eventually, jit-lock-context-fontify will
unfontify the relevant parts of my buffer from the start of the pattern
to point-max, and everything should get fontified correctly.  It does
not, however, when the jit-lock-defer-multiline pattern starts _before_
`window-start': After jit-lock-context-fontify has unfontified the
buffer, redisplay - for some reason I did not investigate - intercepts
this by fontifying the _visible_ part of the buffer without caring about
my pattern.  Eventually, the invisible parts get refontified but the
already fontified part doesn't because, as mentioned before,
font-lock-default-fontify-region does not know jit-lock-defer-multiline
patterns.  Hence, fontification appears incorrect.

I'm afraid there are no simple patches for this.  Hence I provided the
appropriate warnings that level 3 highlighting should be used with
sufficient care.


The feature I propose could be quite useful for people who write regular
expressions only occasionally and I don't want to compromise it on
behalf of the recent controversies on font-lock-comment-delimiter and
font-lock-negation-char-face faces.  On the other hand, I don't want to
give pretext to anyone who plans to introduce yet another feature in the
pre-release phase.  Hence if you think that this should be delayed or
cancelled please tell me so.

I've also experimented with a patch of `show-paren-function' where I
overlay the backslashes in `\\(...\\)' groups with the respective count
of that group.  Hence I don't have to literally step through such pairs
when searching for the subexpressions referenced by match-string,
match-beginning, ...

[-- Attachment #2: lisp-font-lock-regexp.el --]
[-- Type: text/plain, Size: 21762 bytes --]

(defface lisp-font-lock-regexp-delimiter
  '((t (:bold t)))
  "Face for highlighting regexp group delimiters and brackets."
  :group 'font-lock-highlighting-faces)

(defface lisp-font-lock-regexp-backslash
  '((t (:foreground "PaleGreen3")))
  "Face for highlighting the backslash part of regexp group delimiters."
  :group 'font-lock-highlighting-faces)

(defface lisp-font-lock-regexp-group
  '((t (:background "Grey86")))
  "Face for highlighting inner regexp groups."
  :group 'font-lock-highlighting-faces)

(defun lisp-font-lock-regexp-hook ()
  "Automatically turn on regexp highlighting in `emacs-lisp-mode'."
  (setq lisp-font-lock-regexp lisp-font-lock-regexp) ; set buffer-local value
  (when (> lisp-font-lock-regexp 1)
    (set (make-local-variable 'parse-sexp-lookup-properties) t)
    (when (> lisp-font-lock-regexp 2)
      (set (make-local-variable 'font-lock-multiline) t))
    (set (make-local-variable 'font-lock-extra-managed-props)
         (append font-lock-extra-managed-props
                 (list 'syntax-table
                       'lisp-font-lock-regexp
                       'lisp-font-lock-regexp-group
                       'lisp-font-lock-regexp-alt)))))

(defcustom lisp-font-lock-regexp 1
  "*Highlight regular expression in `emacs-lisp-mode'.

The following levels are available:

0 (off) do no highlight regular expressions specially.

1 (minimum) highlight the non-backslash parts of regexp group delimiters with
  `lisp-font-lock-regexp-delimiter' face and delimiter backslashes with
  `lisp-font-lock-regexp-backslash' face.  Group delimiters are the
  backslash-sequences `\\(' `\\(?:' `\\|' and `\\)'.  Delimiters appearing in
  documentation strings or non-string text are not highlighted.  Within proper
  strings, however, *every* instance of such a delimiter will be highlighted
  regardless of its actual or intended semantics.  Hence, you should use these
  backslash-sequences *exclusively* for parenthetical grouping of regexps.  For
  other purposes try something like `(concat \"\\\\\" \"(\")' instead.  Within
  character alternatives write `)\\\\' instead of `\\\\('.

2 (medium) as 1 but also highlight brackets delimiting character alternatives
  within single-line regexp groups with `lisp-font-lock-regexp-delimiter' face.
  Moreover, highlight inner regexp groups with `lisp-font-lock-regexp-group'
  face.  Inner regexp groups are character sequences within `\\(...\\)' and
  `\\(?:...\\)' that appear on a single line and do not contain one of the
  backslash-sequences `\\(' `\\(?:' or `\\)'.  Inner regexp groups may contain
  non-string text provided the respective delimiters appear within a string.

  In addition, 2 will try to set the syntax-table properties of parentheses,
  brackets and semicolons within single-line regexp groups appropriately.  More
  precisely, brackets that do not delimit a character alternative or class,
  parentheses that do not delimit a group, semicolons, and single backslashes
  preceding a parenthesis or bracket, are classified as punctuation characters.

  Note that you can always create a surrounding group with the shy group
  delimiters `\\(?:...\\)' without modifying the semantics of enclosed regexps.

3 (maximum) as 2 but permit operations on regexp groups spanning several lines.
  This option exploits the `font-lock-multiline' text-property which is not
  guaranteed to work reliably and is notorious for delaying redisplay
  considerably.  Hence use this option with *extreme* care!

Setting the default value of this variable does not affect highlighting of live
buffers.  Use the command `lisp-font-lock-regexp' to change highlighting for the
current buffer only."
  :type '(choice (const :tag "off" 0)
                 (const :tag "minimum" 1)
                 (const :tag "medium" 2)
                 (const :tag "maximum" 3))
  :set (lambda (symbol value)
         (set-default symbol value)
         (remove-hook 'emacs-lisp-mode-hook 'lisp-font-lock-regexp-hook)
         (when (and (boundp 'font-lock-mode) ; silly if this is part of font-lock
                    (> value 0))
           (custom-add-option 'emacs-lisp-mode-hook 'lisp-font-lock-regexp-hook)
           (add-hook 'emacs-lisp-mode-hook 'lisp-font-lock-regexp-hook)))
  :version "22.1"
  :group 'font-lock)
(make-variable-buffer-local 'lisp-font-lock-regexp)

(defconst lisp-font-lock-keywords-2
  (append
   lisp-font-lock-keywords-1
   (eval-when-compile
     `( ;; Control structures.  Emacs Lisp forms.
       (,(concat
          "(" (regexp-opt
               '("cond" "if" "while" "let" "let*"
                 "prog" "progn" "progv" "prog1" "prog2" "prog*"
                 "inline" "lambda" "save-restriction" "save-excursion"
                 "save-window-excursion" "save-selected-window"
                 "save-match-data" "save-current-buffer" "unwind-protect"
                 "condition-case" "track-mouse"
                 "eval-after-load" "eval-and-compile" "eval-when-compile"
                 "eval-when"
                 "with-category-table"
                 "with-current-buffer" "with-electric-help"
                 "with-local-quit" "with-no-warnings"
                 "with-output-to-string" "with-output-to-temp-buffer"
                 "with-selected-window" "with-syntax-table"
                 "with-temp-buffer" "with-temp-file" "with-temp-message"
                 "with-timeout" "with-timeout-handler") t)
          "\\>")
        .  1)
       ;; Control structures.  Common Lisp forms.
       (,(concat
          "(" (regexp-opt
               '("when" "unless" "case" "ecase" "typecase" "etypecase"
                 "ccase" "ctypecase" "handler-case" "handler-bind"
                 "restart-bind" "restart-case" "in-package"
                 "break" "ignore-errors"
                 "loop" "do" "do*" "dotimes" "dolist" "the" "locally"
                 "proclaim" "declaim" "declare" "symbol-macrolet"
                 "lexical-let" "lexical-let*" "flet" "labels" "compiler-let"
                 "destructuring-bind" "macrolet" "tagbody" "block" "go"
                 "multiple-value-bind" "multiple-value-prog1"
                 "return" "return-from"
                 "with-accessors" "with-compilation-unit"
                 "with-condition-restarts" "with-hash-table-iterator"
                 "with-input-from-string" "with-open-file"
                 "with-open-stream" "with-output-to-string"
                 "with-package-iterator" "with-simple-restart"
                 "with-slots" "with-standard-io-syntax") t)
          "\\>")
        . 1)
       ;; Exit/Feature symbols as constants.
       (,(concat "(\\(catch\\|throw\\|featurep\\|provide\\|require\\)\\>"
                 "[ \t']*\\(\\sw+\\)?")
        (1 font-lock-keyword-face)
        (2 font-lock-constant-face nil t))
       ;; Erroneous structures.
       ("(\\(abort\\|assert\\|warn\\|check-type\\|cerror\\|error\\|signal\\)\\>" 1 font-lock-warning-face)
       ;; Words inside \\[] tend to be for `substitute-command-keys'.
       ("\\\\\\\\\\[\\(\\sw+\\)\\]" 1 font-lock-constant-face prepend)
       ;; Words inside `' tend to be symbol names.
       ("`\\(\\sw\\sw+\\)'" 1 font-lock-constant-face prepend)
       ;; Constant values.
       ("\\<:\\sw+\\>" 0 font-lock-builtin-face)
       ;; ELisp and CLisp `&' keywords as types.
       ("\\&\\sw+\\>" . font-lock-type-face)
       ((lambda (bound)
          (when (and (local-variable-p 'lisp-font-lock-regexp)
                     (not (zerop lisp-font-lock-regexp)))
            (while (cond
                    ((and (= lisp-font-lock-regexp 3)
                          (get-text-property (point) 'lisp-font-lock-regexp))
                     (re-search-forward
                      "\\(?:\\(\\\\\\\\\\)\\(?:\\(?:\\\\\\\\\\)\\|\\((\\(?:\\?:\\)?\\)\\|\\(|\\)\\|\\()\\)\\|\\([][]\\)\\)\
\\|\\(\\\\[][()]\\)\\|\\(\\[:[a-zA-Z]+:\\]\\)\\|\\(\\[\\)\\|\\(\\]\\)\\|\\([(;)]\\)\\)" bound 'bound))
                    ((and (= lisp-font-lock-regexp 2)
                          (or (and (get-text-property (point) 'lisp-font-lock-regexp)
                                   (re-search-forward
                                    "\\(?:\\(\\\\\\\\\\)\\(?:\\(?:\\\\\\\\\\)\\|\\((\\(?:\\?:\\)?\\)\\|\\(|\\)\\|\\()\\)\\|\\([][]\\)\\)\
\\|\\(\\\\[][()]\\)\\|\\(\\[:[a-zA-Z]+:\\]\\)\\|\\(\\[\\)\\|\\(\\]\\)\\|\\([(;)]\\)\\|\\($\\)\\)" bound 'eol)
                                   (not (match-beginning 11)))
                              (re-search-forward
                               "\\(?:\\(\\\\\\\\\\)\\(?:\\(?:\\\\\\\\\\)\\|\\((\\(?:\\?:\\)?\\)\\|\\(|\\)\\|\\()\\)\\)\\)"
                               bound 'bound))))
                    (t (re-search-forward
                        "\\(?:\\(\\\\\\\\\\)\\(?:\\(?:\\\\\\\\\\)\\|\\((\\(?:\\?:\\)?\\)\\|\\(|\\)\\|\\()\\)\\)\\)"
                        bound 'bound)))
              (let ((face (get-text-property (1- (point)) 'face)))
                (when (or (and (listp face)
                               (memq 'font-lock-string-face face))
                          (eq 'font-lock-string-face face))
                  (cond
                   ((match-beginning 2) ; \\(
                    (font-lock-prepend-text-property
                     (match-beginning 1) (match-end 1)
                     'face 'lisp-font-lock-regexp-backslash)
                    (font-lock-prepend-text-property
                     (match-beginning 2) (match-end 2)
                     'face 'lisp-font-lock-regexp-delimiter)
                    (when (> lisp-font-lock-regexp 1)
                      (let* ((level (or (get-text-property
                                         (point) 'lisp-font-lock-regexp)
                                        0))
                             (from (match-beginning 1))
                             (mid (match-end 2))
                             (to (or (and (= lisp-font-lock-regexp 2)
                                          (line-end-position))
                                     (and (> level 1)
                                          (next-single-property-change
                                           (point) 'lisp-font-lock-regexp))
                                     (save-excursion
                                         ;; beginning of next defun
                                         (if (re-search-forward "^(" nil t)
                                             (1- (point))
                                           (point-max))))))
                        (put-text-property from to 'lisp-font-lock-regexp (1+ level))
                        (put-text-property ; `\\(' is not part of inner group
                         from mid 'lisp-font-lock-regexp-group nil)
                        (put-text-property mid to 'lisp-font-lock-regexp-group t)
                        (when (> lisp-font-lock-regexp 2)
                          (put-text-property from to 'font-lock-multiline t)))))
                   ((match-beginning 3) ; \\|
                    (font-lock-prepend-text-property
                     (match-beginning 1) (match-end 1)
                     'face 'lisp-font-lock-regexp-backslash)
                    (font-lock-prepend-text-property
                     (match-beginning 3) (match-end 3)
                     'face 'lisp-font-lock-regexp-delimiter))
                   ((match-beginning 4) ; \\)
                    (let ((level
                           (when (> lisp-font-lock-regexp 1)
                             (get-text-property (point) 'lisp-font-lock-regexp))))
                      (cond
                       ((and level
                             (not (get-text-property
                                   (point) 'lisp-font-lock-regexp-alt)))
                        (let* ((from
                                (when (get-text-property
                                       (point) 'lisp-font-lock-regexp-group)
                                  (previous-single-property-change
                                   (point) 'lisp-font-lock-regexp-group)))
                               (to (or (and (= lisp-font-lock-regexp 2)
                                            (line-end-position))
                                       (next-single-property-change
                                        (point) 'lisp-font-lock-regexp)
                                       (point-max))))
                          (font-lock-prepend-text-property
                           (match-beginning 1) (match-end 1)
                           'face 'lisp-font-lock-regexp-backslash)
                          (font-lock-prepend-text-property
                           (match-beginning 4) (match-end 4)
                           'face 'lisp-font-lock-regexp-delimiter)
                          (if (> level 1)
                              (put-text-property
                               (match-end 4) to 'lisp-font-lock-regexp (1- level))
                            (remove-text-properties
                             (match-end 4) to '(lisp-font-lock-regexp nil))
                            (when (> lisp-font-lock-regexp 2)
                              (remove-text-properties
                               (match-end 4) to '(font-lock-multiline nil))))
                          (when from
                            (remove-text-properties
                             (match-end 4) to '(lisp-font-lock-regexp-group nil))
                            (font-lock-prepend-text-property
                             from (match-beginning 1)
                             'face 'lisp-font-lock-regexp-group))))
                       ((not level)
                        ;; no open \\( or lisp-font-lock-regexp equals 1
                        (font-lock-prepend-text-property
                         (match-beginning 1) (match-end 1)
                         'face 'lisp-font-lock-regexp-backslash)
                        (font-lock-prepend-text-property
                         (match-beginning 4) (match-end 4)
                         'face 'lisp-font-lock-regexp-delimiter))
                       ((get-text-property (point) 'lisp-font-lock-regexp)
                        (put-text-property
                         (1- (point)) (point) 'syntax-table '(3))))))
                   ;; matches below should occur within parenthetical groups
                   ((match-beginning 5) ; \\[ or \\]
                    (if (get-text-property (point) 'lisp-font-lock-regexp-alt)
                        ;; within alternative
                        (goto-char (1- (point)))
                      (put-text-property
                       (1- (point)) (point) 'syntax-table '(3))))
                   ((match-beginning 6) ; escaped parenthesis or bracket
                    (put-text-property
                     (match-beginning 0) (1+ (match-beginning 0))
                     'syntax-table '(3))
                    ;; reread paren
                    (goto-char (1+ (match-beginning 0))))
                   ((match-beginning 7))
                    ;; POSIX character class, skip to preserve paren syntax
                   ((match-beginning 8) ; [
                    (if (get-text-property (point) 'lisp-font-lock-regexp-alt)
                        ;; already within alternative
                        (put-text-property
                         (1- (point)) (point) 'syntax-table '(3))
                      ;; starting new alternative
                      (let ((to (or (and (= lisp-font-lock-regexp 2)
                                         (line-end-position))
                                    (next-single-property-change
                                     (point) 'lisp-font-lock-regexp)
                                    (point-max))))
                        (font-lock-prepend-text-property
                         (match-beginning 8) (match-end 8)
                         'face 'lisp-font-lock-regexp-delimiter)
                        (put-text-property
                         ;; the following should be reset at \\)
                         (point) to 'lisp-font-lock-regexp-alt t))))
                   ((match-beginning 9) ; ]
                    (let* ((from
                            (when (get-text-property
                                   (point) 'lisp-font-lock-regexp-alt)
                              (previous-single-property-change
                               (point) 'lisp-font-lock-regexp-alt))))
                      (cond
                       ((not from))
                       ;; here likely some cases are missing
                       ((or (and (char-equal (char-before (1- (point))) ?\[ )
                                 ;; []
                                 (= from (1- (point))))
                            (and (char-equal (char-before (1- (point))) ?^ )
                                 (or (and (char-equal (char-before (- (point) 2)) ?\[ )
                                          ;; [^]
                                          (= from (- (point) 2)))
                                     (and (char-equal (char-before (- (point) 2)) ?\\ )
                                          (char-equal (char-equal (- (point) 3)) ?\[ )
                                          ;; [\^]
                                          (= from (- (point) 3)))))
                            (and (char-equal (char-before (1- (point))) ?\\ )
                                 (or (and (char-equal (char-before (- (point) 2)) ?\[ )
                                          ;; [\]
                                          (= from (- (point) 2)))
                                     (and (char-equal (char-before (- (point) 2)) ?^ )
                                          (char-equal (char-before (- (point) 3)) ?\[ )
                                          ;; [^\]
                                          (= from (- (point) 3)))
                                     (and (char-equal (char-before (- (point) 2)) ?\^ )
                                          (char-equal (char-equal (- (point) 3)) ?\\ )
                                          (char-equal (char-equal (- (point) 4)) ?\[ )
                                          ;; [\^\]
                                          (= from (- (point) 4))))))
                        (put-text-property
                         (1- (point)) (point) 'syntax-table '(3)))
                       ((< from (match-beginning 9))
                        (let ((to (or (and (= lisp-font-lock-regexp 2)
                                           (line-end-position))
                                      (next-single-property-change
                                       (point) 'lisp-font-lock-regexp-alt)
                                      (point-max))))
                          (font-lock-prepend-text-property
                           (match-beginning 9) (match-end 9)
                           'face 'lisp-font-lock-regexp-delimiter)
                          (remove-text-properties
                           (point) to '(lisp-font-lock-regexp-alt nil)))))))
                   ((match-beginning 10) ; (;)
                    (put-text-property
                     (1- (point)) (point) 'syntax-table '(3)))))))))
        nil))))
  "Gaudy level highlighting for Lisp modes.")

(defun lisp-font-lock-regexp (arg)
  "Set regular expression highlighting for current buffer.

ARG may be one of 0, 1, 2 or 3.  See the documentation of the variable
`lisp-font-lock-regexp' for the meaning of these values.  Regular expression
highlighting works in Emacs-Lisp mode only."
  (interactive
   (if (eq major-mode 'emacs-lisp-mode)
       (list (read-number "Locally highlight regexps 0 (off), 1 (min), 2 (med), 3 (max): "))
     (error "This option can be used in `emacs-lisp-mode' only")))
  (when (and (boundp 'font-lock-mode) font-lock-mode
             (>= arg 0) (<= arg 3) (/= arg lisp-font-lock-regexp))
    (save-excursion
      (setq lisp-font-lock-regexp arg)
      (font-lock-unfontify-buffer)
      ;; can't reset parse-sexp-lookup-properties and font-lock-multiline since
      ;; they might have been set by other programs or the user, this could be
      ;; resolved by handling them like buffer-invisibility-spec
      (setq font-lock-extra-managed-props
            ;; removing syntax-table from font-lock-extra-managed-props is dirty
            ;; provided someone else did put it there; eventually this should be
            ;; handled by font-lock-syntactic-keywords in an appropriate fashion
            (delq 'syntax-table
                  (delq 'lisp-font-lock-regexp
                        (delq 'lisp-font-lock-regexp-group
                              (delq 'lisp-font-lock-regexp-alt
                                    font-lock-extra-managed-props)))))
      ;; the following is needed to have jit-lock refontify the buffer
      (when jit-lock-mode
        (setq jit-lock-context-unfontify-pos (point-min)))
      (when (> arg 1)
        (set (make-local-variable 'parse-sexp-lookup-properties) t)
        (when (> arg 2)
          (set (make-local-variable 'font-lock-multiline) t))
        (set (make-local-variable 'font-lock-extra-managed-props)
             (append font-lock-extra-managed-props
                     (list 'syntax-table
                           'lisp-font-lock-regexp
                           'lisp-font-lock-regexp-group
                           'lisp-font-lock-regexp-alt))))
      (dolist (window (get-buffer-window-list (current-buffer) nil t))
        ;; refontify contents of any window showing current buffer, avoids
        ;; displaying unfontified buffers
        (with-selected-window window
          (font-lock-fontify-region (window-start) (window-end)))))))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: regexp font-lock highlighting

[Juri Linkov]

What were the reasons for making long face names much longer?

After renaming `font-lock-regexp-backslash-construct' it became
absurdly long: `font-lock-regexp-backslash-grouping-construct'.

The face name is not the place for full description of the face.
There are docstrings for that purpose.  But even the new docstring
is less informative than the face name:

"Font Lock mode face used to highlight grouping constructs in Lisp regexps."

Unlike the face name it doesn't mention the word `backslash'.

Long face names cause several problems for users.  For example,
after `M-x customize-face RET' on this face, its name doesn't fit
into the minibuffer:

Customize face (default font-lock-regexp-backslash-grouping-construct): font-lock-regexp-backslash-grouping-construct

In the *Faces* buffer long face names move example alphabets off the screen:

font-lock-doc-face                            abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-function-name-face                  abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-keyword-face                        abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-negation-char-face                  abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-preprocessor-face                   abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-regexp-grouping-backslash           abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-regexp-backslash-grouping-construct abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-string-face                         abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-type-face                           abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-variable-name-face                  abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
font-lock-warning-face                        abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
fringe                                        abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ
header-line                                   abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ

and other inconveniences.

I think the face names proposed by Martin in his latest patch
are much better: `font-lock-regexp-backslash' and
`font-lock-regexp-group'.  These face name have sensible lengths
and are still descriptive enough.

BTW, I noticed that backslashes and grouping characters are more
readable when they both are highlighted in bold, i.e. I also propose
the following change:

Index: lisp/font-lock.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/font-lock.el,v
retrieving revision 1.265
diff -u -r1.265 font-lock.el
--- lisp/font-lock.el	29 Jun 2005 08:19:17 -0000	1.265
+++ lisp/font-lock.el	3 Jul 2005 00:17:25 -0000
@@ -1826,14 +1827,13 @@
   :group 'font-lock-highlighting-faces)
 
-(defface font-lock-regexp-grouping-backslash
-  '((((class color) (min-colors 16)) :inherit escape-glyph)
-    (t :inherit bold))
+(defface font-lock-regexp-backslash
+  '((t :inherit bold))
   "Font Lock mode face for backslashes in Lisp regexp grouping constructs."
   :group 'font-lock-highlighting-faces)
 
-(defface font-lock-regexp-backslash-grouping-construct
+(defface font-lock-regexp-group
   '((t :inherit bold))
-  "Font Lock mode face used to highlight grouping constructs in Lisp regexps."
+  "Font Lock mode face for grouping constructs in Lisp regexps."
   :group 'font-lock-highlighting-faces)

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: regexp font-lock highlighting

[Luc Teirlinck]

Juri Linkov wrote:

   I think the face names proposed by Martin in his latest patch
   are much better: `font-lock-regexp-backslash' and
   `font-lock-regexp-group'.  These face name have sensible lengths
   and are still descriptive enough.

I agree.

Sincerely,

Luc.

Re: regexp font-lock highlighting

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Date: Sun, 03 Jul 2005 03:09:08 +0300
> Cc: martin rudalics <rudalics@gmx.at>
> 
> What were the reasons for making long face names much longer?
> 
> After renaming `font-lock-regexp-backslash-construct' it became
> absurdly long: `font-lock-regexp-backslash-grouping-construct'.

Oops, sorry: I blew it.  The correct name is
font-lock-regexp-grouping-construct.

This is what happens when people send incomplete diffs, and one is
forced to apply them by hand.

I fixed this in CVS.

Re: regexp font-lock highlighting

[martin rudalics]

Eli Zaretskii wrote:
> This is what happens when people send incomplete diffs, and one is
> forced to apply them by hand.

Sorry.  I'll try to make this never happen again.  martin

Re: regexp font-lock highlighting

[Miles Bader]

2005/7/3, martin rudalics <rudalics@gmx.at>:
> Eli Zaretskii wrote:
> > This is what happens when people send incomplete diffs, and one is
> > forced to apply them by hand.
> 
> Sorry.  I'll try to make this never happen again.  martin

BTW, I think it's not unreasonable to ask people to re-send diffs if
they're hard to apply for some reason.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: regexp font-lock highlighting

[Juri Linkov]

> Note that I removed the "\\(\\?:\\)?" since I find it distracting to
> put yet another face here.  If you believe that you _really_ need it
> you will have to reinsert it, but in that case you have to modify
> match-data cropping as well.  (I do have to modify match-data since
> redisplay wants some valid buffer positions for highlighting.)

I fixed shy group handling in this regexp by allowing ?: to be only
before `(' instead of before any of `(', `|' or `)'.  You may need to
adjust your code before submitting your final patch.

Also I tried to extend this regexp to highlight other special backslash
constructs like `\<', `\_<', `\w', `\sC', but soon discovered that
sometimes there are similar constructs in non-regexp strings that become
inappropriately highlighted.  Unless it's possible to detect whether
a given string is a regexp or not, I think the regexp highlighting
code should be split into multi-level font-lock variables
font-lock-regexp-keywords-1 and font-lock-regexp-keywords-2.
The first variable would highlight only `\\(?:', `\\|' and `\\)' constructs,
but the second - all other regexp constructs.  Everyone agreed to see
regexp constructs sometimes highlighted in wrong places could use
the second level variable.

> Finally, I would use three distinct font-lock faces for regexps:
>
> - One face for highlighting the "\\"s which by default should inherit
>   from `font-lock-string-face' with a dimmed foreground - I'm using
>   Green4 for strings and PaleGreen3 for the "\\"s.  Anyone who doesn't
    =====                      =====

I see everyone tends to change the default color of `font-lock-string-face'
(which is barely visible on light backgrounds) to a green color.
Currently on tty strings are fontified in green, so maybe this is one
of the reasons why people prefer green strings.  This indicates that
we should change the current "RosyBrown" default to one of the shades
of green.

>   like the highlighting could revert to `font-lock-string-face'.
>
> - One face for highlighting the "(", "|" and ")" in these expressions.
>   I find `bold' good here but again would leave it to the user whether
>   she wants to turn off highlighting this.  Moreover, such a face could
>   allow paren-highlighting to _never_ match a paren with that face with
>   a paren with another face.  Consequently, paren-matching could finally
>   provide more trustable information within regular expressions.

I added two new faces: for a backslash and regexp constructs after
a backslash.  The former inherits from escape-glyph to be highlighted
like a backslash in escape sequences, the latter is bold by default.
I think there is no need for a face for shy groups.  It is fontified
now in the same face as other regexp constructs (bold).

> - One face for highlighting the innermost grouping.  Basically,
>   `underline' is not bad here but appears a bit noisy in multiline
>   expressions or things like
>
>   (concat "\\("
>           some-string
>           "\\)")

You could add this face after implementing correct highlighting of
nested groups.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: regexp font-lock highlighting

[martin rudalics]

 > I fixed shy group handling in this regexp by allowing ?: to be only
 > before `(' instead of before any of `(', `|' or `)'.  You may need to
 > adjust your code before submitting your final patch.

If you intend "after" instead of "before" here I have done that already.

 > Also I tried to extend this regexp to highlight other special backslash
 > constructs like `\<', `\_<', `\w', `\sC', but soon discovered that
 > sometimes there are similar constructs in non-regexp strings that become
 > inappropriately highlighted.  Unless it's possible to detect whether
 > a given string is a regexp or not,

... it hardly ever is ...

 >                                    I think the regexp highlighting
 > code should be split into multi-level font-lock variables
 > font-lock-regexp-keywords-1 and font-lock-regexp-keywords-2.
 > The first variable would highlight only `\\(?:', `\\|' and `\\)' constructs,
 > but the second - all other regexp constructs.  Everyone agreed to see
 > regexp constructs sometimes highlighted in wrong places could use
 > the second level variable.

I tried to implement variants of that but it did not turn out well.
Highlighting backslashes only is more distracting than useful.  Having
`<' or `w' stand out with bold face makes groupings much less readable.

 > I see everyone tends to change the default color of `font-lock-string-face'
 > (which is barely visible on light backgrounds) to a green color.
 > Currently on tty strings are fontified in green, so maybe this is one
 > of the reasons why people prefer green strings.  This indicates that
 > we should change the current "RosyBrown" default to one of the shades
 > of green.

... but leave alone font-lock's statement "... character strings might
be brown, because, err, strings are brown ..."

 > I added two new faces: for a backslash and regexp constructs after
 > a backslash.  The former inherits from escape-glyph to be highlighted
 > like a backslash in escape sequences, the latter is bold by default.
 > I think there is no need for a face for shy groups.  It is fontified
 > now in the same face as other regexp constructs (bold).

... splendid

Re: regexp font-lock highlighting

[Juri Linkov]

> I tried to implement variants of that but it did not turn out well.
> Highlighting backslashes only is more distracting than useful.  Having
> `<' or `w' stand out with bold face makes groupings much less readable.

A different face could be used for `<', `w' and other constructs.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: regexp font-lock highlighting

[Stefan Monnier]

>> I tried to implement variants of that but it did not turn out well.
>> Highlighting backslashes only is more distracting than useful.  Having
>> `<' or `w' stand out with bold face makes groupings much less readable.
> A different face could be used for `<', `w' and other constructs.

I can't help wondering what would be the benefit of highlighting things like
\< and \w.  Highlighting \( and \| and \) can make sense since it helps
doing visual paren matching (tho I find blink-matching-open to work fairly
well in most cases, I can see that it sometimes gets confused and also it
currently doesn't understand the concept of the "matching open of an infix
operator").

Please leave \< and \w alone, thank you.


        Stefan

Re: regexp font-lock highlighting

[Juri Linkov]

> I can't help wondering what would be the benefit of highlighting
> things like \< and \w.  Highlighting \( and \| and \) can make sense
> since it helps doing visual paren matching

\< and \w are keywords of the language of regular expressions.
Like keywords of other programming languages highlighted in Emacs
it makes sense to highlight keywords of regular expressions as well.

> Please leave \< and \w alone, thank you.

Highlighting of \< and \w should by no means be the default.
Anyone willing to implement highlighting of all regexp constructs
could add it as an option for those users who like fruit salad
(I'm not speaking for myself, I like moderate highlighting).

-- 
Juri Linkov
http://www.jurta.org/emacs/