Re: Giving "text quotes" syntax in font-lock-syntax-table only

[Ikumi Keita <ikumi@ikumi.que.jp>]

Hi emacs developers,

>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> The best way to do that is to set `syntax-ppss-table` to the
>> syntax-table that should be used.

Thanks for your advice, but unfortunately I couldn't find a good
solution yet.

1. As I wrote previously, another problem occurs if we set constant
`syntax-ppss-table' in the major mode.

2. I actually tried the following implementation:
>>>>> Ikumi Keita <ikumi@ikumi.que.jp> writes:
> If we temporarily bind `syntax-ppss-table' during font lock by `let',
> the problem of conflicting cache emerges again and it causes the
> inverted fontification, if I understand correctly.
----------------------------------------------------------------------
(defvar bug-repro-mode-syntax-table
  (let ((st (make-syntax-table)))
    (modify-syntax-entry ?$ "$" st)
    (modify-syntax-entry ?% "<" st)
    (modify-syntax-entry ?\n ">" st)
    st))

(defvar bug-repro-syntax-ppss-table
  (let ((st (make-syntax-table bug-repro-mode-syntax-table)))
    (modify-syntax-entry ?$ "\"" st)
    st))

(defun bug-repro-syntactic-face-function (state)
  (let ((char (nth 3 state)))
    (if char
        font-lock-string-face
      font-lock-comment-face)))

(define-derived-mode bug-repro-mode nil "BugRepro"
  :syntax-table bug-repro-mode-syntax-table
  (add-hook 'pre-command-hook #'syntax-ppss nil t)
  (setq font-lock-multiline t)
  (setq font-lock-defaults
        '(nil nil nil nil ; ((?$ . "\""))
              (font-lock-syntactic-face-function
               . bug-repro-syntactic-face-function)
	      (font-lock-fontify-region-function
	       . bug-repro-fontify-region))))

(defun bug-repro-fontify-region (beg end loudly)
  (let ((syntax-ppss-table bug-repro-syntax-ppss-table))
    (font-lock-default-fontify-region beg end loudly)))
----------------------------------------------------------------------

But the result was just what I anticipated.

So may I ask the following questions again?
(1) It doesn't work reliably to give "string quotes" syntax to $ in
    SYNTAX-ALIST. Is this an emacs bug or an intented restriction?
(2) If not a bug, is it reasonable to ask to extend the font lock
    framework to allow syntactic fontification of $...$ form? E.g.
    [A] Implement some kind of "separate cache" in syntax-ppss and
        make it available from font lock.
    [B] Extend syntax parse state to include information about "the
        position is in-math state or not" and make
	`font-lock-fontify-syntactically-region' responsive to the
        in-math state in addition to in-comment and in-string states.
        This way, it would no longer be necessary to redefine syntax of
        $ in SYNTAX-ALIST and we can naturally implement syntactic
        fontification of $...$ via `font-lock-syntactic-face-function'.
        This approach would be beneficial for standard tex-mode.el, and
        potentially other program modes as well, if realized.
(3) Or is there some smart way to achieve syntactic fontification of
    $...$ in the current font lock scheme?

Best regards,
Ikumi Keita