bug#31972: 27.0.50; syntax-ppss depth incorrect after font-lock syntax-alist

bug#31972: 27.0.50; syntax-ppss depth incorrect after font-lock syntax-alist

[Brady Trainor]

After `emacs -Q`, I open a file with

--8<---------------cut here---------------start------------->8---
(define-derived-mode dart-mode prog-mode "Dart"
  ""
  (setq comment-start "//")
  (setq comment-end "")
  (setq font-lock-defaults '(nil nil nil ((?/ . "_ 12")
                                          (?\n . ">"))))
  ;; (modify-syntax-entry ?/ "_ 12")
  ;; (modify-syntax-entry ?\n ">")
  )

(provide 'dart-mode)
--8<---------------cut here---------------end--------------->8---

I open a file and `M-x dart-mode RET`, with contents say

--8<---------------cut here---------------start------------->8---
// A comment (with a parenthesized
// note).
--8<---------------cut here---------------end--------------->8---

Then with point on beginning of second line, I try `M-: (car
(syntax-ppss)) RET` and find that depth is reported as 1, rather than 0.
If I comment out the `font-lock-defaults` expression and instead use
`modify-syntax-entry` expressions, I get depth 0 throughout.

I did not look much further for some clarification in `font-lock`, as it
did not seem right to me. You may ask why do I need `syntax-ppss` to
report no depth change in comments? I currently intend to use
`syntax-ppss` for an `indent-line-function`.

Thank you for considering this!

--
Brady



In GNU Emacs 27.0.50 (build 1, x86_64-apple-darwin17.4.0, NS appkit-1561.20 Version 10.13.3 (Build 17D102))
 of 2018-04-11 built on iams-MacBook-Air.local
Repository revision: 57442b6812e9ec565efc39f722e84079dd71d8c0
System Description:  Mac OS X 10.13.3

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list... [3 times]
You can run the command ‘eval-buffer’ with M-x ev-b RET
You can run the command ‘dart-mode’ with M-x dar RET
(1 145 188 nil nil nil 0 nil nil (145) nil)
Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs/HEAD-57442b6/share/info/emacs
 --prefix=/usr/local/Cellar/emacs/HEAD-57442b6 --without-x --with-xml2
 --with-dbus --with-gnutls --with-imagemagick --with-modules --with-rsvg
 --without-pop --with-ns --disable-ns-self-contained'

Configured features:
RSVG IMAGEMAGICK DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS
NS MODULES THREADS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Dart

Minor modes in effect:
  diff-auto-refine-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv dired dired-loaddefs format-spec rfc822 mml
mml-sec password-cache epa epg epg-config gnus-util rmail rmail-loaddefs
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils dart-mode derived vc-git diff-mode
easymenu easy-mmode term/screen term/xterm xterm time-date elec-pair
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind kqueue cocoa
ns multi-tty make-network-process emacs)

Memory information:
((conses 16 212690 10486)
 (symbols 48 20527 1)
 (miscs 40 37 176)
 (strings 32 30827 1661)
 (string-bytes 1 811691)
 (vectors 16 33744)
 (vector-slots 8 683017 8048)
 (floats 8 53 623)
 (intervals 56 318 0)
 (buffers 992 14))

bug#31972: 27.0.50; syntax-ppss depth incorrect after font-lock syntax-alist

[Noam Postavsky]

tags 31972 + notabug
quit

Brady Trainor <mail@bradyt.com> writes:

> (define-derived-mode dart-mode prog-mode "Dart"
>   ""
>   (setq comment-start "//")
>   (setq comment-end "")
>   (setq font-lock-defaults '(nil nil nil ((?/ . "_ 12")
>                                           (?\n . ">"))))
>   ;; (modify-syntax-entry ?/ "_ 12")
>   ;; (modify-syntax-entry ?\n ">")
>   )

> I open a file and `M-x dart-mode RET`, with contents say
>
> // A comment (with a parenthesized
> // note).
>
> Then with point on beginning of second line, I try `M-: (car
> (syntax-ppss)) RET` and find that depth is reported as 1, rather than 0.
> If I comment out the `font-lock-defaults` expression and instead use
> `modify-syntax-entry` expressions, I get depth 0 throughout.

That looks all correct to me.  In the first case, the "//" doesn't start
a comment, it's only fontified as if it starts a comment.  So of course
when you're inside a paren you'll see depth 1.  In the second case, you
set "//" as a comment start, so you see depth 0.

I think you want

(define-derived-mode dart-mode prog-mode "Dart"
  ""
  (setq comment-start "//")
  (setq comment-end "")
  (setq font-lock-defaults '(nil nil nil nil)) ; Same as '(nil).
  (modify-syntax-entry ?/ "_ 12")
  (modify-syntax-entry ?\n ">"))