bug#37751: 27.0.50; Vue mode indentation stopped working at 0b3982b1a3

bug#37751: 27.0.50; Vue mode indentation stopped working at 0b3982b1a3

[Gary Oberbrunner]

[-- Attachment #1: Type: text/plain, Size: 6558 bytes --]

I'm using vue-mode, based on mmm-mode with Emacs 27 nightly on Windows 10.
As of commit 0b3982b1a38, indentation stopped working in vue-mode buffers.
To repro, create a simple .vue file:
~~~~~~~~~~~~~~~
<template>
</template>
<script lang="ts">
import * as d3 from 'd3'
function foo() {

}
</script>
<style>
</style>
~~~~~~~~~~~~~~~
In emacs -q, load the 'vue-mode' package, go to the blank line in function
foo(), and type TAB. In commit ca7b6a4451, this works (indents the cursor).
In the next commit, 0b3982b1a3, it stopped working.
The commit in question is this:
* 0b3982b1a3 Stefan Monnier, 4 months ago:
| . . . . * lisp/emacs-lisp/syntax.el: Use syntax-ppss-table
for syntax-propertize.

This init file should suffice to repro the issue:
~~~~~~~~~~~~~~~~~~~~~~~~
(require 'package)
(package-initialize)
(require 'use-package)
(setq use-package-verbose t
      use-package-always-ensure t)

;;; Vue mode, based on mmm-mode -- set up for .vue files (html/css/script)
(use-package vue-mode
  :mode "\\.vue$")
~~~~~~~~~~~~~~~~~~~~~~~~~

I traced it as far as the fact that (syntax-ppss (point)) returns 0 on that
blank line in the failing commit, where it should return 1 (and it does in
the prev commit). However (parse-partial-sexp (point-min) (point)) works in
both commits, so it's something about the ppss syntax tables I think.

Note that vue-mode is based on mmm-mode, so it's really three modes in one
file. At the function foo(), it should be in typescript mode.

Also note that a regular typescript-mode file (foo.ts) does not have this
problem, only a multi-mode file like Vue.

I'm happy to help with any further info -- the syntax stuff is pretty
opaque to me without some help, however. :-) I haven't dug into that code
since the '80s and it looks a lot different now.

In GNU Emacs 27.0.50 (build 1, x86_64-w64-mingw32)
 of 2019-10-14 built on TOWER1
Repository revision: ca7b6a4451ede087a1f6796ec5aab6243fcee108
Repository branch: HEAD
Windowing system distributor 'Microsoft Corp.', version 10.0.18362
System Description: Microsoft Windows 10 Pro (v10.0.1903.18362.418)

Recent messages:
Configuring package vue-mode...done

Configured using:
 'configure --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32
 --build=x86_64-w64-mingw32 --with-jpeg --with-xpm --with-png
 --with-tiff --with-rsvg --with-xml2 --with-gnutls --without-imagemagick
 --without-makeinfo --prefix=c:/emacs/emacs-trunk-2019-oldtest'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS NOTIFY W32NOTIFY ACL GNUTLS
LIBXML2 ZLIB TOOLKIT_SCROLL_BARS THREADS JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: en_US.utf-8
  locale-coding-system: cp1252

Major mode: vue

Minor modes in effect:
  lsp-ui-mode: t
  lsp-ui-doc-mode: t
  flycheck-mode: t
  lsp--managed-mode: t
  lsp-mode: t
  company-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired dired-loaddefs rfc822 mml
mml-sec epa epg mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail typescript-mode derived cl
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs css-mode smie eww mm-url gnus nnheader gnus-util rmail
rmail-loaddefs rfc2047 rfc2045 ietf-drums time-date mail-utils mm-util
mail-prsvr url-queue url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf mailcap shr text-property-search svg
xml browse-url format-spec company-oddmuse company-keywords
company-etags etags fileloop generator company-gtags
company-dabbrev-code company-dabbrev company-files company-capf
company-cmake company-xcode company-clang company-semantic company-eclim
company-template company-bbdb lsp-ui lsp-ui-flycheck lsp-ui-doc
goto-addr lsp-ui-imenu lsp-ui-peek lsp-ui-sideline flycheck find-func
view lsp-clients lsp-vhdl lsp-haxe lsp-erlang lsp-fsharp lsp-metals
lsp-elm lsp-dart lsp-clojure lsp-go lsp-xml lsp-css lsp-intelephense
lsp-vetur lsp-html lsp-solargraph lsp-rust lsp-pyls lsp sgml-mode dom
vue-mode ssass-mode vue-html-mode mmm-cmds mmm-mode mmm-univ mmm-class
mmm-region mmm-auto mmm-vars mmm-utils mmm-compat company-lsp lsp-mode
ewoc markdown-mode edit-indirect rx color noutline outline tree-widget
wid-edit xref url-util spinner network-stream puny nsm rmc inline imenu
ht filenotify f s em-glob esh-util dash-functional dash bindat
flymake-proc flymake compile comint ansi-color ring warnings thingatpt
project company edmacro kmacro pcase yasnippet-snippets yasnippet
cl-extra help-mode use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key easy-mmode
use-package-core finder-inf info package easymenu epg-config
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json subr-x map url-vars seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib elec-pair mule-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads w32notify dbusbind w32 lcms2 multi-tty
make-network-process emacs)

Memory information:
((conses 16 232307 16412)
 (symbols 48 23938 6)
 (strings 32 76532 3820)
 (string-bytes 1 2460804)
 (vectors 16 36419)
 (vector-slots 8 543452 22158)
 (floats 8 274 121)
 (intervals 56 614 148)
 (buffers 992 18))

-- 
Gary

[-- Attachment #2: Type: text/html, Size: 7509 bytes --]

bug#37751: 27.0.50; Vue mode indentation stopped working at 0b3982b1a3

[Eli Zaretskii]

> From: Gary Oberbrunner <garyo@oberbrunner.com>
> Date: Mon, 14 Oct 2019 18:08:32 -0400
> 
> I'm using vue-mode, based on mmm-mode with Emacs 27 nightly on Windows 10.
> As of commit 0b3982b1a38, indentation stopped working in vue-mode buffers.

Please report that to the developers of vue-mode first, if you didn't
already.  If they conclude this is a core Emacs problem, and provide
evidence for that, we will look into this.

Thanks.

bug#37751: 27.0.50; Vue mode indentation stopped working at 0b3982b1a3

[Dmitry Gutov]

On 15.10.2019 1:08, Gary Oberbrunner wrote:
> 
> Note that vue-mode is based on mmm-mode, so it's really three modes in 
> one file. At the function foo(), it should be in typescript mode.

Could you please try applying the patch below to your installed mmm-mode 
version and restarting Emacs?

diff --git a/mmm-vars.el b/mmm-vars.el
index b567bb0..0926046 100644
--- a/mmm-vars.el
+++ b/mmm-vars.el
@@ -135,6 +135,7 @@
      font-lock-mark-block-function       ; Override this?
      font-lock-syntactic-keywords
      font-lock-syntactic-face-function
+    syntax-ppss-table
      parse-sexp-ignore-comments  ; Fixes indentation in PHP-mode?
      ;; Can be different in different buffers
      (c-basic-offset

bug#37751: 27.0.50; Vue mode indentation stopped working at 0b3982b1a3

[Gary Oberbrunner]

[-- Attachment #1: Type: text/plain, Size: 1568 bytes --]

I've reported it to vue-mode, but vue-mode is a thin wrapper around
mmm-mode (mode for multiple major modes in the same buffer). It may be hard
for the mmm-mode maintainer to understand my issue since I'm using a
derived mode, not mmm-mode directly.
I was hoping Stefan Monnier might chime in with some ideas -- I think I'll
have to dive into it, and I'm OK with that, but could use a bit of
guidance. I suspect it's related to the fact that mmm-mode is switching
major modes within the same buffer and trying to adjust everything on the
fly, even when the buffer hasn't changed. (In mmm-mode it switches modes
when you enter a certain region of the file.)

For instance is there any way to temporarily completely turn off the
syntax-ppss caching mechanism, so I can really see if that's the root cause?

I can definitely confirm that holding everything else constant except that
single emacs commit of Stefan's, 0b3982b1a38, triggers the failure. But I
understand it's probably some interaction between that and mmm-mode.


On Tue, Oct 15, 2019 at 2:28 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Gary Oberbrunner <garyo@oberbrunner.com>
> > Date: Mon, 14 Oct 2019 18:08:32 -0400
> >
> > I'm using vue-mode, based on mmm-mode with Emacs 27 nightly on Windows
> 10.
> > As of commit 0b3982b1a38, indentation stopped working in vue-mode
> buffers.
>
> Please report that to the developers of vue-mode first, if you didn't
> already.  If they conclude this is a core Emacs problem, and provide
> evidence for that, we will look into this.
>
> Thanks.
>


-- 
Gary

[-- Attachment #2: Type: text/html, Size: 2206 bytes --]

bug#37751: 27.0.50; Vue mode indentation stopped working at 0b3982b1a3

[Eli Zaretskii]

> From: Gary Oberbrunner <garyo@oberbrunner.com>
> Date: Wed, 16 Oct 2019 12:13:15 -0400
> Cc: 37751@debbugs.gnu.org
> 
> I've reported it to vue-mode, but vue-mode is a thin wrapper around mmm-mode (mode for multiple major
> modes in the same buffer). It may be hard for the mmm-mode maintainer to understand my issue since I'm
> using a derived mode, not mmm-mode directly.
> I was hoping Stefan Monnier might chime in with some ideas -- I think I'll have to dive into it, and I'm OK with
> that, but could use a bit of guidance. I suspect it's related to the fact that mmm-mode is switching major
> modes within the same buffer and trying to adjust everything on the fly, even when the buffer hasn't changed.
> (In mmm-mode it switches modes when you enter a certain region of the file.)

Stefan doesn't regularly read this list, but I've CC'ed him now.

> For instance is there any way to temporarily completely turn off the syntax-ppss caching mechanism, so I can
> really see if that's the root cause?
> 
> I can definitely confirm that holding everything else constant except that single emacs commit of Stefan's,
> 0b3982b1a38, triggers the failure. But I understand it's probably some interaction between that and
> mmm-mode.
> 
> On Tue, Oct 15, 2019 at 2:28 AM Eli Zaretskii <eliz@gnu.org> wrote:
> 
>  > From: Gary Oberbrunner <garyo@oberbrunner.com>
>  > Date: Mon, 14 Oct 2019 18:08:32 -0400
>  > 
>  > I'm using vue-mode, based on mmm-mode with Emacs 27 nightly on Windows 10.
>  > As of commit 0b3982b1a38, indentation stopped working in vue-mode buffers.
> 
>  Please report that to the developers of vue-mode first, if you didn't
>  already.  If they conclude this is a core Emacs problem, and provide
>  evidence for that, we will look into this.
> 
>  Thanks.
> 
> -- 
> Gary

bug#37751: 27.0.50; Vue mode indentation stopped working at 0b3982b1a3

[Gary Oberbrunner]

[-- Attachment #1: Type: text/plain, Size: 2030 bytes --]

Dmitry's reply above helped me -- that fixes my issue. Thank you!

On Wed, Oct 16, 2019 at 1:53 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Gary Oberbrunner <garyo@oberbrunner.com>
> > Date: Wed, 16 Oct 2019 12:13:15 -0400
> > Cc: 37751@debbugs.gnu.org
> >
> > I've reported it to vue-mode, but vue-mode is a thin wrapper around
> mmm-mode (mode for multiple major
> > modes in the same buffer). It may be hard for the mmm-mode maintainer to
> understand my issue since I'm
> > using a derived mode, not mmm-mode directly.
> > I was hoping Stefan Monnier might chime in with some ideas -- I think
> I'll have to dive into it, and I'm OK with
> > that, but could use a bit of guidance. I suspect it's related to the
> fact that mmm-mode is switching major
> > modes within the same buffer and trying to adjust everything on the fly,
> even when the buffer hasn't changed.
> > (In mmm-mode it switches modes when you enter a certain region of the
> file.)
>
> Stefan doesn't regularly read this list, but I've CC'ed him now.
>
> > For instance is there any way to temporarily completely turn off the
> syntax-ppss caching mechanism, so I can
> > really see if that's the root cause?
> >
> > I can definitely confirm that holding everything else constant except
> that single emacs commit of Stefan's,
> > 0b3982b1a38, triggers the failure. But I understand it's probably some
> interaction between that and
> > mmm-mode.
> >
> > On Tue, Oct 15, 2019 at 2:28 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> >  > From: Gary Oberbrunner <garyo@oberbrunner.com>
> >  > Date: Mon, 14 Oct 2019 18:08:32 -0400
> >  >
> >  > I'm using vue-mode, based on mmm-mode with Emacs 27 nightly on
> Windows 10.
> >  > As of commit 0b3982b1a38, indentation stopped working in vue-mode
> buffers.
> >
> >  Please report that to the developers of vue-mode first, if you didn't
> >  already.  If they conclude this is a core Emacs problem, and provide
> >  evidence for that, we will look into this.
> >
> >  Thanks.
> >
> > --
> > Gary
>


-- 
Gary

[-- Attachment #2: Type: text/html, Size: 2940 bytes --]

bug#37751: 27.0.50; Vue mode indentation stopped working at 0b3982b1a3

[Eli Zaretskii]

> From: Gary Oberbrunner <garyo@oberbrunner.com>
> Date: Wed, 16 Oct 2019 16:13:56 -0400
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, 37751@debbugs.gnu.org
> 
> Dmitry's reply above helped me -- that fixes my issue. Thank you!

Thanks, I'm therefore closing this bug report.