bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Jostein Kjønigsen]

When I:
=======

1. Create a new TS-file, activate typescript-ts-mode
2. Write any text or code with HTML-like qualities.
3. Write more text or code

I experience that:
==================

The fontification after this point is permanently incorrect.

Instead I expected that:
========================

Fontification to remain correct.

Examples:
=========

To trigger this bug you only need to include HTML-like constructs in
a string:

     const example = "<123>";

Or just use a TypeScript hard casts:

     const result = <IService>service;

After this point the remaining parse-tree becomes invalid, because the
remaining parse-tree is not reported as a jsx-expreesion.



--



In GNU Emacs 29.0.60 (build 1, x86_64-pc-linux-gnu, GTK+ Version
  3.24.34, cairo version 1.16.0) of 2022-11-29 built on ThinkPad-T14s
Repository revision: c43cdfd639518c5f2e7e7cbbaf1eca40ac957eb0
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12201003
System Description: Ubuntu 22.10

Configured using:
  'configure --with-tree-sitter'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
   value of $LC_MONETARY: nb_NO.UTF-8
   value of $LC_NUMERIC: nb_NO.UTF-8
   value of $LC_TIME: nb_NO.UTF-8
   value of $LANG: en_US.UTF-8
   locale-coding-system: utf-8-unix

Major mode: TypeScript

Minor modes in effect:
   global-git-commit-mode: t
   magit-auto-revert-mode: t
   lsp-diagnostics-mode: t
   lsp-headerline-breadcrumb-mode: t
   lsp-modeline-workspace-status-mode: t
   lsp-modeline-diagnostics-mode: t
   lsp-modeline-code-actions-mode: t
   electric-pair-mode: t
   treesit-explore-mode: t
   lsp-completion-mode: t
   editorconfig-mode: t
   flycheck-mode: t
   which-function-mode: t
   nlinum-mode: t
   company-mode: t
   global-ede-mode: t
   dap-tooltip-mode: t
   dap-ui-many-windows-mode: t
   dap-ui-controls-mode: t
   dap-ui-mode: t
   treemacs-filewatch-mode: t
   treemacs-follow-mode: t
   treemacs-git-mode: t
   treemacs-fringe-indicator-mode: t
   dap-auto-configure-mode: t
   dap-mode: t
   global-undo-tree-mode: t
   undo-tree-mode: t
   doom-modeline-mode: t
   projectile-mode: t
   ido-yes-or-no-mode: t
   helm-mode: t
   helm-minibuffer-history-mode: t
   helm--remap-mouse-mode: t
   async-bytecomp-package-mode: t
   delete-selection-mode: t
   global-auto-revert-mode: t
   server-mode: t
   shell-dirtrack-mode: t
   global-hl-line-mode: t
   lsp-managed-mode: t
   lsp-mode: t
   yas-global-mode: t
   yas-minor-mode: t
   tooltip-mode: t
   global-eldoc-mode: t
   eldoc-mode: t
   show-paren-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   column-number-mode: t
   line-number-mode: t
   transient-mark-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t

Load-path shadows:
/home/jostein/.emacs.d/elpa/transient-20221127.2242/transient hides 
/home/jostein/build/emacs/lisp/transient
/home/jostein/.emacs.d/elpa/eglot-20221020.1010/eglot hides 
/home/jostein/build/emacs/lisp/progmodes/eglot

Features:
(shadow sort emacsbug typescript-ts-mode js git-rebase vc-hg vc-bzr
vc-src vc-sccs vc-svn vc-cvs vc-rcs log-view vc bug-reference
magit-extras magit-bookmark magit-submodule magit-obsolete magit-blame
magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch
magit-clone magit-remote magit-commit magit-sequence magit-notes
magit-worktree magit-tag magit-merge magit-branch magit-reset
magit-files magit-refs magit-status magit magit-repos magit-apply
magit-wip magit-log magit-diff smerge-mode git-commit log-edit message
sendmail yank-media rfc822 mml mml-sec epa derived mm-decode mm-bodies
mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev
gmm-utils mailheader pcvs-util magit-core magit-autorevert magit-margin
magit-transient magit-process with-editor magit-mode transient magit-git
magit-base magit-section crm compat-27 compat-26 dired-aux ede/dired
flyspell ispell mule-util executable misearch multi-isearch
helm-bookmark helm-net helm-adaptive treemacs-bookmarks treemacs-tags
bookmark lsp-diagnostics lsp-headerline lsp-icons lsp-modeline
face-remap mail-extr helm-command helm-elisp helm-eval helm-info view
vc-git diff-mode vc-dispatcher disp-table elec-pair csharp-mode treesit
cc-langs cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align
cc-engine cc-vars cc-defs winner ffap tramp-archive tramp-gvfs
tramp-cache warnings time-stamp zeroconf dbus add-log lsp-zig lsp-steep
lsp-svelte lsp-sqls lsp-ruby-syntax-tree lsp-yaml lsp-xml lsp-vimscript
lsp-vhdl lsp-volar lsp-vetur lsp-html lsp-verilog lsp-vala lsp-v
lsp-typeprof lsp-ttcn3 lsp-toml lsp-terraform lsp-tex lsp-sorbet
lsp-solargraph lsp-rust lsp-rf lsp-remark lsp-racket lsp-r
lsp-purescript lsp-pylsp lsp-pyls lsp-php lsp-pls lsp-perlnavigator
lsp-perl lsp-openscad lsp-ocaml lsp-magik lsp-nix lsp-nim lsp-nginx
lsp-mint lsp-marksman lsp-markdown lsp-lua lsp-kotlin lsp-json
lsp-javascript lsp-idris lsp-haxe lsp-groovy lsp-hack lsp-graphql
lsp-gleam lsp-go lsp-completion lsp-gdscript lsp-fsharp lsp-fortran
lsp-eslint lsp-erlang lsp-emmet lsp-elixir lsp-elm lsp-dockerfile
lsp-dhall lsp-d lsp-css lsp-csharp gnutls lsp-crystal lsp-cmake
lsp-clojure lsp-semantic-tokens lsp-clangd lsp-beancount lsp-bash
lsp-astro lsp-ansible lsp-angular lsp-ada lsp-actionscript
ido-completing-read+ memoize minibuf-eldef elisp-slime-nav paredit
editorconfig editorconfig-core editorconfig-core-handle
editorconfig-fnmatch flycheck highlight-symbol which-func edebug debug
backtrace nlinum linum company-oddmuse company-keywords company-etags
etags fileloop generator company-gtags company-dabbrev-code
company-dabbrev company-files company-clang company-capf company-cmake
company-semantic company-template company-bbdb company-web-html
company-web company-css web-completion-data company eww url-queue shr
pixel-fill kinsoku url-file svg mm-url gnus nnheader gnus-util
mail-utils range mm-util mail-prsvr ede/speedbar ede/files ede
ede/detect ede/base ede/auto ede/source eieio-base eieio-speedbar
speedbar ezimage dframe eieio-custom cedet dap-mouse dap-ui lsp-treemacs
lsp-treemacs-generic lsp-treemacs-themes treemacs-treelib treemacs
treemacs-header-line treemacs-compatibility treemacs-mode
treemacs-interface treemacs-persistence treemacs-filewatch-mode
treemacs-follow-mode treemacs-rendering treemacs-annotations
treemacs-async treemacs-workspaces treemacs-dom treemacs-visuals
treemacs-fringe-indicator treemacs-scope pulse treemacs-faces
treemacs-icons treemacs-themes treemacs-core-utils pfuture
treemacs-logging treemacs-customization treemacs-macros gdb-mi bindat
gud bui bui-list bui-info bui-entry bui-core bui-history bui-button
bui-utils lsp-lens dap-gdb-lldb dap-netcore dap-node dap-utils dom xml
dap-pwsh lsp-pwsh dap-python dap-mode dap-tasks dap-launch lsp-docker
yaml posframe dap-overlays undo-tree diff queue doom-modeline
doom-modeline-segments doom-modeline-env doom-modeline-core
all-the-icons all-the-icons-faces data-material data-weathericons
data-octicons data-fileicons data-faicons data-alltheicons shrink-path
compat compat-macs projectile lisp-mnt grep ibuf-ext ibuffer
ibuffer-loaddefs helm-imenu ob-plantuml org ob ob-tangle ob-ref ob-lob
ob-table ob-exp org-macro org-footnote org-src ob-comint org-pcomplete
org-list org-faces org-entities org-version ob-emacs-lisp ob-core
ob-eval org-table oc-basic bibtex ol org-keys oc org-compat org-macs
org-loaddefs find-func cal-menu calendar cal-loaddefs ido-yes-or-no ido
helm-mode helm-misc helm-files image-dired image-dired-tags
image-dired-external image-dired-util xdg image-mode dired
dired-loaddefs exif tramp tramp-loaddefs trampver tramp-integration
cus-edit pp cus-load files-x tramp-compat parse-time iso8601 time-date
ls-lisp helm-buffers helm-occur helm-tags helm-locate helm-grep
helm-regexp format-spec helm-utils helm-help helm-types helm
helm-global-bindings helm-easymenu edmacro kmacro helm-core easy-mmode
async-bytecomp helm-source helm-multi-match helm-lib async helm-config
delsel cl-extra autorevert server powershell advice shell pcomplete
hl-line lsp-mode lsp-protocol yasnippet help-mode xref project
tree-widget wid-edit spinner pcase network-stream puny nsm markdown-mode
color thingatpt noutline outline icons lv inline imenu ht filenotify f
f-shortdoc shortdoc s ewoc epg rfc6068 epg-config dash dracula-theme
compile-eslint compile text-property-search comint ansi-osc ansi-color
ring cl finder-inf git-timemachine-autoloads rx
helm-projectile-autoloads expand-region-autoloads
all-the-icons-autoloads dracula-theme-autoloads eglot-autoloads
multiple-cursors-autoloads tree-sitter-langs-autoloads
projectile-autoloads nlinum-autoloads doom-modeline-autoloads
rust-mode-autoloads editorconfig-autoloads helm-autoloads
helm-core-autoloads async-autoloads assess-autoloads m-buffer-autoloads
cargo-autoloads package-lint-autoloads flycheck-autoloads
company-autoloads magit-autoloads magit-section-autoloads
web-mode-autoloads paredit-autoloads helpful-autoloads
elisp-refs-autoloads js2-mode-autoloads yaml-mode-autoloads
powershell-autoloads dap-mode-autoloads lsp-docker-autoloads
yaml-autoloads lsp-treemacs-autoloads treemacs-autoloads
posframe-autoloads hydra-autoloads pfuture-autoloads
ace-window-autoloads avy-autoloads lsp-mode-autoloads
markdown-mode-autoloads ht-autoloads git-commit-autoloads
with-editor-autoloads transient-autoloads compat-autoloads
pcache-autoloads f-autoloads popup-autoloads s-autoloads info
dash-autoloads macrostep-autoloads package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process emacs)

Memory information:
((conses 16 707483 89040)
  (symbols 48 57220 127)
  (strings 32 225052 10207)
  (string-bytes 1 6824489)
  (vectors 16 145796)
  (vector-slots 8 2442138 178412)
  (floats 8 1069 547)
  (intervals 56 14048 1723)
  (buffers 992 52))

-- 
Vennlig hilsen
*Jostein Kjønigsen*

jostein@kjonigsen.net 🍵 jostein@gmail.com
https://jostein.kjønigsen.no <https://jostein.kjønigsen.no>

bug#59691: Html-like constructs fail in typescript

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi Jostein!

I already have a fix for this bug lined up, and will send the fix as
soon as I get my other stuff applied.  It relies on another patch that
is waiting.

The solution is to create two modes: typescript-ts-mode and
tsx-ts-mode. They have separate parsers, and should be treated as
separate languages.  That means that in a .ts file we enable the
'typescript' language, and in .tsx we enable the 'tsx' language.  This
exact issue is why they have two languages.

In other words, we cannot support this in typescript-ts-mode.

I hope to get this in pretty soon :)

Theo

bug#59691: Html-like constructs fail in typescript

[Jostein Kjønigsen]

Nice!

Should we until further notice assume that js-ts-mode suffers from the same issues, and that a jsx-ts-mode might be needed too? To me it at least sounds plausible. 

Speaking of duplication… Now that we have tree-sitter wouldn’t it be possible to use the same parser and/or the same major-modes for JS/JSX as we already do for TS/TSX? JS/JSX is just Typescript without the type-annotations, right?

Or are there good reasons for having separate modes for these?

—
Jostein Kjønigsen
https://jostein.kjønigsen.net

> On 29 Nov 2022, at 22:04, Theodor Thornhill <theo@thornhill.no> wrote:
> 
> 
> Hi Jostein!
> 
> I already have a fix for this bug lined up, and will send the fix as
> soon as I get my other stuff applied.  It relies on another patch that
> is waiting.
> 
> The solution is to create two modes: typescript-ts-mode and
> tsx-ts-mode. They have separate parsers, and should be treated as
> separate languages.  That means that in a .ts file we enable the
> 'typescript' language, and in .tsx we enable the 'tsx' language.  This
> exact issue is why they have two languages.
> 
> In other words, we cannot support this in typescript-ts-mode.
> 
> I hope to get this in pretty soon :)
> 
> Theo

bug#59691: Html-like constructs fail in typescript

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 29 November 2022 22:37:25 CET, "Jostein Kjønigsen" <jostein@secure.kjonigsen.net> wrote:
>Nice!
>
>Should we until further notice assume that js-ts-mode suffers from the same issues, and that a jsx-ts-mode might be needed too? To me it at least sounds plausible. 

No, because there are no ambiguities in the grammar with types and jsx.

>
>Speaking of duplication… Now that we have tree-sitter wouldn’t it be possible to use the same parser and/or the same major-modes for JS/JSX as we already do for TS/TSX? JS/JSX is just Typescript without the type-annotations, right?
>
>Or are there good reasons for having separate modes for these?
>

Not sure. I like that a mode maps to a tree-sitter grammar, and personally I don't want to intertwine things to early.

>

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Yuan Fu]

Theodor Thornhill <theo@thornhill.no> writes:

> On 29 November 2022 22:37:25 CET, "Jostein Kjønigsen" <jostein@secure.kjonigsen.net> wrote:
>>Nice!
>>
>>Should we until further notice assume that js-ts-mode suffers from
>> the same issues, and that a jsx-ts-mode might be needed too? To me
>> it at least sounds plausible.
>
> No, because there are no ambiguities in the grammar with types and jsx.
>
>>
>>Speaking of duplication… Now that we have tree-sitter wouldn’t it be
>> possible to use the same parser and/or the same major-modes for
>> JS/JSX as we already do for TS/TSX? JS/JSX is just Typescript
>> without the type-annotations, right?
>>
>>Or are there good reasons for having separate modes for these?
>>
>
> Not sure. I like that a mode maps to a tree-sitter grammar, and personally I don't want to intertwine things to early.

No strong opinions here, but currently a user could install
tree-sitter-js, and find and enable js-ts-mode, which is
straightforward, which is good. Since these four modes doesn’t require
too much boilerplate, I think it’s pretty good right now.

Yuan

bug#59691: Html-like constructs fail in typescript

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: jostein@kjonigsen.net, casouri@gmail.com, eliz@gnu.org
> Date: Tue, 29 Nov 2022 22:01:01 +0100
> 
> The solution is to create two modes: typescript-ts-mode and
> tsx-ts-mode. They have separate parsers, and should be treated as
> separate languages.  That means that in a .ts file we enable the
> 'typescript' language, and in .tsx we enable the 'tsx' language.  This
> exact issue is why they have two languages.

This means the script posted by Yuan, which only creates a single grammar
library for Typescript (the one for tsx), should be changed to instead
produce two libraries, right?

bug#59691: Html-like constructs fail in typescript

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 2057 bytes --]

On 29.11.2022 23:20, Jostein Kjønigsen wrote:
>
>> On 29 Nov 2022, at 22:48, Theodor Thornhill<theo@thornhill.no>  wrote:
>>
>> 
>>
>>> On 29 November 2022 22:37:25 CET, "Jostein Kjønigsen"<jostein@secure.kjonigsen.net>  wrote:
>>> Nice!
>>>
>>> Should we until further notice assume that js-ts-mode suffers from the same issues, and that a jsx-ts-mode might be needed too? To me it at least sounds plausible.
>> No, because there are no ambiguities in the grammar with types and jsx.
>>
> Note this behaviour was triggered even when a HTML-tag was contained inside a plain string.
>
> Even without hard typescript casts, there are places where I suspect the same issues can bleed into js-ts-mode.
>
> I’ll try to do more testing tomorrow.

First of all - good news!

Contrary to my expectations, I've tested and I cannot reproduce this 
issue in js-ts-mode.

Even more good news:

Looking deeper into this using treesit-explorer-mode (an extremely 
helpful tool, Yuan!), I found I may have misinterpreted the state of the 
parse-tree in previous report.

Based on that, I would like to revise this bug report:

  * HTML-like constructs inside strings are --/not/-- treated at
    jsx_opening_elements,
  * only angle-bracket "hard" casts (which isn't present in Javascript)
    is causing issues for fontification.

Also, reading up, from what I can tell "hard casts" using angle-brackets 
are no longer encouraged as the default way to cast:

     const service = <IService>object;

This is because the above code will cause a compiler error if used in 
TSX-files (as opposed to TS-files). Instead "as" expressions are 
preferred, because they work equally well for both TS & TSX-files:

     const service = object as IService;

That means that writing idiomatic TypeScript with typescrip-ts-mode 
should produce the expected behaviour, while one may encounter issues 
with older code.

I'm not sure introducing a new major-mode for this 1 aspect of 
TypeScript development is worth it?

Does anyone else have an opinion on this?

--
Jostein

[-- Attachment #2: Type: text/html, Size: 3111 bytes --]

bug#59691: Html-like constructs fail in typescript

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 30 November 2022 13:35:49 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: jostein@kjonigsen.net, casouri@gmail.com, eliz@gnu.org
>> Date: Tue, 29 Nov 2022 22:01:01 +0100
>> 
>> The solution is to create two modes: typescript-ts-mode and
>> tsx-ts-mode. They have separate parsers, and should be treated as
>> separate languages.  That means that in a .ts file we enable the
>> 'typescript' language, and in .tsx we enable the 'tsx' language.  This
>> exact issue is why they have two languages.
>
>This means the script posted by Yuan, which only creates a single grammar
>library for Typescript (the one for tsx), should be changed to instead
>produce two libraries, right?

Yes, I can fix that when the two modes are in :)

bug#59691: Html-like constructs fail in typescript

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 30 November 2022 13:59:20 CET, "Jostein Kjønigsen" <jostein@secure.kjonigsen.net> wrote:
>On 29.11.2022 23:20, Jostein Kjønigsen wrote:
>> 
>>> On 29 Nov 2022, at 22:48, Theodor Thornhill<theo@thornhill.no>  wrote:
>>> 
>>> 
>>> 
>>>> On 29 November 2022 22:37:25 CET, "Jostein Kjønigsen"<jostein@secure.kjonigsen.net>  wrote:
>>>> Nice!
>>>> 
>>>> Should we until further notice assume that js-ts-mode suffers from the same issues, and that a jsx-ts-mode might be needed too? To me it at least sounds plausible.
>>> No, because there are no ambiguities in the grammar with types and jsx.
>>> 
>> Note this behaviour was triggered even when a HTML-tag was contained inside a plain string.
>> 
>> Even without hard typescript casts, there are places where I suspect the same issues can bleed into js-ts-mode.
>> 
>> I’ll try to do more testing tomorrow.
>
>First of all - good news!
>
>Contrary to my expectations, I've tested and I cannot reproduce this issue in js-ts-mode.
>

Yeah, that's what I've seen as well.

>Even more good news:
>
>Looking deeper into this using treesit-explorer-mode (an extremely helpful tool, Yuan!), I found I may have misinterpreted the state of the parse-tree in previous report.
>
>Based on that, I would like to revise this bug report:
>
> * HTML-like constructs inside strings are --/not/-- treated at
>   jsx_opening_elements,

That's correct!

> * only angle-bracket "hard" casts (which isn't present in Javascript)
>   is causing issues for fontification.
>
>Also, reading up, from what I can tell "hard casts" using angle-brackets are no longer encouraged as the default way to cast:
>
>    const service = <IService>object;
>
>This is because the above code will cause a compiler error if used in TSX-files (as opposed to TS-files). Instead "as" expressions are preferred, because they work equally well for both TS & TSX-files:
>
>    const service = object as IService;
>
>That means that writing idiomatic TypeScript with typescrip-ts-mode should produce the expected behaviour, while one may encounter issues with older code.
>
>I'm not sure introducing a new major-mode for this 1 aspect of TypeScript development is worth it?
>
>Does anyone else have an opinion on this?
>
>--
>Jostein

I think that because this is an actual feature it makes sense to have two modes. But defaulting to tsx as the language of choice until now is ok too.

Theo

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 30 Nov 2022 02:22:06 -0800
> Cc: Jostein Kjønigsen <jostein@secure.kjonigsen.net>,
>  59691@debbugs.gnu.org,
>  jostein@kjonigsen.net,
>  eliz@gnu.org
> 
> No strong opinions here, but currently a user could install
> tree-sitter-js, and find and enable js-ts-mode, which is
> straightforward, which is good. Since these four modes doesn’t require
> too much boilerplate, I think it’s pretty good right now.

Can we do that automatically?  For example, can typescript-ts-mode call
js-ts-mode when it detects that it is necessary?

bug#59691: Html-like constructs fail in typescript

[Eli Zaretskii]

> Date: Wed, 30 Nov 2022 14:00:49 +0100
> From: Theodor Thornhill <theo@thornhill.no>
> CC: 59691@debbugs.gnu.org, jostein@kjonigsen.net, casouri@gmail.com
> 
> 
> 
> On 30 November 2022 13:35:49 CET, Eli Zaretskii <eliz@gnu.org> wrote:
> >> From: Theodor Thornhill <theo@thornhill.no>
> >> Cc: jostein@kjonigsen.net, casouri@gmail.com, eliz@gnu.org
> >> Date: Tue, 29 Nov 2022 22:01:01 +0100
> >> 
> >> The solution is to create two modes: typescript-ts-mode and
> >> tsx-ts-mode. They have separate parsers, and should be treated as
> >> separate languages.  That means that in a .ts file we enable the
> >> 'typescript' language, and in .tsx we enable the 'tsx' language.  This
> >> exact issue is why they have two languages.
> >
> >This means the script posted by Yuan, which only creates a single grammar
> >library for Typescript (the one for tsx), should be changed to instead
> >produce two libraries, right?
> 
> Yes, I can fix that when the two modes are in :)

Thanks.  But now I hear that the original analysis could be incomplete, and
there actually is no problem, according to Jostein's last messages?

bug#59691: Html-like constructs fail in typescript

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Wed, 30 Nov 2022 14:00:49 +0100
>> From: Theodor Thornhill <theo@thornhill.no>
>> CC: 59691@debbugs.gnu.org, jostein@kjonigsen.net, casouri@gmail.com
>> 
>> 
>> 
>> On 30 November 2022 13:35:49 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> From: Theodor Thornhill <theo@thornhill.no>
>> >> Cc: jostein@kjonigsen.net, casouri@gmail.com, eliz@gnu.org
>> >> Date: Tue, 29 Nov 2022 22:01:01 +0100
>> >> 
>> >> The solution is to create two modes: typescript-ts-mode and
>> >> tsx-ts-mode. They have separate parsers, and should be treated as
>> >> separate languages.  That means that in a .ts file we enable the
>> >> 'typescript' language, and in .tsx we enable the 'tsx' language.  This
>> >> exact issue is why they have two languages.
>> >
>> >This means the script posted by Yuan, which only creates a single grammar
>> >library for Typescript (the one for tsx), should be changed to instead
>> >produce two libraries, right?
>> 
>> Yes, I can fix that when the two modes are in :)
>
> Thanks.  But now I hear that the original analysis could be incomplete, and
> there actually is no problem, according to Jostein's last messages?

I don't think I agree with him. There are other constructs such as
generics that also can cause these ambiguities, IIRC

Theo

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 30 November 2022 15:09:44 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 30 Nov 2022 02:22:06 -0800
>> Cc: Jostein Kjønigsen <jostein@secure.kjonigsen.net>,
>>  59691@debbugs.gnu.org,
>>  jostein@kjonigsen.net,
>>  eliz@gnu.org
>> 
>> No strong opinions here, but currently a user could install
>> tree-sitter-js, and find and enable js-ts-mode, which is
>> straightforward, which is good. Since these four modes doesn’t require
>> too much boilerplate, I think it’s pretty good right now.
>
>Can we do that automatically?  For example, can typescript-ts-mode call
>js-ts-mode when it detects that it is necessary?

That's not what we want. 

There are three languages here from treesitters pov.

Typescript
Tsx
JavaScript

They are all different, and should be treated as such, imo :)

Theo

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Eli Zaretskii]

> Date: Wed, 30 Nov 2022 16:21:37 +0100
> From: Theodor Thornhill <theo@thornhill.no>
> CC: jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org, jostein@kjonigsen.net
> 
> >> No strong opinions here, but currently a user could install
> >> tree-sitter-js, and find and enable js-ts-mode, which is
> >> straightforward, which is good. Since these four modes doesn’t require
> >> too much boilerplate, I think it’s pretty good right now.
> >
> >Can we do that automatically?  For example, can typescript-ts-mode call
> >js-ts-mode when it detects that it is necessary?
> 
> That's not what we want. 
> 
> There are three languages here from treesitters pov.
> 
> Typescript
> Tsx
> JavaScript
> 
> They are all different, and should be treated as such, imo :)

Our automatic turning-on of major-modes looks at the file-name extension and
little else (magic-mode-alist is not useful here).  So if a file whose
extension is XYZ can have more than one applicable major-mode, we should try
to do something to turn on the correct mode automatically.  What can we do?

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 30 November 2022 17:05:59 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Wed, 30 Nov 2022 16:21:37 +0100
>> From: Theodor Thornhill <theo@thornhill.no>
>> CC: jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org, jostein@kjonigsen.net
>> 
>> >> No strong opinions here, but currently a user could install
>> >> tree-sitter-js, and find and enable js-ts-mode, which is
>> >> straightforward, which is good. Since these four modes doesn’t require
>> >> too much boilerplate, I think it’s pretty good right now.
>> >
>> >Can we do that automatically?  For example, can typescript-ts-mode call
>> >js-ts-mode when it detects that it is necessary?
>> 
>> That's not what we want. 
>> 
>> There are three languages here from treesitters pov.
>> 
>> Typescript
>> Tsx
>> JavaScript
>> 
>> They are all different, and should be treated as such, imo :)
>
>Our automatic turning-on of major-modes looks at the file-name extension and
>little else (magic-mode-alist is not useful here).  So if a file whose
>extension is XYZ can have more than one applicable major-mode, we should try
>to do something to turn on the correct mode automatically.  What can we do?

We are lucky because the extensions in question are ts, tsx and js. So we don't need any smartness :)

Theo

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Eli Zaretskii]

> Date: Wed, 30 Nov 2022 19:10:04 +0100
> From: Theodor Thornhill <theo@thornhill.no>
> CC: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>  jostein@kjonigsen.net
> 
> >Our automatic turning-on of major-modes looks at the file-name extension and
> >little else (magic-mode-alist is not useful here).  So if a file whose
> >extension is XYZ can have more than one applicable major-mode, we should try
> >to do something to turn on the correct mode automatically.  What can we do?
> 
> We are lucky because the extensions in question are ts, tsx and js. So we don't need any smartness :)

And the extensions always say unequivocally what mode to use?  Then yes, we
are lucky, and need just one more major mode (and one more grammar library).

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 30 November 2022 19:19:04 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Wed, 30 Nov 2022 19:10:04 +0100
>> From: Theodor Thornhill <theo@thornhill.no>
>> CC: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>>  jostein@kjonigsen.net
>> 
>> >Our automatic turning-on of major-modes looks at the file-name extension and
>> >little else (magic-mode-alist is not useful here).  So if a file whose
>> >extension is XYZ can have more than one applicable major-mode, we should try
>> >to do something to turn on the correct mode automatically.  What can we do?
>> 
>> We are lucky because the extensions in question are ts, tsx and js. So we don't need any smartness :)
>
>And the extensions always say unequivocally what mode to use?  Then yes, we
>are lucky, and need just one more major mode (and one more grammar library).

That's right!

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 30 November 2022 19:19:04 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Wed, 30 Nov 2022 19:10:04 +0100
>> From: Theodor Thornhill <theo@thornhill.no>
>> CC: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>>  jostein@kjonigsen.net
>> 
>> >Our automatic turning-on of major-modes looks at the file-name extension and
>> >little else (magic-mode-alist is not useful here).  So if a file whose
>> >extension is XYZ can have more than one applicable major-mode, we should try
>> >to do something to turn on the correct mode automatically.  What can we do?
>> 
>> We are lucky because the extensions in question are ts, tsx and js. So we don't need any smartness :)
>
>And the extensions always say unequivocally what mode to use?  Then yes, we
>are lucky, and need just one more major mode (and one more grammar library).

I have already made that commit, but it is waiting on the  indent style patch :)

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1148 bytes --]

Theodor Thornhill <theo@thornhill.no> writes:

> On 30 November 2022 19:19:04 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>>> Date: Wed, 30 Nov 2022 19:10:04 +0100
>>> From: Theodor Thornhill <theo@thornhill.no>
>>> CC: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>>>  jostein@kjonigsen.net
>>> 
>>> >Our automatic turning-on of major-modes looks at the file-name extension and
>>> >little else (magic-mode-alist is not useful here).  So if a file whose
>>> >extension is XYZ can have more than one applicable major-mode, we should try
>>> >to do something to turn on the correct mode automatically.  What can we do?
>>> 
>>> We are lucky because the extensions in question are ts, tsx and js. So we don't need any smartness :)
>>
>>And the extensions always say unequivocally what mode to use?  Then yes, we
>>are lucky, and need just one more major mode (and one more grammar library).
>
> I have already made that commit, but it is waiting on the  indent style patch :)

Ok, because it seems we didn't want the indent style patch let's fix
this bug instead :)

Attached is the new major mode.

What do you think?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-new-TypeScript-mode-tsx-ts-mode.patch --]
[-- Type: text/x-diff, Size: 12214 bytes --]

From 5cdae535295d92dfe7feb5ad595e836f31bd3e85 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Tue, 29 Nov 2022 21:39:38 +0100
Subject: [PATCH] Add new TypeScript mode tsx-ts-mode

There are in fact two languages supporting TypeScript for tree-sitter.
Because TSX causes some ambiguities with types there are two grammars,
one called typescript and one called tsx.  To account for this and to
be as correct as possible we enable using both.

* lisp/progmodes/typescript-ts-mode.el
(typescript-ts-mode--indent-rules): Change to a function to accomodate
the two languages.
(typescript-ts-mode--font-lock-settings): Change to a function to
accomodate the two languages.
(typescript-ts-mode--base-mode): Parent mode for typescript-ts-mode
and tsx-ts-mode.
(typescript-ts-mode): Derive from typescript-ts-mode--base-mode and
extend with language specific settings
(tsx-ts-mode): New major mode that derives from
typescript-ts-mode--base-mode and extend it with language specific
settings

Add autoload cookies for the respective file type extensions: .ts and
.tsx.

* etc/NEWS: Mention the new mode.
---
 etc/NEWS                             |   7 +-
 lisp/progmodes/typescript-ts-mode.el | 155 +++++++++++++++++----------
 2 files changed, 106 insertions(+), 56 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 4e091a5fed..439d20960b 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -2972,7 +2972,12 @@ A major mode based on the tree-sitter library for editing programs
 in the TypeScript language.  It includes support for font-locking,
 indentation, and navigation.
 
-** New major mode 'c-ts-mode'.
+** New mode 'tsx-ts-mode'.
+A major mode based on the tree-sitter library for editing programs
+in the TypeScript language, with support for TSX.  It includes
+support for font-locking, indentation, and navigation.
+
+** New mode 'c-ts-mode'.
 A major mode based on the tree-sitter library for editing programs
 in the C language.  It includes support for font-locking,
 indentation, Imenu, which-func, and navigation.
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 6c926a4e3e..c1beaf3134 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -22,6 +22,10 @@
 ;; You should have received a copy of the GNU General Public License
 ;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+
+;;; Commentary:
+;;
+
 ;;; Code:
 
 (require 'treesit)
@@ -56,8 +60,10 @@ typescript-ts-mode--syntax-table
     table)
   "Syntax table for `typescript-ts-mode'.")
 
-(defvar typescript-ts-mode--indent-rules
-  `((tsx
+(defun typescript-ts-mode--indent-rules (language)
+  "Rules used for indentation.
+Argument LANGUAGE is either 'typescript' or 'tsx'."
+  `((,language
      ((parent-is "program") parent-bol 0)
      ((node-is "}") parent-bol 0)
      ((node-is ")") parent-bol 0)
@@ -82,14 +88,13 @@ typescript-ts-mode--indent-rules
      ((parent-is "arrow_function") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "parenthesized_expression") parent-bol typescript-ts-mode-indent-offset)
 
-     ;; TSX
-     ((parent-is "jsx_opening_element") parent typescript-ts-mode-indent-offset)
-     ((node-is "jsx_closing_element") parent 0)
-     ((parent-is "jsx_element") parent typescript-ts-mode-indent-offset)
-     ((node-is "/") parent 0)
-     ((parent-is "jsx_self_closing_element") parent typescript-ts-mode-indent-offset)
-     (no-node parent-bol 0)))
-  "Tree-sitter indent rules.")
+     ,@(when (eq language 'tsx)
+         `(((parent-is "jsx_opening_element") parent typescript-ts-mode-indent-offset)
+           ((node-is "jsx_closing_element") parent 0)
+           ((parent-is "jsx_element") parent typescript-ts-mode-indent-offset)
+           ((node-is "/") parent 0)
+           ((parent-is "jsx_self_closing_element") parent typescript-ts-mode-indent-offset)))
+     (no-node parent-bol 0))))
 
 (defvar typescript-ts-mode--keywords
   '("!" "abstract" "as" "async" "await" "break"
@@ -110,14 +115,16 @@ typescript-ts-mode--operators
     "&&" "||" "!" "?.")
   "TypeScript operators for tree-sitter font-locking.")
 
-(defvar typescript-ts-mode--font-lock-settings
+(defun typescript-ts-mode--font-lock-settings (language)
+  "Tree-sitter font-lock settings.
+Argument LANGUAGE is either 'typescript' or 'tsx'."
   (treesit-font-lock-rules
-   :language 'tsx
+   :language language
    :override t
    :feature 'comment
    `((comment) @font-lock-comment-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'constant
    `(((identifier) @font-lock-constant-face
@@ -125,13 +132,13 @@ typescript-ts-mode--font-lock-settings
 
      [(true) (false) (null)] @font-lock-constant-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'keyword
    `([,@typescript-ts-mode--keywords] @font-lock-keyword-face
      [(this) (super)] @font-lock-keyword-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'string
    `((regex pattern: (regex_pattern)) @font-lock-string-face
@@ -139,7 +146,7 @@ typescript-ts-mode--font-lock-settings
      (template_string) @js--fontify-template-string
      (template_substitution ["${" "}"] @font-lock-builtin-face))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'declaration
    `((function
@@ -177,7 +184,7 @@ typescript-ts-mode--font-lock-settings
              (identifier) @font-lock-function-name-face)
       value: (array (number) (function))))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'identifier
    `((nested_type_identifier
@@ -208,7 +215,7 @@ typescript-ts-mode--font-lock-settings
        (_ (_ (identifier) @font-lock-variable-name-face))
        (_ (_ (_ (identifier) @font-lock-variable-name-face)))]))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'expression
    '((assignment_expression
@@ -223,7 +230,7 @@ typescript-ts-mode--font-lock-settings
        (member_expression
         property: (property_identifier) @font-lock-function-name-face)]))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'pattern
    `((pair_pattern
@@ -231,7 +238,7 @@ typescript-ts-mode--font-lock-settings
 
      (array_pattern (identifier) @font-lock-variable-name-face))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'jsx
    `((jsx_opening_element
@@ -248,31 +255,31 @@ typescript-ts-mode--font-lock-settings
 
      (jsx_attribute (property_identifier) @font-lock-constant-face))
 
-   :language 'tsx
+   :language language
    :feature 'number
    `((number) @font-lock-number-face
      ((identifier) @font-lock-number-face
       (:match "^\\(:?NaN\\|Infinity\\)$" @font-lock-number-face)))
 
-   :language 'tsx
+   :language language
    :feature 'operator
    `([,@typescript-ts-mode--operators] @font-lock-operator-face
      (ternary_expression ["?" ":"] @font-lock-operator-face))
 
-   :language 'tsx
+   :language language
    :feature 'bracket
    '((["(" ")" "[" "]" "{" "}"]) @font-lock-bracket-face)
 
-   :language 'tsx
+   :language language
    :feature 'delimiter
    '((["," "." ";" ":"]) @font-lock-delimiter-face)
 
-   :language 'tsx
+   :language language
    :feature 'escape-sequence
    :override t
    '((escape_sequence) @font-lock-escape-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'property
    `((pair value: (identifier) @font-lock-variable-name-face)
@@ -280,17 +287,71 @@ typescript-ts-mode--font-lock-settings
      ((shorthand_property_identifier) @font-lock-property-face)
 
      ((shorthand_property_identifier_pattern)
-      @font-lock-property-face)))
-  "Tree-sitter font-lock settings.")
+      @font-lock-property-face))))
 
 ;;;###autoload
 (add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))
 
 ;;;###autoload
-(add-to-list 'auto-mode-alist '("\\.tsx\\'" . typescript-ts-mode))
+(add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))
 
 ;;;###autoload
-(define-derived-mode typescript-ts-mode prog-mode "TypeScript"
+(define-derived-mode typescript-ts-mode--base-mode prog-mode "TypeScript"
+  "Major mode for editing TypeScript."
+  :group 'typescript
+  :syntax-table typescript-ts-mode--syntax-table
+
+  ;; Comments.
+  (setq-local comment-start "// ")
+  (setq-local comment-end "")
+  (setq-local comment-start-skip "\\(?://+\\|/\\*+\\)\\s *")
+  (setq-local comment-end-skip
+              (rx (* (syntax whitespace))
+                  (group (or (syntax comment-end)
+                             (seq (+ "*") "/")))))
+
+  ;; Electric
+  (setq-local electric-indent-chars
+              (append "{}():;," electric-indent-chars))
+
+  ;; Navigation.
+  (setq-local treesit-defun-type-regexp
+              (rx (or "class_declaration"
+                      "method_definition"
+                      "function_declaration"
+                      "lexical_declaration")))
+  ;; Imenu.
+  (setq-local imenu-create-index-function #'js--treesit-imenu)
+
+  ;; Which-func (use imenu).
+  (setq-local which-func-functions nil))
+
+;;;###autoload
+(define-derived-mode typescript-ts-mode typescript-ts-mode--base-mode "TypeScript"
+  "Major mode for editing TypeScript."
+  :group 'typescript
+  :syntax-table typescript-ts-mode--syntax-table
+
+  (when (treesit-ready-p 'typescript)
+    (treesit-parser-create 'typescript)
+
+    ;; Indent.
+    (setq-local treesit-simple-indent-rules
+                (typescript-ts-mode--indent-rules 'typescript))
+
+    ;; Font-lock.
+    (setq-local treesit-font-lock-settings
+                (typescript-ts-mode--font-lock-settings 'typescript))
+    (setq-local treesit-font-lock-feature-list
+                '((comment declaration)
+                  (keyword string)
+                  (constant expression identifier number pattern property)
+                  (bracket delimiter)))
+
+    (treesit-major-mode-setup)))
+
+;;;###autoload
+(define-derived-mode tsx-ts-mode typescript-ts-mode--base-mode "TypeScript[TSX]"
   "Major mode for editing TypeScript."
   :group 'typescript
   :syntax-table typescript-ts-mode--syntax-table
@@ -301,43 +362,27 @@ typescript-ts-mode
     ;; Comments.
     (setq-local comment-start "// ")
     (setq-local comment-end "")
-    (setq-local comment-start-skip (rx (group "/" (or (+ "/") (+ "*")))
-                                       (* (syntax whitespace))))
+    (setq-local comment-start-skip "\\(?://+\\|/\\*+\\)\\s *")
     (setq-local comment-end-skip
                 (rx (* (syntax whitespace))
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
 
-    ;; Electric
-    (setq-local electric-indent-chars
-                (append "{}():;," electric-indent-chars))
-
     ;; Indent.
-    (setq-local treesit-simple-indent-rules typescript-ts-mode--indent-rules)
-
-    ;; Navigation.
-    (setq-local treesit-defun-type-regexp
-                (rx (or "class_declaration"
-                        "method_definition"
-                        "function_declaration"
-                        "lexical_declaration")))
+    (setq-local treesit-simple-indent-rules
+                (typescript-ts-mode--indent-rules 'tsx))
 
     ;; Font-lock.
-    (setq-local treesit-font-lock-settings typescript-ts-mode--font-lock-settings)
+    (setq-local treesit-font-lock-settings
+                (typescript-ts-mode--font-lock-settings 'tsx))
     (setq-local treesit-font-lock-feature-list
-                '(( comment declaration)
-                  ( keyword string)
-                  ( constant expression identifier jsx number pattern property)
-                  ( bracket delimiter)))
-    ;; Imenu.
-    (setq-local imenu-create-index-function #'js--treesit-imenu)
-
-    ;; Which-func (use imenu).
-    (setq-local which-func-functions nil)
+                '((comment declaration)
+                  (keyword string)
+                  (constant expression identifier jsx number pattern property)
+                  (bracket delimiter)))
 
     (treesit-major-mode-setup)))
 
-
 (provide 'typescript-ts-mode)
 
 ;;; typescript-ts-mode.el ends here
-- 
2.34.1

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>  jostein@kjonigsen.net
> Date: Thu, 01 Dec 2022 07:01:55 +0100
> 
> diff --git a/etc/NEWS b/etc/NEWS
> index 4e091a5fed..439d20960b 100644
> --- a/etc/NEWS
> +++ b/etc/NEWS
> @@ -2972,7 +2972,12 @@ A major mode based on the tree-sitter library for editing programs
>  in the TypeScript language.  It includes support for font-locking,
>  indentation, and navigation.
>  
> -** New major mode 'c-ts-mode'.
> +** New mode 'tsx-ts-mode'.
> +A major mode based on the tree-sitter library for editing programs
> +in the TypeScript language, with support for TSX.  It includes
> +support for font-locking, indentation, and navigation.
> +
> +** New mode 'c-ts-mode'.

Looks like some "git merge" snafu?  You are in fact reverting a change I
made in NEWS yesterday.

> diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
> index 6c926a4e3e..c1beaf3134 100644
> --- a/lisp/progmodes/typescript-ts-mode.el
> +++ b/lisp/progmodes/typescript-ts-mode.el

I don't see a change to auto-mode-alist to turn on each mode for the files
it supports?  I thought this was the idea?  Or is this because we don't want
tree-sitter based modes to be turned on by default?  In that case, how do we
explain to users that they should use each mode in the relevant cases?

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1748 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>>  jostein@kjonigsen.net
>> Date: Thu, 01 Dec 2022 07:01:55 +0100
>> 
>> diff --git a/etc/NEWS b/etc/NEWS
>> index 4e091a5fed..439d20960b 100644
>> --- a/etc/NEWS
>> +++ b/etc/NEWS
>> @@ -2972,7 +2972,12 @@ A major mode based on the tree-sitter library for editing programs
>>  in the TypeScript language.  It includes support for font-locking,
>>  indentation, and navigation.
>>  
>> -** New major mode 'c-ts-mode'.
>> +** New mode 'tsx-ts-mode'.
>> +A major mode based on the tree-sitter library for editing programs
>> +in the TypeScript language, with support for TSX.  It includes
>> +support for font-locking, indentation, and navigation.
>> +
>> +** New mode 'c-ts-mode'.
>
> Looks like some "git merge" snafu?  You are in fact reverting a change I
> made in NEWS yesterday.
>

You're right - I think I fixed it now.

>> diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
>> index 6c926a4e3e..c1beaf3134 100644
>> --- a/lisp/progmodes/typescript-ts-mode.el
>> +++ b/lisp/progmodes/typescript-ts-mode.el
>
> I don't see a change to auto-mode-alist to turn on each mode for the files
> it supports?  I thought this was the idea?  Or is this because we don't want
> tree-sitter based modes to be turned on by default?  In that case, how do we
> explain to users that they should use each mode in the relevant cases?

I think there should be


;;;###autoload
(add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))

;;;###autoload
(add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))

No?  If not, then I'm confused.

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-new-TypeScript-mode-tsx-ts-mode.patch --]
[-- Type: text/x-diff, Size: 12144 bytes --]

From 79c446694b36ccad56d09993f13e46052ba979d1 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Tue, 29 Nov 2022 21:39:38 +0100
Subject: [PATCH] Add new TypeScript mode tsx-ts-mode

There are in fact two languages supporting TypeScript for tree-sitter.
Because TSX causes some ambiguities with types there are two grammars,
one called typescript and one called tsx.  To account for this and to
be as correct as possible we enable using both.

* lisp/progmodes/typescript-ts-mode.el
(typescript-ts-mode--indent-rules): Change to a function to accomodate
the two languages.
(typescript-ts-mode--font-lock-settings): Change to a function to
accomodate the two languages.
(typescript-ts-mode--base-mode): Parent mode for typescript-ts-mode
and tsx-ts-mode.
(typescript-ts-mode): Derive from typescript-ts-mode--base-mode and
extend with language specific settings
(tsx-ts-mode): New major mode that derives from
typescript-ts-mode--base-mode and extend it with language specific
settings

Add autoload cookies for the respective file type extensions: .ts and
.tsx.

* etc/NEWS: Mention the new mode.
---
 etc/NEWS                             |   5 +
 lisp/progmodes/typescript-ts-mode.el | 155 +++++++++++++++++----------
 2 files changed, 105 insertions(+), 55 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 4e091a5fed..1faf3bc097 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -2972,6 +2972,11 @@ A major mode based on the tree-sitter library for editing programs
 in the TypeScript language.  It includes support for font-locking,
 indentation, and navigation.
 
+** New major mode 'tsx-ts-mode'.
+A major mode based on the tree-sitter library for editing programs
+in the TypeScript language, with support for TSX.  It includes
+support for font-locking, indentation, and navigation.
+
 ** New major mode 'c-ts-mode'.
 A major mode based on the tree-sitter library for editing programs
 in the C language.  It includes support for font-locking,
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 6c926a4e3e..c1beaf3134 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -22,6 +22,10 @@
 ;; You should have received a copy of the GNU General Public License
 ;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+
+;;; Commentary:
+;;
+
 ;;; Code:
 
 (require 'treesit)
@@ -56,8 +60,10 @@ typescript-ts-mode--syntax-table
     table)
   "Syntax table for `typescript-ts-mode'.")
 
-(defvar typescript-ts-mode--indent-rules
-  `((tsx
+(defun typescript-ts-mode--indent-rules (language)
+  "Rules used for indentation.
+Argument LANGUAGE is either 'typescript' or 'tsx'."
+  `((,language
      ((parent-is "program") parent-bol 0)
      ((node-is "}") parent-bol 0)
      ((node-is ")") parent-bol 0)
@@ -82,14 +88,13 @@ typescript-ts-mode--indent-rules
      ((parent-is "arrow_function") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "parenthesized_expression") parent-bol typescript-ts-mode-indent-offset)
 
-     ;; TSX
-     ((parent-is "jsx_opening_element") parent typescript-ts-mode-indent-offset)
-     ((node-is "jsx_closing_element") parent 0)
-     ((parent-is "jsx_element") parent typescript-ts-mode-indent-offset)
-     ((node-is "/") parent 0)
-     ((parent-is "jsx_self_closing_element") parent typescript-ts-mode-indent-offset)
-     (no-node parent-bol 0)))
-  "Tree-sitter indent rules.")
+     ,@(when (eq language 'tsx)
+         `(((parent-is "jsx_opening_element") parent typescript-ts-mode-indent-offset)
+           ((node-is "jsx_closing_element") parent 0)
+           ((parent-is "jsx_element") parent typescript-ts-mode-indent-offset)
+           ((node-is "/") parent 0)
+           ((parent-is "jsx_self_closing_element") parent typescript-ts-mode-indent-offset)))
+     (no-node parent-bol 0))))
 
 (defvar typescript-ts-mode--keywords
   '("!" "abstract" "as" "async" "await" "break"
@@ -110,14 +115,16 @@ typescript-ts-mode--operators
     "&&" "||" "!" "?.")
   "TypeScript operators for tree-sitter font-locking.")
 
-(defvar typescript-ts-mode--font-lock-settings
+(defun typescript-ts-mode--font-lock-settings (language)
+  "Tree-sitter font-lock settings.
+Argument LANGUAGE is either 'typescript' or 'tsx'."
   (treesit-font-lock-rules
-   :language 'tsx
+   :language language
    :override t
    :feature 'comment
    `((comment) @font-lock-comment-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'constant
    `(((identifier) @font-lock-constant-face
@@ -125,13 +132,13 @@ typescript-ts-mode--font-lock-settings
 
      [(true) (false) (null)] @font-lock-constant-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'keyword
    `([,@typescript-ts-mode--keywords] @font-lock-keyword-face
      [(this) (super)] @font-lock-keyword-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'string
    `((regex pattern: (regex_pattern)) @font-lock-string-face
@@ -139,7 +146,7 @@ typescript-ts-mode--font-lock-settings
      (template_string) @js--fontify-template-string
      (template_substitution ["${" "}"] @font-lock-builtin-face))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'declaration
    `((function
@@ -177,7 +184,7 @@ typescript-ts-mode--font-lock-settings
              (identifier) @font-lock-function-name-face)
       value: (array (number) (function))))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'identifier
    `((nested_type_identifier
@@ -208,7 +215,7 @@ typescript-ts-mode--font-lock-settings
        (_ (_ (identifier) @font-lock-variable-name-face))
        (_ (_ (_ (identifier) @font-lock-variable-name-face)))]))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'expression
    '((assignment_expression
@@ -223,7 +230,7 @@ typescript-ts-mode--font-lock-settings
        (member_expression
         property: (property_identifier) @font-lock-function-name-face)]))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'pattern
    `((pair_pattern
@@ -231,7 +238,7 @@ typescript-ts-mode--font-lock-settings
 
      (array_pattern (identifier) @font-lock-variable-name-face))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'jsx
    `((jsx_opening_element
@@ -248,31 +255,31 @@ typescript-ts-mode--font-lock-settings
 
      (jsx_attribute (property_identifier) @font-lock-constant-face))
 
-   :language 'tsx
+   :language language
    :feature 'number
    `((number) @font-lock-number-face
      ((identifier) @font-lock-number-face
       (:match "^\\(:?NaN\\|Infinity\\)$" @font-lock-number-face)))
 
-   :language 'tsx
+   :language language
    :feature 'operator
    `([,@typescript-ts-mode--operators] @font-lock-operator-face
      (ternary_expression ["?" ":"] @font-lock-operator-face))
 
-   :language 'tsx
+   :language language
    :feature 'bracket
    '((["(" ")" "[" "]" "{" "}"]) @font-lock-bracket-face)
 
-   :language 'tsx
+   :language language
    :feature 'delimiter
    '((["," "." ";" ":"]) @font-lock-delimiter-face)
 
-   :language 'tsx
+   :language language
    :feature 'escape-sequence
    :override t
    '((escape_sequence) @font-lock-escape-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'property
    `((pair value: (identifier) @font-lock-variable-name-face)
@@ -280,17 +287,71 @@ typescript-ts-mode--font-lock-settings
      ((shorthand_property_identifier) @font-lock-property-face)
 
      ((shorthand_property_identifier_pattern)
-      @font-lock-property-face)))
-  "Tree-sitter font-lock settings.")
+      @font-lock-property-face))))
 
 ;;;###autoload
 (add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))
 
 ;;;###autoload
-(add-to-list 'auto-mode-alist '("\\.tsx\\'" . typescript-ts-mode))
+(add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))
 
 ;;;###autoload
-(define-derived-mode typescript-ts-mode prog-mode "TypeScript"
+(define-derived-mode typescript-ts-mode--base-mode prog-mode "TypeScript"
+  "Major mode for editing TypeScript."
+  :group 'typescript
+  :syntax-table typescript-ts-mode--syntax-table
+
+  ;; Comments.
+  (setq-local comment-start "// ")
+  (setq-local comment-end "")
+  (setq-local comment-start-skip "\\(?://+\\|/\\*+\\)\\s *")
+  (setq-local comment-end-skip
+              (rx (* (syntax whitespace))
+                  (group (or (syntax comment-end)
+                             (seq (+ "*") "/")))))
+
+  ;; Electric
+  (setq-local electric-indent-chars
+              (append "{}():;," electric-indent-chars))
+
+  ;; Navigation.
+  (setq-local treesit-defun-type-regexp
+              (rx (or "class_declaration"
+                      "method_definition"
+                      "function_declaration"
+                      "lexical_declaration")))
+  ;; Imenu.
+  (setq-local imenu-create-index-function #'js--treesit-imenu)
+
+  ;; Which-func (use imenu).
+  (setq-local which-func-functions nil))
+
+;;;###autoload
+(define-derived-mode typescript-ts-mode typescript-ts-mode--base-mode "TypeScript"
+  "Major mode for editing TypeScript."
+  :group 'typescript
+  :syntax-table typescript-ts-mode--syntax-table
+
+  (when (treesit-ready-p 'typescript)
+    (treesit-parser-create 'typescript)
+
+    ;; Indent.
+    (setq-local treesit-simple-indent-rules
+                (typescript-ts-mode--indent-rules 'typescript))
+
+    ;; Font-lock.
+    (setq-local treesit-font-lock-settings
+                (typescript-ts-mode--font-lock-settings 'typescript))
+    (setq-local treesit-font-lock-feature-list
+                '((comment declaration)
+                  (keyword string)
+                  (constant expression identifier number pattern property)
+                  (bracket delimiter)))
+
+    (treesit-major-mode-setup)))
+
+;;;###autoload
+(define-derived-mode tsx-ts-mode typescript-ts-mode--base-mode "TypeScript[TSX]"
   "Major mode for editing TypeScript."
   :group 'typescript
   :syntax-table typescript-ts-mode--syntax-table
@@ -301,43 +362,27 @@ typescript-ts-mode
     ;; Comments.
     (setq-local comment-start "// ")
     (setq-local comment-end "")
-    (setq-local comment-start-skip (rx (group "/" (or (+ "/") (+ "*")))
-                                       (* (syntax whitespace))))
+    (setq-local comment-start-skip "\\(?://+\\|/\\*+\\)\\s *")
     (setq-local comment-end-skip
                 (rx (* (syntax whitespace))
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
 
-    ;; Electric
-    (setq-local electric-indent-chars
-                (append "{}():;," electric-indent-chars))
-
     ;; Indent.
-    (setq-local treesit-simple-indent-rules typescript-ts-mode--indent-rules)
-
-    ;; Navigation.
-    (setq-local treesit-defun-type-regexp
-                (rx (or "class_declaration"
-                        "method_definition"
-                        "function_declaration"
-                        "lexical_declaration")))
+    (setq-local treesit-simple-indent-rules
+                (typescript-ts-mode--indent-rules 'tsx))
 
     ;; Font-lock.
-    (setq-local treesit-font-lock-settings typescript-ts-mode--font-lock-settings)
+    (setq-local treesit-font-lock-settings
+                (typescript-ts-mode--font-lock-settings 'tsx))
     (setq-local treesit-font-lock-feature-list
-                '(( comment declaration)
-                  ( keyword string)
-                  ( constant expression identifier jsx number pattern property)
-                  ( bracket delimiter)))
-    ;; Imenu.
-    (setq-local imenu-create-index-function #'js--treesit-imenu)
-
-    ;; Which-func (use imenu).
-    (setq-local which-func-functions nil)
+                '((comment declaration)
+                  (keyword string)
+                  (constant expression identifier jsx number pattern property)
+                  (bracket delimiter)))
 
     (treesit-major-mode-setup)))
 
-
 (provide 'typescript-ts-mode)
 
 ;;; typescript-ts-mode.el ends here
-- 
2.34.1

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>  jostein@kjonigsen.net
> Date: Thu, 01 Dec 2022 09:12:58 +0100
> 
> > Looks like some "git merge" snafu?  You are in fact reverting a change I
> > made in NEWS yesterday.
> >
> 
> You're right - I think I fixed it now.

Yes, thanks.

> > I don't see a change to auto-mode-alist to turn on each mode for the files
> > it supports?  I thought this was the idea?  Or is this because we don't want
> > tree-sitter based modes to be turned on by default?  In that case, how do we
> > explain to users that they should use each mode in the relevant cases?
> 
> I think there should be
> 
> 
> ;;;###autoload
> (add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))
> 
> ;;;###autoload
> (add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))

Yes, I missed that, sorry.  But the NEWS entry should clearly say that one
mode is the default for *.ts files, the other for *.tsx files.

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1134 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>>  jostein@kjonigsen.net
>> Date: Thu, 01 Dec 2022 09:12:58 +0100
>> 
>> > Looks like some "git merge" snafu?  You are in fact reverting a change I
>> > made in NEWS yesterday.
>> >
>> 
>> You're right - I think I fixed it now.
>
> Yes, thanks.
>
>> > I don't see a change to auto-mode-alist to turn on each mode for the files
>> > it supports?  I thought this was the idea?  Or is this because we don't want
>> > tree-sitter based modes to be turned on by default?  In that case, how do we
>> > explain to users that they should use each mode in the relevant cases?
>> 
>> I think there should be
>> 
>> 
>> ;;;###autoload
>> (add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))
>> 
>> ;;;###autoload
>> (add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))
>
> Yes, I missed that, sorry.  But the NEWS entry should clearly say that one
> mode is the default for *.ts files, the other for *.tsx files.

No worries :)

Is this patch suitable?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-new-TypeScript-mode-tsx-ts-mode.patch --]
[-- Type: text/x-diff, Size: 12323 bytes --]

From bef31846598dfce861a32abdb8cc34c884ce12cc Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Tue, 29 Nov 2022 21:39:38 +0100
Subject: [PATCH] Add new TypeScript mode tsx-ts-mode

There are in fact two languages supporting TypeScript for tree-sitter.
Because TSX causes some ambiguities with types there are two grammars,
one called typescript and one called tsx.  To account for this and to
be as correct as possible we enable using both.

* lisp/progmodes/typescript-ts-mode.el
(typescript-ts-mode--indent-rules): Change to a function to accomodate
the two languages.
(typescript-ts-mode--font-lock-settings): Change to a function to
accomodate the two languages.
(typescript-ts-mode--base-mode): Parent mode for typescript-ts-mode
and tsx-ts-mode.
(typescript-ts-mode): Derive from typescript-ts-mode--base-mode and
extend with language specific settings
(tsx-ts-mode): New major mode that derives from
typescript-ts-mode--base-mode and extend it with language specific
settings

Add autoload cookies for the respective file type extensions: .ts and
.tsx.

* etc/NEWS: Mention the new mode.
---
 etc/NEWS                             |   9 +-
 lisp/progmodes/typescript-ts-mode.el | 155 +++++++++++++++++----------
 2 files changed, 108 insertions(+), 56 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 4e091a5fed..2c1c220f51 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -2970,7 +2970,14 @@ when visiting JSON files.
 ** New major mode 'typescript-ts-mode'.
 A major mode based on the tree-sitter library for editing programs
 in the TypeScript language.  It includes support for font-locking,
-indentation, and navigation.
+indentation, and navigation.  This mode will be auto-enabled for
+files with the '.ts' extension.
+
+** New major mode 'tsx-ts-mode'.
+A major mode based on the tree-sitter library for editing programs
+in the TypeScript language, with support for TSX.  It includes
+support for font-locking, indentation, and navigation.  This mode
+will be auto-enabled for files with the '.tsx' extension.
 
 ** New major mode 'c-ts-mode'.
 A major mode based on the tree-sitter library for editing programs
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 6c926a4e3e..c1beaf3134 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -22,6 +22,10 @@
 ;; You should have received a copy of the GNU General Public License
 ;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+
+;;; Commentary:
+;;
+
 ;;; Code:
 
 (require 'treesit)
@@ -56,8 +60,10 @@ typescript-ts-mode--syntax-table
     table)
   "Syntax table for `typescript-ts-mode'.")
 
-(defvar typescript-ts-mode--indent-rules
-  `((tsx
+(defun typescript-ts-mode--indent-rules (language)
+  "Rules used for indentation.
+Argument LANGUAGE is either 'typescript' or 'tsx'."
+  `((,language
      ((parent-is "program") parent-bol 0)
      ((node-is "}") parent-bol 0)
      ((node-is ")") parent-bol 0)
@@ -82,14 +88,13 @@ typescript-ts-mode--indent-rules
      ((parent-is "arrow_function") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "parenthesized_expression") parent-bol typescript-ts-mode-indent-offset)
 
-     ;; TSX
-     ((parent-is "jsx_opening_element") parent typescript-ts-mode-indent-offset)
-     ((node-is "jsx_closing_element") parent 0)
-     ((parent-is "jsx_element") parent typescript-ts-mode-indent-offset)
-     ((node-is "/") parent 0)
-     ((parent-is "jsx_self_closing_element") parent typescript-ts-mode-indent-offset)
-     (no-node parent-bol 0)))
-  "Tree-sitter indent rules.")
+     ,@(when (eq language 'tsx)
+         `(((parent-is "jsx_opening_element") parent typescript-ts-mode-indent-offset)
+           ((node-is "jsx_closing_element") parent 0)
+           ((parent-is "jsx_element") parent typescript-ts-mode-indent-offset)
+           ((node-is "/") parent 0)
+           ((parent-is "jsx_self_closing_element") parent typescript-ts-mode-indent-offset)))
+     (no-node parent-bol 0))))
 
 (defvar typescript-ts-mode--keywords
   '("!" "abstract" "as" "async" "await" "break"
@@ -110,14 +115,16 @@ typescript-ts-mode--operators
     "&&" "||" "!" "?.")
   "TypeScript operators for tree-sitter font-locking.")
 
-(defvar typescript-ts-mode--font-lock-settings
+(defun typescript-ts-mode--font-lock-settings (language)
+  "Tree-sitter font-lock settings.
+Argument LANGUAGE is either 'typescript' or 'tsx'."
   (treesit-font-lock-rules
-   :language 'tsx
+   :language language
    :override t
    :feature 'comment
    `((comment) @font-lock-comment-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'constant
    `(((identifier) @font-lock-constant-face
@@ -125,13 +132,13 @@ typescript-ts-mode--font-lock-settings
 
      [(true) (false) (null)] @font-lock-constant-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'keyword
    `([,@typescript-ts-mode--keywords] @font-lock-keyword-face
      [(this) (super)] @font-lock-keyword-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'string
    `((regex pattern: (regex_pattern)) @font-lock-string-face
@@ -139,7 +146,7 @@ typescript-ts-mode--font-lock-settings
      (template_string) @js--fontify-template-string
      (template_substitution ["${" "}"] @font-lock-builtin-face))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'declaration
    `((function
@@ -177,7 +184,7 @@ typescript-ts-mode--font-lock-settings
              (identifier) @font-lock-function-name-face)
       value: (array (number) (function))))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'identifier
    `((nested_type_identifier
@@ -208,7 +215,7 @@ typescript-ts-mode--font-lock-settings
        (_ (_ (identifier) @font-lock-variable-name-face))
        (_ (_ (_ (identifier) @font-lock-variable-name-face)))]))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'expression
    '((assignment_expression
@@ -223,7 +230,7 @@ typescript-ts-mode--font-lock-settings
        (member_expression
         property: (property_identifier) @font-lock-function-name-face)]))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'pattern
    `((pair_pattern
@@ -231,7 +238,7 @@ typescript-ts-mode--font-lock-settings
 
      (array_pattern (identifier) @font-lock-variable-name-face))
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'jsx
    `((jsx_opening_element
@@ -248,31 +255,31 @@ typescript-ts-mode--font-lock-settings
 
      (jsx_attribute (property_identifier) @font-lock-constant-face))
 
-   :language 'tsx
+   :language language
    :feature 'number
    `((number) @font-lock-number-face
      ((identifier) @font-lock-number-face
       (:match "^\\(:?NaN\\|Infinity\\)$" @font-lock-number-face)))
 
-   :language 'tsx
+   :language language
    :feature 'operator
    `([,@typescript-ts-mode--operators] @font-lock-operator-face
      (ternary_expression ["?" ":"] @font-lock-operator-face))
 
-   :language 'tsx
+   :language language
    :feature 'bracket
    '((["(" ")" "[" "]" "{" "}"]) @font-lock-bracket-face)
 
-   :language 'tsx
+   :language language
    :feature 'delimiter
    '((["," "." ";" ":"]) @font-lock-delimiter-face)
 
-   :language 'tsx
+   :language language
    :feature 'escape-sequence
    :override t
    '((escape_sequence) @font-lock-escape-face)
 
-   :language 'tsx
+   :language language
    :override t
    :feature 'property
    `((pair value: (identifier) @font-lock-variable-name-face)
@@ -280,17 +287,71 @@ typescript-ts-mode--font-lock-settings
      ((shorthand_property_identifier) @font-lock-property-face)
 
      ((shorthand_property_identifier_pattern)
-      @font-lock-property-face)))
-  "Tree-sitter font-lock settings.")
+      @font-lock-property-face))))
 
 ;;;###autoload
 (add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))
 
 ;;;###autoload
-(add-to-list 'auto-mode-alist '("\\.tsx\\'" . typescript-ts-mode))
+(add-to-list 'auto-mode-alist '("\\.tsx\\'" . tsx-ts-mode))
 
 ;;;###autoload
-(define-derived-mode typescript-ts-mode prog-mode "TypeScript"
+(define-derived-mode typescript-ts-mode--base-mode prog-mode "TypeScript"
+  "Major mode for editing TypeScript."
+  :group 'typescript
+  :syntax-table typescript-ts-mode--syntax-table
+
+  ;; Comments.
+  (setq-local comment-start "// ")
+  (setq-local comment-end "")
+  (setq-local comment-start-skip "\\(?://+\\|/\\*+\\)\\s *")
+  (setq-local comment-end-skip
+              (rx (* (syntax whitespace))
+                  (group (or (syntax comment-end)
+                             (seq (+ "*") "/")))))
+
+  ;; Electric
+  (setq-local electric-indent-chars
+              (append "{}():;," electric-indent-chars))
+
+  ;; Navigation.
+  (setq-local treesit-defun-type-regexp
+              (rx (or "class_declaration"
+                      "method_definition"
+                      "function_declaration"
+                      "lexical_declaration")))
+  ;; Imenu.
+  (setq-local imenu-create-index-function #'js--treesit-imenu)
+
+  ;; Which-func (use imenu).
+  (setq-local which-func-functions nil))
+
+;;;###autoload
+(define-derived-mode typescript-ts-mode typescript-ts-mode--base-mode "TypeScript"
+  "Major mode for editing TypeScript."
+  :group 'typescript
+  :syntax-table typescript-ts-mode--syntax-table
+
+  (when (treesit-ready-p 'typescript)
+    (treesit-parser-create 'typescript)
+
+    ;; Indent.
+    (setq-local treesit-simple-indent-rules
+                (typescript-ts-mode--indent-rules 'typescript))
+
+    ;; Font-lock.
+    (setq-local treesit-font-lock-settings
+                (typescript-ts-mode--font-lock-settings 'typescript))
+    (setq-local treesit-font-lock-feature-list
+                '((comment declaration)
+                  (keyword string)
+                  (constant expression identifier number pattern property)
+                  (bracket delimiter)))
+
+    (treesit-major-mode-setup)))
+
+;;;###autoload
+(define-derived-mode tsx-ts-mode typescript-ts-mode--base-mode "TypeScript[TSX]"
   "Major mode for editing TypeScript."
   :group 'typescript
   :syntax-table typescript-ts-mode--syntax-table
@@ -301,43 +362,27 @@ typescript-ts-mode
     ;; Comments.
     (setq-local comment-start "// ")
     (setq-local comment-end "")
-    (setq-local comment-start-skip (rx (group "/" (or (+ "/") (+ "*")))
-                                       (* (syntax whitespace))))
+    (setq-local comment-start-skip "\\(?://+\\|/\\*+\\)\\s *")
     (setq-local comment-end-skip
                 (rx (* (syntax whitespace))
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
 
-    ;; Electric
-    (setq-local electric-indent-chars
-                (append "{}():;," electric-indent-chars))
-
     ;; Indent.
-    (setq-local treesit-simple-indent-rules typescript-ts-mode--indent-rules)
-
-    ;; Navigation.
-    (setq-local treesit-defun-type-regexp
-                (rx (or "class_declaration"
-                        "method_definition"
-                        "function_declaration"
-                        "lexical_declaration")))
+    (setq-local treesit-simple-indent-rules
+                (typescript-ts-mode--indent-rules 'tsx))
 
     ;; Font-lock.
-    (setq-local treesit-font-lock-settings typescript-ts-mode--font-lock-settings)
+    (setq-local treesit-font-lock-settings
+                (typescript-ts-mode--font-lock-settings 'tsx))
     (setq-local treesit-font-lock-feature-list
-                '(( comment declaration)
-                  ( keyword string)
-                  ( constant expression identifier jsx number pattern property)
-                  ( bracket delimiter)))
-    ;; Imenu.
-    (setq-local imenu-create-index-function #'js--treesit-imenu)
-
-    ;; Which-func (use imenu).
-    (setq-local which-func-functions nil)
+                '((comment declaration)
+                  (keyword string)
+                  (constant expression identifier jsx number pattern property)
+                  (bracket delimiter)))
 
     (treesit-major-mode-setup)))
 
-
 (provide 'typescript-ts-mode)
 
 ;;; typescript-ts-mode.el ends here
-- 
2.34.1

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>  jostein@kjonigsen.net
> Date: Thu, 01 Dec 2022 11:52:00 +0100
> 
> > Yes, I missed that, sorry.  But the NEWS entry should clearly say that one
> > mode is the default for *.ts files, the other for *.tsx files.
> 
> No worries :)
> 
> Is this patch suitable?

LGTM, thanks.

Yuan, if you are okay with this, please install.

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Yuan Fu]

> On Dec 1, 2022, at 4:10 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>> jostein@kjonigsen.net
>> Date: Thu, 01 Dec 2022 11:52:00 +0100
>> 
>>> Yes, I missed that, sorry.  But the NEWS entry should clearly say that one
>>> mode is the default for *.ts files, the other for *.tsx files.
>> 
>> No worries :)
>> 
>> Is this patch suitable?
> 
> LGTM, thanks.
> 
> Yuan, if you are okay with this, please install.

Cool. I applied the patch, thanks Theo! I changed typescript-ts-mode—base-mode to typescript-ts-base-mode, since I don’t think anyone has strong opinions about its naming, and typescript-ts-base-mode is more consistent with other tree-sitter base modes.

Yuan

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Yuan Fu <casouri@gmail.com> writes:

>> On Dec 1, 2022, at 4:10 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>>> From: Theodor Thornhill <theo@thornhill.no>
>>> Cc: casouri@gmail.com, jostein@secure.kjonigsen.net, 59691@debbugs.gnu.org,
>>> jostein@kjonigsen.net
>>> Date: Thu, 01 Dec 2022 11:52:00 +0100
>>> 
>>>> Yes, I missed that, sorry.  But the NEWS entry should clearly say that one
>>>> mode is the default for *.ts files, the other for *.tsx files.
>>> 
>>> No worries :)
>>> 
>>> Is this patch suitable?
>> 
>> LGTM, thanks.
>> 
>> Yuan, if you are okay with this, please install.
>
> Cool. I applied the patch, thanks Theo! I changed typescript-ts-mode—base-mode to typescript-ts-base-mode, since I don’t think anyone has strong opinions about its naming, and typescript-ts-base-mode is more consistent with other tree-sitter base modes.
>
> Yuan

Thanks!  I believe this can be closed.  What do you think, Jostein?

Theo

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Jostein Kjønigsen]

>> Cool. I applied the patch, thanks Theo! I changed 
>> typescript-ts-mode—base-mode to typescript-ts-base-mode, since I 
>> don’t think anyone has strong opinions about its naming, and 
>> typescript-ts-base-mode is more consistent with other tree-sitter 
>> base modes.
>> Yuan
> Thanks!  I believe this can be closed.  What do you think, Jostein?
>
> Theo

Hey guys.

Amazing work! Sorry for being somewhat slow wrt to testing.

I had to rebuild some tree-sitter libraries, and the build-code in 
admin/notes/tree-sitter/ seemingly didn't work out of the box for both 
plain typescript and tsx. Yadda, yadda, yadda. I've modified the scripts 
to also build for typescript/tsx. Do you wan't me to send in a patch for 
that, Yuan?

Back to the point: Either way, I've tested both TS-code and TSX-code 
now, and it seems to work much better.

I still see some errors in TSX-code I can't recall seing in the old mode 
(and I might report those issues, once I have time to collect more 
accurate data on them), but for now I definitely think what we have here 
warrants marking this particular bug as closed. :)

--
Jostein

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 2 December 2022 22:28:36 CET, "Jostein Kjønigsen" <jostein@secure.kjonigsen.net> wrote:
>
>>> Cool. I applied the patch, thanks Theo! I changed typescript-ts-mode—base-mode to typescript-ts-base-mode, since I don’t think anyone has strong opinions about its naming, and typescript-ts-base-mode is more consistent with other tree-sitter base modes.
>>> Yuan
>> Thanks!  I believe this can be closed.  What do you think, Jostein?
>> 
>> Theo
>
>Hey guys.
>
>Amazing work! Sorry for being somewhat slow wrt to testing.
>
>I had to rebuild some tree-sitter libraries, and the build-code in admin/notes/tree-sitter/ seemingly didn't work out of the box for both plain typescript and tsx. Yadda, yadda, yadda. I've modified the scripts to also build for typescript/tsx. Do you wan't me to send in a patch for that, Yuan?
>

I've already sent a patch for that :)


>Back to the point: Either way, I've tested both TS-code and TSX-code now, and it seems to work much better.
>

Yay!

>I still see some errors in TSX-code I can't recall seing in the old mode (and I might report those issues, once I have time to collect more accurate data on them), but for now I definitely think what we have here warrants marking this particular bug as closed. :)

Great news! We'll get to the other stuff in due time :)

Theo

bug#59691: 29.0.60; typescript-ts-mode: any HTML-like elements causes fontification to become invalid and remaining parse-tree to become jsx-expression

[Eli Zaretskii]

> Date: Fri, 2 Dec 2022 22:28:36 +0100
> Cc: 59691@debbugs.gnu.org, jostein@kjonigsen.net
> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
> 
> I still see some errors in TSX-code I can't recall seing in the old mode 
> (and I might report those issues, once I have time to collect more 
> accurate data on them), but for now I definitely think what we have here 
> warrants marking this particular bug as closed. :)

Closing.

Btw, you can always close yourself, by sending to the NNN-done address, as I
did here.