From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Jostein_Kj=C3=B8nigsen?= Newsgroups: gmane.emacs.devel Subject: Re: Comprehensive JSX support in Emacs Date: Sat, 16 Feb 2019 15:50:27 -0500 Message-ID: <4f6a7f78-caaf-4d1d-bf7a-d8415141efe6@www.fastmail.com> References: <1423022755.65233.1550120813763@privateemail.com> Reply-To: jostein@kjonigsen.net Content-Type: multipart/alternative; boundary=5c6381ca0d494bfe84eedb62a26d2379 Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="252260"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Cyrus-JMAP/3.1.5-832-gba113d7-fmstable-20190201v1 Cc: emacs-devel@gnu.org To: "Jackson Ray Hamilton" Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Feb 16 21:51:31 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gv6vo-0013RJ-PC for ged-emacs-devel@m.gmane.org; Sat, 16 Feb 2019 21:51:30 +0100 Original-Received: from localhost ([127.0.0.1]:60877 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gv6vn-0006tY-NG for ged-emacs-devel@m.gmane.org; Sat, 16 Feb 2019 15:51:27 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:60471) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gv6v1-0006sz-TU for emacs-devel@gnu.org; Sat, 16 Feb 2019 15:50:42 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gv6ux-000661-3K for emacs-devel@gnu.org; Sat, 16 Feb 2019 15:50:39 -0500 Original-Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:35353) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gv6uv-00064x-1O for emacs-devel@gnu.org; Sat, 16 Feb 2019 15:50:34 -0500 Original-Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 2E4A03186; Sat, 16 Feb 2019 15:50:29 -0500 (EST) Original-Received: from imap1 ([10.202.2.51]) by compute6.internal (MEProxy); Sat, 16 Feb 2019 15:50:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= secure.kjonigsen.net; h=message-id:in-reply-to:references:date :from:reply-to:to:cc:subject:content-type; s=fm2; bh=ecnPMp1uMYg zdTTfABrE0ztIKlUY1guCwu/6hvSFQY4=; b=BG3c3byRC4+MV2RVvST7UlvG+oe d1q8on2LxtRG5tsNhjglDAhe0LjrmRvIcNZkw0I7IlPB4Dhv0HknWUq/n1xreZK0 dsnl95xhBu9Lq4nCZgeUiCGXbqj6XNptsd3HW4pYTndGi/i1SMheCryA7TeDQrGE D853RLZ61xNXQLSlnAERayFShhE3JrFachW+F8p3XcLQUSj2I36s9KefPQbaAHTR cKKJHNGdqk2XWbMxepxCa4WhjYQwiMz1ZE7xoP1Nrf4I/9RUXYGRIcJXdR2iRW58 r2QY7ekBGq1nqs075ximvQjRZWUp9X/9/2kDN9RDdB84cU+mmlpxtI+Es6A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:references:reply-to:subject:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=ecnPMp1uMYgzdTTfA BrE0ztIKlUY1guCwu/6hvSFQY4=; b=oNAny5hFzsbm+K8pKGuUrSJOpb3ZjXh/b Ov8teoQX541A1B8E9KJCiXAWg73X3EO75vx+An1blMVCnYJn+2TOV/6fd5PA9RiR p7A9zwlYXCU2lipQaMeZ13U4j7DkLDzhSVP4XVRWzvwqiTuWZLMuffE1/ZvHuyCV vq+Bt0pGornVTzWu3BWk2A6RJaZ+4RCwuikKVUu1VYjzqROTdIeiMusP+0oQSeEO lAtcsTOb54CNVR6b+DDJW3xDTlk6ZGD9Y7q1U8NAyuJQ/IAI8Q43ZUx4jqg2cVyj BWivmWAtmpHTvRm7B1RR5bIW0vZgZRtXdi96H2Da6Q7wVAwuRkeeA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtledruddtledgudeghecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu fedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnegoufhushhpvggtthffoh hmrghinhculdegledmnecujfgurhepofgfkfgjfhffhfhrvffutgesrgdtreerreerjeen ucfhrhhomheplfhoshhtvghinhgpmfhjpphnihhgshgvnhcuoehjohhsthgvihhnsehsvg gtuhhrvgdrkhhjohhnihhgshgvnhdrnhgvtheqnecuffhomhgrihhnpehnihhgshgvnhdr nhhopdgslhhoghhsphhothdrtghomhdpghhithhhuhgsrdgtohhmnecurfgrrhgrmhepmh grihhlfhhrohhmpehjohhsthgvihhnsehsvggtuhhrvgdrkhhjohhnihhgshgvnhdrnhgv thenucevlhhushhtvghrufhiiigvpedt X-ME-Proxy: Original-Received: by mailuser.nyi.internal (Postfix, from userid 501) id 4D86FD43B7; Sat, 16 Feb 2019 15:50:28 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface X-Me-Personality: 19302177 In-Reply-To: <1423022755.65233.1550120813763@privateemail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 64.147.123.25 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:233421 Archived-At: --5c6381ca0d494bfe84eedb62a26d2379 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable Hey there Jackson! This sounds *great*! As a maintainer of (third-party) Typescript-mode, I just wanted to shout= out that these days React is often not just written in Javascript, but = also in TypeScript. With TypeScript you can get much better in-editor co= de-intelligence than you can with regular Javascript, and support for th= is is provided by (third-party) package "tide" which I also co-maintain.= For both these packages a good JSX/TSX-configuration and good JSX/TSX-su= pport is often requested. If you go through with this endeavor, it would be nice if it could be do= ne in a way which meant the TypeScript-world would have to fork/clone yo= ur efforts, but could rather build on it or extend it. I'm not exactly sure *how* that could best be done, but I just thought I= 'd bring this particular angle to your attention in case it wasn't alrea= dy on your radar. -- Kind Regards Jostein Kj=C3=B8nigsen jostein@kjonigsen.net =F0=9F=8D=B5 jostein@gmail.com https://jostein.kj=C3=B8nigsen.no = On Thu, Feb 14, 2019, at 6:08 AM, Jackson Ray Hamilton wrote: > Hello Emacs maintainers, > A few years ago, I took up the challenge of implementing indentation s= upport for the JavaScript extension =E2=80=9CJSX=E2=80=9D (by Facebook, = for their =E2=80=9CReact=E2=80=9D UI library), and we got my patches mer= ged into js-mode. Now I think it=E2=80=99s time we reexamined and began = to improve Emacs=E2=80=99 JSX+React support. > My initial implementation was not too great, because it failed to inde= nt JSX code properly in many common scenarios. My hueristics for finding= HTML-like code inside a JS buffer were too conservative, due to a (perh= aps excessive/misguided) attention to performance in particular areas of= the code. Since my patch=E2=80=99s release, I=E2=80=99ve been watching = from a distance as the bug reports have piled up. > A package called =E2=80=9Crjsx-mode=E2=80=9D came onto the scene not t= oo later, which seems to solve indentation problems using an AST (buildi= ng on js2-mode=E2=80=99s AST); however, according to Steve Yegge=E2=80=99= s post-mortem on js2-mode (http://steve-yegge.blogspot.com/2008/03/js2-m= ode-new-javascript-mode-for-emacs.html), relying on an AST being availab= le after every keystroke is probably not optimal for performance, especi= ally with large files. rjsx-mode also performs syntax highlighting (agai= n, using its AST), which js-mode still does not do at all for JSX. > Recently, I finally resolved to =E2=80=9Cdo my laundry,=E2=80=9D with = respect to addressing the JSX indentation issues accumulated within debb= ugs and the js2-mode GitHub repo (where lots of js-mode bugs accidentall= y get filed). In the past week, I assembled some test cases, drafted som= e algorithms, and proceeded to implement a number of improvements to the= indentation logic. > As I started playing with the code again, many new ideas for improving= JSX+React support began popping into my head, beyond just indentation i= mprovements. I wanted to share those ideas here, and see if anyone wants= to thumbs-up them before I go ahead and implement them (perhaps avoidin= g some bad ideas and/or rework). > Also, I wanted to share my work-in-progress indentation reimplementati= on, and determine if I am not going completely in the wrong direction, a= nd if an alternate plan I=E2=80=99ll also present for indentation is at = all viable in comparison. > Proposition: Comprehensive JSX support in Emacs > First: Consider reworking obtuse APIs > JSX indentation support is currently enabled by activating =E2=80=9Cjs= -jsx-mode=E2=80=9D, using a function called =E2=80=9Cjs-jsx-indent-line=E2= =80=9D, and =E2=80=9Csgml-=E2=80=9D variables control the indentation of= the JSX code. > I think introducing JSX support in the form of a derivative mode may h= ave been a mistake. It wasn=E2=80=99t initially obvious to some people t= o use this mode. There was not an auto-mode-alist item for it, either. I= t also had the effect of creating a =E2=80=9Cmatrix=E2=80=9D of modes, w= here, for instance, Flycheck has to support all the different JavaScript= modes like this: > ` :modes (js-mode js-jsx-mode js2-mode js2-jsx-mode js3-mode rjsx-mode= )` > which seems messy to me. > And what if we want to support more JavaScript syntax extensions, like= Facebook=E2=80=99s =E2=80=9CFlow,=E2=80=9D and support the use of multi= ple extensions at once? We wouldn=E2=80=99t want to also have a js-flow-= mode, js-jsx-flow-mode, js2-flow-mode, js2-jsx-flow-mode=E2=80=A6 theref= ore, I think it=E2=80=99d be better to start scaling back, and instead e= nable JSX, Flow, et al support with a variable. > Also, upon reflection, I=E2=80=99m becoming more certain that controll= ing JavaScript indentation (even HTML-like indentation) with sgml- varia= bles was unintuitive and therefore also a mistake. I think JSX should si= mply be indented using js-indent-level, perhaps optionally in combinatio= n with a js-jsx-attribute-offset, to complement the sgml-attribute-offse= t that we would otherwise be eliminating as an option. However, having a= dded sgml-attribute-offset to Emacs myself merely as a precaution after = a minor breaking change I made to SGML indentation a few years ago, I th= ink we should let users actually demand JSX attribute indentation deltas= , rather than assume they desire that level of granularity (considering = we went so long without it in sgml-mode). Rather, users have indicated i= n bug reports that they=E2=80=99d prefer better *defaults* for the inden= tation (assuming indentation not matching =E2=80=9Cofficial=E2=80=9D con= ventions is simply =E2=80=9Cerroneous;=E2=80=9D and what a blessing to h= ave conventions). > I think JSX shoud =E2=80=9Cjust work=E2=80=9D in Emacs; i.e., with js-= mode, whenever a user opens a file using JSX syntax, adapting to the use= r=E2=80=99s conventions for JS. > Deprecation: Guide users to new, nicer APIs > * Deprecate js-jsx-mode (and js2-jsx-mode downstream, and have rjsx-m= ode derive from js2-mode). Both modes will still exist, but will be mark= ed obsolete, and will simply make a file-local copy of js-syntax-extensi= ons (see below) with 'jsx added to the front of the list, instead of bin= ding indentation like they currently do. > * Deprecate js-jsx-indent-line, marking it obsolete. It will still ex= ist, but it will simply call through to js-indent-line, with a copy of j= s-syntax-extensions let-bound, with 'jsx added to the front of the list.= > New features: > * js-syntax-extensions: A list that will include symbols representing= syntax extensions for which we will enable indentation/fontification. > * In theory, if we supported more syntax extensions, those would go= here as well; also, if any of the extensions happened to conflict with = each other in some ways but not others, we could use the order of the li= st to determine the =E2=80=9Cprecedence=E2=80=9D of the extensions. > * Having one variable for this gives us an opportunity to document = all the supported syntaxes in one place, so more people will discover wh= at=E2=80=99s all available. > * Add a Customize interface for adding to this list like a set, pic= king from a set of options. (Not sure which widget would be best to use = for that.) > * js-indent-line will switch to a JSX indentation strategy when calle= d while js-syntax-extensions contains 'jsx. > * js-mode will automatically detect whether a file uses JSX syntax us= ing some heuristic. Suggestions: > * 'jsx is in js-syntax-extensions via init file, mode hook, or in .= dir-locals.el > * buffer-file-name has extension =E2=80=9C.jsx=E2=80=9D (some peopl= e use this) > * =E2=80=9C^\(var\|let\|const\|import\) React=E2=80=9D matches with= in the first 1000 lines after any keystroke. (Under typical compiler con= figuration, =E2=80=9CReact=E2=80=9D must be a JavaScript variable in sco= pe in order for the compiled JSX to function properly, and most people i= mport or require() stuff at the beginning of their files. React is usual= ly used in combination with a JavaScript compiler, implying the user pro= bably isn=E2=80=99t using React as a global variable, because compilers = make that practice obsolete, so I expect this check will be pretty relia= ble) > * Add auto-mode-alist item mapping =E2=80=9C.jsx=E2=80=9D files to js= -mode, enabling the buffer-file-name heuristic. > * As syntax extensions are enabled/disabled, update the mode name; ex= : > * No syntax extensions: "JavaScript" > * js-syntax-extensions =3D (jsx): "JavaScript[JSX]" > * js-syntax-extensions =3D (flow jsx): "JavaScript[Flow,JSX]" > * Fill in current gaps in indentation support for JSX. > * Add font locking for JSX syntax, similar if not the same as sgml-mo= de=E2=80=99s font locking. It is enabled when js-syntax-extensions conta= ins 'jsx. > Would appreciate feedback on these feature propositions; thanks! > Indentation: WIP > As I mentioned, I=E2=80=99ve already started hacking on the indentatio= n. WIP patches are attached for preliminary review of approach (please d= on=E2=80=99t merge these yet, I haven=E2=80=99t vetted them thoroughly o= utside make test cases). > I started by reworking my =E2=80=9CJSX detection=E2=80=9D code to use = sgml-get-context to figure out if we were inside JSX, which was a lot mo= re reliable than the previous heuristic (JSX pretty much always had to b= e wrapped in parens). I then proceeded to mostly solve Bug#24896 by dist= inguishing =E2=80=9C<=E2=80=9D and =E2=80=9C>=E2=80=9D in JS and JSX, so= the sgml-mode indentation code could parse the code without tripping ov= er arrow functions (=E2=80=9C=3D>=E2=80=9D), thinking they were part of = HTML =E2=80=9C=E2=80=9D. I disambiguated the symbols in a synt= ax-propertize-function by doing a quick parse near every =E2=80=9C<=E2=80= =9D and =E2=80=9C>=E2=80=9D char to determine if the surrounding code co= uld only be syntactically valid as JSX, and not as JS. This seems to hav= e fixed a lot of the failing cases. However, it still fails in a weird c= ase where JSX appears nested inside JS, where that JS is also nested ins= ide an outer JSX, a situation that just isn=E2=80=99t relevant in SGML, = and thus the indenter doesn=E2=80=99t seem to support it. > `// JS expressions should not break indentation` > `// (https://github.com/mooz/js2-mode/issues/462).` > `return (` > ` ` > ` ` > ` (` > `
nothing
` > ` )} />` > ` ` > ` ` > ` ` > `)` > Maybe I could fix this recursive indentation issue by parsing the JSX = further, and marking any =E2=80=9C{=E2=80=9D, =E2=80=9C}=E2=80=9D pairs = therein with some syntax property that would get sgml-indent-line to tre= at it like some nested thing. However, that also might *not* work, and I= don=E2=80=99t think I want to jump through any more hoops to make sgml-= indent-line work inside js-mode. > Also, since I=E2=80=99ve already started parsing JSX to disambiguate i= t from JS, I figure I may as well finish the parse (it=E2=80=99s pretty = simple), since I could use that parse as an opportunity to mark characte= rs for JSX-contextual font locking. And, since I=E2=80=99m adding these = markings to buffer positions, I now have a great new way to figure out i= f I=E2=80=99m inside JSX when I=E2=80=99m indenting. So I=E2=80=99m thin= king it may be time to graduate from sgml-indent-line delegation, and us= e the new data we now have to implement the JSX indentation logic in its= entirety in js-mode. This way we could definitely support recursive JSX= indentation, and also align all the indentation conventions with those = used by the React community, and potentially maximize performance by avo= iding a lot of checks that poor sgml-mode has to do for HTML=E2=80=99s l= oose and more complex semantics. > Please let me know what you think of my current patches and the direct= ion they=E2=80=99re going in, and if porting a simplified sgml-indent-li= ne to js-mode is not too crazy of an idea! > Thanks, Jackson >=20 > *Attachments:* > * 0001-Add-failing-tests-for-JSX-indentation-bugs.patch > * 0002-Refactor-JSX-indentation-code-to-improve-enclosing-J.patch > * 0003-Add-new-failing-unclosed-JSX-test-and-separate-such-.patch > * 0004-js-syntax-propertize-Disambiguate-JS-from-JSX-fixing.patch --5c6381ca0d494bfe84eedb62a26d2379 Content-Type: text/html;charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hey there Jacks= on!

This sounds great!

As a maintainer of (third-party) Typescript-mode, I just= wanted to shout out that these days React is often not just written in = Javascript, but also in TypeScript. With TypeScript you can get much bet= ter in-editor code-intelligence than you can with regular Javascript, an= d support for this is provided by (third-party) package "tide" which I a= lso co-maintain.

For both these packages a = good JSX/TSX-configuration and good JSX/TSX-support is often requested.<= /div>

If you go through with this endeavor, it would = be nice if it could be done in a way which meant the TypeScript-world wo= uld have to fork/clone your efforts, but could rather build on it or ext= end it.

I'm not exactly sure how tha= t could best be done, but I just thought I'd bring this particular angle= to your attention in case it wasn't already on your radar.

--
Ki= nd Regards
Jostein Kj=C3=B8nigsen
<= /div>

<= a href=3D"mailto:jostein@kjonigsen.net">jostein@kjonigsen.net =F0=9F= =8D=B5 jostein@gmail.com
https://jostein.kj=C3=B8nigsen.no


On Thu, Feb 14, 2019, at 6:08 AM, Jackson Ray Hamilton= wrote:

Hel= lo Emacs maintainers,

A few years ago, I took up the challenge= of implementing indentation support for the JavaScript extension =E2=80= =9CJSX=E2=80=9D (by Facebook, for their =E2=80=9CReact=E2=80=9D UI libra= ry), and we got my patches merged into js-mode.  Now I think it=E2=80= =99s time we reexamined and began to improve Emacs=E2=80=99 JSX+React su= pport.

My initial implementation was not too great, because it= failed to indent JSX code properly in many common scenarios.  My h= ueristics for finding HTML-like code inside a JS buffer were too conserv= ative, due to a (perhaps excessive/misguided) attention to performance i= n particular areas of the code.  Since my patch=E2=80=99s release, = I=E2=80=99ve been watching from a distance as the bug reports have piled= up.

A package called =E2=80=9Crjsx-mode=E2=80=9D came onto th= e scene not too later, which seems to solve indentation problems using a= n AST (building on js2-mode=E2=80=99s AST); however, according to Steve = Yegge=E2=80=99s post-mortem on js2-mode (http://= steve-yegge.blogspot.com/2008/03/js2-mode-new-javascript-mode-for-emacs.= html), relying on an AST being available after every keystroke is pr= obably not optimal for performance, especially with large files.  r= jsx-mode also performs syntax highlighting (again, using its AST), which= js-mode still does not do at all for JSX.

Recently, I finally= resolved to =E2=80=9Cdo my laundry,=E2=80=9D with respect to addressing= the JSX indentation issues accumulated within debbugs and the js2-mode = GitHub repo (where lots of js-mode bugs accidentally get filed).  I= n the past week, I assembled some test cases, drafted some algorithms, a= nd proceeded to implement a number of improvements to the indentation lo= gic.

As I started playing with the code again, many new ideas = for improving JSX+React support began popping into my head, beyond just = indentation improvements.  I wanted to share those ideas here, and = see if anyone wants to thumbs-up them before I go ahead and implement th= em (perhaps avoiding some bad ideas and/or rework).

Also, I wa= nted to share my work-in-progress indentation reimplementation, and dete= rmine if I am not going completely in the wrong direction, and if an alt= ernate plan I=E2=80=99ll also present for indentation is at all viable i= n comparison.

Proposition: Comprehensive JSX support in Emacs=

First: Consider reworking obtuse APIs

JSX indent= ation support is currently enabled by activating =E2=80=9Cjs-jsx-mode=E2= =80=9D, using a function called =E2=80=9Cjs-jsx-indent-line=E2=80=9D, an= d =E2=80=9Csgml-=E2=80=9D variables control the indentation of the JSX c= ode.

I think introducing JSX support in the form of a derivati= ve mode may have been a mistake.  It wasn=E2=80=99t initially obvio= us to some people to use this mode.  There was not an auto-mode-ali= st item for it, either.  It also had the effect of creating a =E2=80= =9Cmatrix=E2=80=9D of modes, where, for instance, Flycheck has to suppor= t all the different JavaScript modes like this:

 &n= bsp;  :modes (js-mode js-jsx-mode js2-mode js2-jsx-mode js3-mode rj= sx-mode)

which seems messy to me.

And what if= we want to support more JavaScript syntax extensions, like Facebook=E2=80= =99s =E2=80=9CFlow,=E2=80=9D and support the use of multiple extensions = at once?  We wouldn=E2=80=99t want to also have a js-flow-mode, js-= jsx-flow-mode, js2-flow-mode, js2-jsx-flow-mode=E2=80=A6 therefore, I th= ink it=E2=80=99d be better to start scaling back, and instead enable JSX= , Flow, et al support with a variable.

Also, upon reflection, = I=E2=80=99m becoming more certain that controlling JavaScript indentatio= n (even HTML-like indentation) with sgml- variables was unintuitive and = therefore also a mistake.  I think JSX should simply be indented us= ing js-indent-level, perhaps optionally in combination with a js-jsx-att= ribute-offset, to complement the sgml-attribute-offset that we would oth= erwise be eliminating as an option.  However, having added sgml-att= ribute-offset to Emacs myself merely as a precaution after a minor break= ing change I made to SGML indentation a few years ago, I think we should= let users actually demand JSX attribute indentation deltas, rather than= assume they desire that level of granularity (considering we went so lo= ng without it in sgml-mode).  Rather, users have indicated in bug r= eports that they=E2=80=99d prefer better defaults for the indenta= tion (assuming indentation not matching =E2=80=9Cofficial=E2=80=9D conve= ntions is simply =E2=80=9Cerroneous;=E2=80=9D and what a blessing to hav= e conventions).

I think JSX shoud =E2=80=9Cjust work=E2=80=9D = in Emacs; i.e., with js-mode, whenever a user opens a file using JSX syn= tax, adapting to the user=E2=80=99s conventions for JS.

Depre= cation: Guide users to new, nicer APIs

  • Deprecate js-jsx-= mode (and js2-jsx-mode downstream, and have rjsx-mode derive from js2-mo= de).  Both modes will still exist, but will be marked obsolete, and= will simply make a file-local copy of js-syntax-extensions (see below) = with 'jsx added to the front of the list, instead of binding indentation= like they currently do.
  • Deprecate js-jsx-indent-line, marki= ng it obsolete.  It will still exist, but it will simply call throu= gh to js-indent-line, with a copy of js-syntax-extensions let-bound, wit= h 'jsx added to the front of the list.

New features:

  • js-syntax-extensions: A list that will include symbol= s representing syntax extensions for which we will enable indentation/fo= ntification.
    • In theory, if we supported more syntax ext= ensions, those would go here as well; also, if any of the extensions hap= pened to conflict with each other in some ways but not others, we could = use the order of the list to determine the =E2=80=9Cprecedence=E2=80=9D = of the extensions.
    • Having one variable for this gives us an = opportunity to document all the supported syntaxes in one place, so more= people will discover what=E2=80=99s all available.
    • Add a Cu= stomize interface for adding to this list like a set, picking from a set= of options.  (Not sure which widget would be best to use for that.= )
  • js-indent-line will switch to a JSX indentation = strategy when called while js-syntax-extensions contains 'jsx.
    • <= li>
    js-mode will automatically detect whether a file uses JSX syntax= using some heuristic.  Suggestions:
    • 'jsx is in js= -syntax-extensions via init file, mode hook, or in .dir-locals.el
    • buffer-file-name has extension =E2=80=9C.jsx=E2=80=9D (some people= use this)
    • =E2=80=9C^\(var\|let\|const\|import\) React=E2=80= =9D matches within the first 1000 lines after any keystroke.  (Unde= r typical compiler configuration, =E2=80=9CReact=E2=80=9D must be a Java= Script variable in scope in order for the compiled JSX to function prope= rly, and most people import or require() stuff at the beginning of their= files.  React is usually used in combination with a JavaScript com= piler, implying the user probably isn=E2=80=99t using React as a global = variable, because compilers make that practice obsolete, so I expect thi= s check will be pretty reliable)
  • Add auto-mode-ali= st item mapping =E2=80=9C.jsx=E2=80=9D files to js-mode, enabling the bu= ffer-file-name heuristic.
  • As syntax extensions are enab= led/disabled, update the mode name; ex:
    • No syntax exten= sions: "JavaScript"
    • js-syntax-extensions =3D (jsx): "JavaScr= ipt[JSX]"
    • js-syntax-extensions =3D (flow jsx): "JavaScript[F= low,JSX]"
  • Fill in current gaps in indentation supp= ort for JSX.
  • Add font locking for JSX syntax, similar if not= the same as sgml-mode=E2=80=99s font locking.  It is enabled when = js-syntax-extensions contains 'jsx.

Would appreciate fee= dback on these feature propositions; thanks!

Indentation: WIP=

As I mentioned, I=E2=80=99ve already started hacking on the = indentation.  WIP patches are attached for preliminary review of ap= proach (please don=E2=80=99t merge these yet, I haven=E2=80=99t vetted t= hem thoroughly outside make test cases).

I started by reworkin= g my =E2=80=9CJSX detection=E2=80=9D code to use sgml-get-context to fig= ure out if we were inside JSX, which was a lot more reliable than the pr= evious heuristic (JSX pretty much always had to be wrapped in parens).&n= bsp; I then proceeded to mostly solve Bug#24896 by distinguishing =E2=80= =9C<=E2=80=9D and =E2=80=9C>=E2=80=9D in JS and JSX, so the sgml-m= ode indentation code could parse the code without tripping over arrow fu= nctions (=E2=80=9C=3D>=E2=80=9D), thinking they were part of HTML =E2= =80=9C<elements>=E2=80=9D.  I disambiguated the symbols in a = syntax-propertize-function by doing a quick parse near every =E2=80=9C&l= t;=E2=80=9D and =E2=80=9C>=E2=80=9D char to determine if the surround= ing code could only be syntactically valid as JSX, and not as JS.  = This seems to have fixed a lot of the failing cases.  However, it s= till fails in a weird case where JSX appears nested inside JS, where tha= t JS is also nested inside an outer JSX, a situation that just isn=E2=80= =99t relevant in SGML, and thus the indenter doesn=E2=80=99t seem to sup= port it.

// JS expressions should not break ind= entation
// (https://github.com/mooz/js2-mode= /issues/462).
return (
<= code>  <Router>
   = <Bar>
      &= lt;Route exact path=3D"/foo" render=3D{() =3D> (
        <div>nothing<= /div>
      )} /&= gt;
      <Route = exact path=3D"/bar" />
    = </Bar>
  </Router>
)

Maybe I could fix this rec= ursive indentation issue by parsing the JSX further, and marking any =E2= =80=9C{=E2=80=9D, =E2=80=9C}=E2=80=9D pairs therein with some syntax pro= perty that would get sgml-indent-line to treat it like some nested thing= .  However, that also might not work, and I don=E2=80=99t th= ink I want to jump through any more hoops to make sgml-indent-line work = inside js-mode.

Also, since I=E2=80=99ve already started parsi= ng JSX to disambiguate it from JS, I figure I may as well finish the par= se (it=E2=80=99s pretty simple), since I could use that parse as an oppo= rtunity to mark characters for JSX-contextual font locking.  And, s= ince I=E2=80=99m adding these markings to buffer positions, I now have a= great new way to figure out if I=E2=80=99m inside JSX when I=E2=80=99m = indenting.  So I=E2=80=99m thinking it may be time to graduate from= sgml-indent-line delegation, and use the new data we now have to implem= ent the JSX indentation logic in its entirety in js-mode.  This way= we could definitely support recursive JSX indentation, and also align a= ll the indentation conventions with those used by the React community, a= nd potentially maximize performance by avoiding a lot of checks that poo= r sgml-mode has to do for HTML=E2=80=99s loose and more complex semantic= s.

Please let me know what you think of my current patches and= the direction they=E2=80=99re going in, and if porting a simplified sgm= l-indent-line to js-mode is not too crazy of an idea!

Thanks, = Jackson


Attachments:
  • = 0001-Add-failing-tests-for-JSX-indentation-bugs.patch
  • 0002-R= efactor-JSX-indentation-code-to-improve-enclosing-J.patch
  • 00= 03-Add-new-failing-unclosed-JSX-test-and-separate-such-.patch
    • 0004-js-syntax-propertize-Disambiguate-JS-from-JSX-fixing.patch

--5c6381ca0d494bfe84eedb62a26d2379--