From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: =?utf-8?Q?Jostein=20Kj=C3=B8nigsen?= Newsgroups: gmane.emacs.devel Subject: Re: Regarding Emacs, js.el, template-strings and syntax-tables Date: Thu, 24 Aug 2017 14:13:38 +0200 Message-ID: <1503576818.100758.1083575560.7079FD9A@webmail.messagingengine.com> References: <1503557767.41308.1083341824.4A2103C1@webmail.messagingengine.com> Reply-To: jostein@kjonigsen.net NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="_----------=_15035768181007580" Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1503576985 7055 195.159.176.226 (24 Aug 2017 12:16:25 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 24 Aug 2017 12:16:25 +0000 (UTC) Cc: Stefan Monnier To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Aug 24 14:16:18 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dkr3W-0001Af-Dc for ged-emacs-devel@m.gmane.org; Thu, 24 Aug 2017 14:16:14 +0200 Original-Received: from localhost ([::1]:48317 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dkr3b-0005v5-A3 for ged-emacs-devel@m.gmane.org; Thu, 24 Aug 2017 08:16:19 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51914) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dkr16-0004FO-CF for emacs-devel@gnu.org; Thu, 24 Aug 2017 08:13:45 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dkr11-000783-Qo for emacs-devel@gnu.org; Thu, 24 Aug 2017 08:13:44 -0400 Original-Received: from out2-smtp.messagingengine.com ([66.111.4.26]:60501) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dkr11-00077d-9I for emacs-devel@gnu.org; Thu, 24 Aug 2017 08:13:39 -0400 Original-Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 816F320C6B; Thu, 24 Aug 2017 08:13:38 -0400 (EDT) Original-Received: from web5 ([10.202.2.215]) by compute6.internal (MEProxy); Thu, 24 Aug 2017 08:13:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= secure.kjonigsen.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:reply-to:subject:to:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=rrIJJxtVjOGAsArWdut4ex+8gwpym4fYQwtltMnTJ +A=; b=1K6TVCQPV3DownZeWfwpWXSK+0+zQyVDoZ6JcoQd6P8A/Q9vD4hAktwby bLIKPmoCa/gSNLI6xdgB2uC0IuzqKCmL7vXjZUqtqrtRAbAi3otrMxfc0JQIRqDR TJTNyFheLdDkIjwTaMAXvt21xEe7ffo+IOmLh6bZOl05ULwMH5PsJZ396XH42mh2 7YbMs55g3l3QQkerV3wREDLa9bY54BQVYd66zqek74PimLxfpeCu0wo4dEYFeiUZ pUAnpuRuDAt5CDV5rfxWwYb47fGQY/lC/d7EE2VbFPSNe0NlglNNEvKq0Gl8iRWo g0pEgtK0ea2puYZyqgx0ItabWV/ow== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :reply-to:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=rrIJJxtVjOGAsArWdut4ex+8gwpym4fYQwtltMnTJ+A=; b=kkLciESsVdQt 3hMRMkEv++Zjg+UuNn+ioz2HV8HPy1km2qmLNSTqT6FBrcmaoSLB9hc13A5ICTEU 3D5Jn9NqroboMp2eYVj6yUAmnHbpqPllTg/0VcNr22PHFOQTep+9+BrrD37wbt1x WoMVV3Srejfy/orjznpnQFRTtBjBGtJNh26KA578LBQrneOaMM5YmhDQHRmaGC+x P66fO5QQJp3KlpEV9M5QUu4ujorYxbOYZYCxJGCZD8dJv9gp/fTl+3ezlYfaRMFN bMfJxmOy4n8Ozo/+gEI6imPwpscTQxliPzp4XDfH0L9dLK6PMHZJ9+0xU7CfXDR1 8pF6XcHq+A== X-ME-Sender: Original-Received: by mailuser.nyi.internal (Postfix, from userid 99) id 52E009E2E8; Thu, 24 Aug 2017 08:13:38 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface - ajax-13b5a8c9 In-Reply-To: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.111.4.26 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:217757 Archived-At: This is a multi-part message in MIME format. --_----------=_15035768181007580 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" On Thu, Aug 24, 2017, at 12:53 PM, Stefan Monnier wrote: > Indeed this problem has been with us in sh-script.el for many years. > There are several different aspects at play: > - how should font-lock display `name`. Should it have a normal > face as> if it weren't inside a string? Or should have a kind of com= bination> of string-face and something else? I'd vote for "plain" face, and name being styled as it would have been outside the string-context. If a mode-author wants to fancy it up, we should definitely have a way to easily accommodate that though. Having a separate expression-in-string-or-comment face is probably the best approach in that regard. > - how should we handle nestings of such constructs. That's a good question, but I think a even better question would be to ask what that would look like, and *if* we should strive to support it in the first place. The most contrived (again JS) example I can think of would be inline code- references in a comment in a expression in a string. Something like: let greeting =3D "Hello, ${ /** @name string here refers to the employer, not the employee! */ employee_name }"; If we were to support these kind of endless constructs (which I'm sure someone will find a way to make), we would need to consider this a level of nesting, just like we do with parens and curly braces and what not. > - several parts of Emacs like to know if we're inside > code/string/comment (usually using (nth [3|4|8] (syntax-ppss))). > Should these all consider "name" as being inside code? > Or inside string? Or should we revisit all those cases one-by-one? It's hard to come up with a blanket solution for this, just like that. I definitely recognize the problem though. In increasing the expressiveness of the syntax-construction, it's generally hard to say "all new occurrences of this new construct map perfectly to this old understanding of the same property" (as in the Liskov Substitution Principle). If a change means revisiting cases one by one, the Emacs ecosystem will be in a state of mess for years to come, so that's clearly not an option. How about instead extending (syntax-ppss) to keep all existing behaviour for all existing properties, but include some new ones with more information? Then mode-authors which wants to exploit these new properties can probe for those in a reasonable backwards- compatible manner? Ideally these extra properties should contain at least the following: * is in escape string or comment * start of this-level string/comment escape-sequence (nth 8 contains the start of the string/comment)* level of string/comment-expression nesting As far as I can see, that should about cover it. > In sh-script, we use syntax-propertize to make sure the whole > string and> its contents is all considered as a string and nothing else. = This was> the simplest solution and probably corresponds to the current be= havior> of js.el (and typescript.el) as well. >=20 While obviously the simplest, it's syntactically *inaccurate *and this inaccuracy is causing issues in other packages who rely on accurate syntax- information. It would be really nice if we could have our cake and eat it too. -- Jostein Kj=C3=B8nigsen jostein@kjonigsen.net =F0=9F=8D=B5 jostein@gmail.com https://jostein.kjonigsen.net --_----------=_15035768181007580 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8"
On Thu, Aug 24, 2017, at 12:53 PM, Stefan Monnier wrote:
Indeed this problem has been with us in sh-script.el for m= any years.
There are several different aspects at play:
- how should font-lock display `name`.  Should it have a normal f= ace as
  if it weren't inside a string?  Or should have a kind of c= ombination
  of string-face and something else?

I'd vote for "plain" face, and name being styled as it would have been= outside the string-context. If a mode-author wants to fancy it up, we shou= ld definitely have a way to easily accommodate that though.

Having a separate expression-in-string-or-comment face is probably the= best approach in that regard.

- how should we handle nestings of such= constructs.

That's a good question, but I think a even better question would be to= ask what that would look like, and if we should strive to support i= t in the first place.

The most contrived (again JS) example I can think of would be inline c= ode-references in a comment in a expression in a string. Something like:

    let greeting =3D "Hello, ${ /** @name string here r= efers to  the employer, not the employee! */ employee_name }";

If we were to support these kind of endless constructs (which I'm sure= someone will find a way to make), we would need to consider this a level o= f nesting, just like we do with parens and curly braces and what not.

- several parts of Emacs like to know i= f we're inside
  code/string/comment (usually using (nth [3|4|8] (syntax-ppss)))= .
  Should these all consider "name" as being inside code?
  Or inside string?  Or should we revisit all those cases on= e-by-one?

It's hard to come up with a blanket solution for this, just like that.= I definitely recognize the problem though.

In increasing the expressiveness of the syntax-construction, it's gene= rally hard to say "all new occurrences of this new construct map perfectly = to this old understanding of the same property" (as in the Liskov Substitut= ion Principle).

If a change means revisiting cases one by one, the Emacs ecosystem wil= l be in a state of mess for years to come, so that's clearly not an option.=

How about instead extending (syntax-ppss) to keep all existing behavio= ur for all existing properties, but include some new ones with more informa= tion? Then mode-authors which wants to exploit these new properties can pro= be for those in a reasonable backwards-compatible manner?

Ideally these extra properties should contain at least the following:<= br>

* is in escape string or comment
* start of this-level string/comment escape-sequence (nth 8 contains t= he start of the string/comment)
* level of string/comment-expression nesting

As far as I can see, that should about cover it.

In sh-script, we use syntax-proper= tize to make sure the whole string and
its contents is all considered as a string and nothing else.  Thi= s was
the simplest solution and probably corresponds to the current behavior=
of js.el (and typescript.el) as well.


While obviously the simplest, it's syntactically <= i>inaccurate and this inaccuracy is causing issues in other packages wh= o rely on accurate syntax-information.

It would be really nice if we could have our cake = and eat it too.

--
Jostein Kj=C3=B8nigsen

jostein@k= jonigsen.net =F0=9F=8D=B5 jostein@= gmail.com
https://= jostein.kjonigsen.net



--_----------=_15035768181007580--