On Thu, Aug 24, 2017, at 12:53 PM, Stefan Monnier wrote:
> Indeed this problem has been with us in sh-script.el for many years.
> There are several different aspects at play:
> - how should font-lock display `name`.  Should it have a normal
>   face as>   if it weren't inside a string?  Or should have a kind of combination>   of string-face and something else?

I'd vote for "plain" face, and name being styled as it would have been
outside the string-context. If a mode-author wants to fancy it up, we
should definitely have a way to easily accommodate that though.
Having a separate expression-in-string-or-comment face is probably the
best approach in that regard.
> - how should we handle nestings of such constructs.

That's a good question, but I think a even better question would be to
ask what that would look like, and *if* we should strive to support it
in the first place.
The most contrived (again JS) example I can think of would be inline code-
references in a comment in a expression in a string. Something like:
    let greeting = "Hello, ${ /** @name string here refers to  the
    employer, not the employee! */ employee_name }";
If we were to support these kind of endless constructs (which I'm sure
someone will find a way to make), we would need to consider this a level
of nesting, just like we do with parens and curly braces and what not.
> - several parts of Emacs like to know if we're inside
>   code/string/comment (usually using (nth [3|4|8] (syntax-ppss))).
>   Should these all consider "name" as being inside code?
>   Or inside string?  Or should we revisit all those cases one-by-one?
It's hard to come up with a blanket solution for this, just like that. I
definitely recognize the problem though.
In increasing the expressiveness of the syntax-construction, it's
generally hard to say "all new occurrences of this new construct map
perfectly to this old understanding of the same property" (as in the
Liskov Substitution Principle).
If a change means revisiting cases one by one, the Emacs ecosystem
will be in a state of mess for years to come, so that's clearly not
an option.
How about instead extending (syntax-ppss) to keep all existing
behaviour for all existing properties, but include some new ones
with more information? Then mode-authors which wants to exploit
these new properties can probe for those in a reasonable backwards-
compatible manner?
Ideally these extra properties should contain at least the following:

* is in escape string or comment
* start of this-level string/comment escape-sequence (nth 8 contains the
  start of the string/comment)* level of string/comment-expression nesting

As far as I can see, that should about cover it.

> In sh-script, we use syntax-propertize to make sure the whole
> string and> its contents is all considered as a string and nothing else.  This was> the simplest solution and probably corresponds to the current behavior> of js.el (and typescript.el) as well.
> 

While obviously the simplest, it's syntactically *inaccurate *and this
inaccuracy is causing issues in other packages who rely on accurate syntax-
information.
It would be really nice if we could have our cake and eat it too.

--
Jostein Kjønigsen

jostein@kjonigsen.net 🍵 jostein@gmail.com
https://jostein.kjonigsen.net