Should yaml-ts-mode inherit from prog-mode?

Should yaml-ts-mode inherit from prog-mode?

[Romanos Skiadas]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

Hello,

I was using yaml-ts-mode in Emacs 30, and to my surprise I found it
inherits from text-mode.

Now, a case can be made for that for sure. The existing rather widely used
mode uses text-mode, although that seems like an arbitrary decision by a
single maintainer:
https://github.com/yoshiki/yaml-mode/issues/17
https://github.com/yoshiki/yaml-mode/issues/27
https://github.com/yoshiki/yaml-mode/issues/52
https://github.com/yoshiki/yaml-mode/pull/85

(I'm not sure Emacs maintainers are beholden to a decision made some jumps
away and whether this is a strong enough argument to keep yaml-ts-mode as
is.)

However, and here is where things kind of break down a bit: I would
personally argue that YAML has a code feel to it, what with all the bash
scripts that end up in it. Personally, I feel that YAML is more of a prog
language than JSON (whatever that means), and according to the spec it is a
superset of JSON. In fact, the JSON mode shipped with Emacs inherits from
prog-mode!

So, what do people think? Should yaml-ts-mode be changed?

Best,
rski

[-- Attachment #2: Type: text/html, Size: 1808 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Basil Contovounesios]

Romanos Skiadas [2023-02-28 14:24 +0200] wrote:

> I was using yaml-ts-mode in Emacs 30, and to my surprise I found it inherits from text-mode.
>
> Now, a case can be made for that for sure. The existing rather widely used mode uses text-mode, although that seems like an arbitrary
> decision by a single maintainer:
> https://github.com/yoshiki/yaml-mode/issues/17
> https://github.com/yoshiki/yaml-mode/issues/27 
> https://github.com/yoshiki/yaml-mode/issues/52
> https://github.com/yoshiki/yaml-mode/pull/85
>
> (I'm not sure Emacs maintainers are beholden to a decision made some jumps away and whether this is a strong enough argument to
> keep yaml-ts-mode as is.)
>
> However, and here is where things kind of break down a bit: I would personally argue that YAML has a code feel to it, what with all the
> bash scripts that end up in it. Personally, I feel that YAML is more of a prog language than JSON (whatever that means), and according to
> the spec it is a superset of JSON. In fact, the JSON mode shipped with Emacs inherits from prog-mode!
>
> So, what do people think? Should yaml-ts-mode be changed?

[ Copying in Randy, yaml-ts-mode's author/maintainer. ]

FWIW, my vote is for treating YAML and JSON consistently: if all three
JSON modes in Emacs and GNU ELPA are prog-modes, then so should
yaml-ts-mode; or if yaml-ts-mode is a text-mode, then so should all
three JSON modes.

I suspect most of that ship if not the entire thing has already sailed,
especially in the case of the more established json-mode, but there's
always hope ;).

Thanks,

-- 
Basil

Re: Should yaml-ts-mode inherit from prog-mode?

[Randy Taylor]

On Tuesday, February 28th, 2023 at 10:52, Basil Contovounesios <contovob@tcd.ie> wrote:
> Romanos Skiadas [2023-02-28 14:24 +0200] wrote:
>
> > I was using yaml-ts-mode in Emacs 30, and to my surprise I found it inherits from text-mode.
> >
> > Now, a case can be made for that for sure. The existing rather widely used mode uses text-mode, although that seems like an arbitrary
> > decision by a single maintainer:
> > https://github.com/yoshiki/yaml-mode/issues/17
> > https://github.com/yoshiki/yaml-mode/issues/27
> > https://github.com/yoshiki/yaml-mode/issues/52
> > https://github.com/yoshiki/yaml-mode/pull/85
> >
> > (I'm not sure Emacs maintainers are beholden to a decision made some jumps away and whether this is a strong enough argument to
> > keep yaml-ts-mode as is.)
> >
> > However, and here is where things kind of break down a bit: I would personally argue that YAML has a code feel to it, what with all the
> > bash scripts that end up in it. Personally, I feel that YAML is more of a prog language than JSON (whatever that means), and according to
> > the spec it is a superset of JSON. In fact, the JSON mode shipped with Emacs inherits from prog-mode!
> >
> > So, what do people think? Should yaml-ts-mode be changed?
>
>
> [ Copying in Randy, yaml-ts-mode's author/maintainer. ]

Thanks.

>
> FWIW, my vote is for treating YAML and JSON consistently: if all three
> JSON modes in Emacs and GNU ELPA are prog-modes, then so should
> yaml-ts-mode; or if yaml-ts-mode is a text-mode, then so should all
> three JSON modes.
>
> I suspect most of that ship if not the entire thing has already sailed,
> especially in the case of the more established json-mode, but there's
> always hope ;).
>
> Thanks,
>
> --
> Basil

I think the JSON modes are prog-modes just because they are attached to the javascript modes in some way (usually derived from it). I'm not sure how much weight we can assign to certain modes deriving from other modes and saying we should do the same (especially in the case of ELPA/MELPA modes); instead we should ask which mode makes the most sense to derive from. Although there is the argument for consistency that you mentioned...

Deriving from prog-mode feels weird since YAML is not a programming language.

A lot of people (per OP's linked GitHub issues) seem to think/expect it to derive from prog-mode, but I'm not sure how correct those assumptions are. It seems they just want their prog-mode hooks to work with it.

If everyone thinks it should derive from prog-mode and there are no bad consequences for doing so, then I'm not against it.

Do any Emacs maintainers have any thoughts on the matter?

Re: Should yaml-ts-mode inherit from prog-mode?

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 890 bytes --]

On Wed, Mar 1, 2023, 9:10 AM Randy Taylor <dev@rjt.dev> wrote:

>
> I think the JSON modes are prog-modes just because they are attached to
> the javascript modes in some way (usually derived from it). I'm not sure
> how much weight we can assign to certain modes deriving from other modes
> and saying we should do the same (especially in the case of ELPA/MELPA
> modes); instead we should ask which mode makes the most sense to derive
> from. Although there is the argument for consistency that you mentioned...
>

Perhaps since data serialization formats can be used to store data
structures in source files in a portable way, it makes sense to write
generic modes for them that can be adapted to specific PL modes?
I'm sure JSON developed in the opposite direction, but would it have been
better to start with JSON mode then embed it in JavaScript/Python/etc modes
from the start?
Lynn

[-- Attachment #2: Type: text/html, Size: 1450 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 2180 bytes --]

yaml-mode maintainer here. I saw the thread just by chance and found it
amusing that my decision was characterized as arbitrary, when the same
could be said about json-mode being indirectly derived from prog-mode
due to it looking a lot like JavaScript, not due to some deeper decision
making.

> Deriving from prog-mode feels weird since YAML is not a programming
> language.

Yes, that was pretty much my reasoning. To quote the `prog-mode`
docstring: "Major mode for editing programming language source code."

Unless there's some XSLT equivalent for YAML, I do not see it being
programming language source code at all.

> A lot of people (per OP's linked GitHub issues) seem to think/expect
> it to derive from prog-mode, but I'm not sure how correct those
> assumptions are. It seems they just want their prog-mode hooks to work
> with it.

Yes, I think it's a convenience argument really. I believe that
correctness/consistency are more important because such a decision is
long-lasting.

> If everyone thinks it should derive from prog-mode and there are no
> bad consequences for doing so, then I'm not against it.

- If the mode is already established, changing what it derives from
  would make customizations relying on that fail
- If there are any hooks associated with prog-mode that are
  inappropriate for editing YAML, that would be an annoying consequence

More generally speaking, maybe the whole idea of having a hierarchy of
major modes needs to be rethought. It is difficult to get such a
hierarchy right, especially when there are multiple ways to categorize a
major mode. YAML has been argued to be several things (configuration,
serialization, text) and having to choose one in particular cannot work
out in a satisfying way unless it's about a very basic property not
based on opinion (I believe it to *not* be a configuration format at
all, but many people disagree and keep using it despite obvious
deficiencies) or personal preference (YAML is often touted as JSON
superset, but who actually uses that property, some malicious hackers
aside?). Perhaps a keyword system as used by finder.el would make more
sense, plus keyword-specific hooks to be run.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

On Wed, Mar 1, 2023, 11:36 AM Vasilij Schneidermann <mail@vasilij.de> wrote:

>
> > Deriving from prog-mode feels weird since YAML is not a programming
> > language.
>
> Yes, that was pretty much my reasoning. To quote the `prog-mode`
> docstring: "Major mode for editing programming language source code."
>
> Unless there's some XSLT equivalent for YAML, I do not see it being
> programming language source code at all.
>

Perhaps "programming language" is intended to mean that a given file is
"legal" is decidable (which lets TeX off the hook).

YAML has been argued to be several things (configuration,
> serialization, text) and having to choose one in particular cannot work
> out in a satisfying way unless it's about a very basic property not
> based on opinion
>

The YAML spec is insistent that it is a data serialization format.

Lynn

[-- Attachment #2: Type: text/html, Size: 1720 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Philip Kaludercic]

Vasilij Schneidermann <mail@vasilij.de> writes:

> More generally speaking, maybe the whole idea of having a hierarchy of
> major modes needs to be rethought. It is difficult to get such a
> hierarchy right, especially when there are multiple ways to categorize a
> major mode.

Deriving from prog-mode or from text-mode doesn't mean much in practice.
I see it as less of a formal taxonomy of programming languages, and more
of a convenience measure to make changes that (vaguely speaking) "all
programming modes" or "all text-related modes" have in common.  E.g. I
have this in my init.el:

  (add-hook 'text-mode-hook #'flyspell-mode)
  (add-hook 'prog-mode-hook #'flyspell-prog-mode)

and I asked myself would I prefer yaml-mode to use the former or the
latter mode, I'd say that `flyspell-prog-mode' were a better fit.

Of course if neither makes sense or it is too inconsistent, there is no
reason YAML modes would have to inherit from either.

Re: Should yaml-ts-mode inherit from prog-mode?

[Rudolf Schlatte]

Philip Kaludercic <philipk@posteo.net> writes:

> Deriving from prog-mode or from text-mode doesn't mean much in practice.
> I see it as less of a formal taxonomy of programming languages, and more
> of a convenience measure to make changes that (vaguely speaking) "all
> programming modes" or "all text-related modes" have in common.  E.g. I
> have this in my init.el:
>
>   (add-hook 'text-mode-hook #'flyspell-mode)
>   (add-hook 'prog-mode-hook #'flyspell-prog-mode)

Similarly, I use variable-width fonts for text mode buffers and fixed
width fonts for prog-mode buffers.  Since in yaml indentation is
significant, it would be preferrable for me to have yaml-mode inherit
from prog-mode, if only for convenience's sake.

Re: Should yaml-ts-mode inherit from prog-mode?

[Yuan Fu]

> On Mar 3, 2023, at 1:00 AM, Rudolf Schlatte <rudi@constantly.at> wrote:
> 
> Philip Kaludercic <philipk@posteo.net> writes:
> 
>> Deriving from prog-mode or from text-mode doesn't mean much in practice.
>> I see it as less of a formal taxonomy of programming languages, and more
>> of a convenience measure to make changes that (vaguely speaking) "all
>> programming modes" or "all text-related modes" have in common.  E.g. I
>> have this in my init.el:
>> 
>>  (add-hook 'text-mode-hook #'flyspell-mode)
>>  (add-hook 'prog-mode-hook #'flyspell-prog-mode)
> 
> Similarly, I use variable-width fonts for text mode buffers and fixed
> width fonts for prog-mode buffers.  Since in yaml indentation is
> significant, it would be preferrable for me to have yaml-mode inherit
> from prog-mode, if only for convenience's sake.
> 
> 

Agreed. I think of prog-mode as “programming mode” as opposed to “prose mode”, aka text-mode. In that sense, deriving from prog-mode makes more sense to me.

In text related to programming, one would assume monospace font, flyspell-prog-mode, structured text, etc.

Yuan

Re: Should yaml-ts-mode inherit from prog-mode?

[Juri Linkov]

>>>  (add-hook 'text-mode-hook #'flyspell-mode)
>>>  (add-hook 'prog-mode-hook #'flyspell-prog-mode)
>>
>> Similarly, I use variable-width fonts for text mode buffers and fixed
>> width fonts for prog-mode buffers.  Since in yaml indentation is
>> significant, it would be preferrable for me to have yaml-mode inherit
>> from prog-mode, if only for convenience's sake.
>
> Agreed. I think of prog-mode as “programming mode” as opposed to
> “prose mode”, aka text-mode. In that sense, deriving from prog-mode
> makes more sense to me.

Like there is already 'lisp-data-mode', it could be generalized to 'data-mode'
to have 3 mode hierarchy roots:

  prog-mode
  text-mode
  data-mode

Re: Should yaml-ts-mode inherit from prog-mode?

[Daniel Fleischer]

Romanos Skiadas <rom.skiad@gmail.com> writes:

> I was using yaml-ts-mode in Emacs 30, and to my surprise I found it inherits from text-mode.

> However, and here is where things kind of break down a bit: I would personally argue that YAML has a code feel to it,
> what with all the bash scripts that end up in it. Personally, I feel that YAML is more of a prog language than JSON
> (whatever that means), and according to the spec it is a superset of JSON. In fact, the JSON mode shipped with Emacs
> inherits from prog-mode!

I think yaml doesn't look at all like a programming language. There are
no keywords, only free text. The bash scripts you mention are defined in
terms of a multi line string but these strings can contain anything, not
just code snippets. There's no need for parenthesis or oven quotation
marks, it's very lax. It looks much more a free text than a structured
language. 

-- 
Daniel Fleischer

Re: Should yaml-ts-mode inherit from prog-mode?

[Daniel Fleischer]

Romanos Skiadas <rom.skiad@gmail.com> writes:

> I was using yaml-ts-mode in Emacs 30, and to my surprise I found it inherits from text-mode.
>
> However, and here is where things kind of break down a bit: I would personally argue that YAML has a code feel to it,
> what with all the bash scripts that end up in it. Personally, I feel that YAML is more of a prog language than JSON
> (whatever that means), and according to the spec it is a superset of JSON. In fact, the JSON mode shipped with Emacs
> inherits from prog-mode!

I think yaml doesn't look at all like a programming language. There are
no keywords, only free text. The bash scripts you mention are defined in
terms of a multi line string but these strings can contain anything, not
just code snippets. There's no need for parenthesis or oven quotation
marks, it's very lax. It looks much more like free text than a
structured language.

-- 
Daniel Fleischer

Re: Should yaml-ts-mode inherit from prog-mode?

[Dmitry Gutov]

On 28/02/2023 19:56, Daniel Fleischer wrote:
> I think yaml doesn't look at all like a programming language. There are
> no keywords, only free text. The bash scripts you mention are defined in
> terms of a multi line string but these strings can contain anything, not
> just code snippets. There's no need for parenthesis or oven quotation
> marks, it's very lax. It looks much more like free text than a
> structured language.

That's fair impression, but it might not hurt to inherit from prog-mode too.

One criterion that somebody wise once proposed, is whether the language 
has or requires some specific indentation logic, one that's totally 
different from indent-relative.

YAML passes that test solidly. And I'm talking about its basic 
structure, not bash (or JSON, or etc) insertions.

Re: Should yaml-ts-mode inherit from prog-mode?

[Basil Contovounesios]

[ Copying Romanos (who's not subscribed to the list) back into this
  subthread.  Please remember to use 'reply to all'. ]

Dmitry Gutov [2023-02-28 20:33 +0200] wrote:

> On 28/02/2023 19:56, Daniel Fleischer wrote:
>> I think yaml doesn't look at all like a programming language. There are
>> no keywords, only free text. The bash scripts you mention are defined in
>> terms of a multi line string but these strings can contain anything, not
>> just code snippets. There's no need for parenthesis or oven quotation
>> marks, it's very lax. It looks much more like free text than a
>> structured language.
>
> That's fair impression, but it might not hurt to inherit from prog-mode too.
>
> One criterion that somebody wise once proposed, is whether the language has or
> requires some specific indentation logic, one that's totally different from
> indent-relative.
>
> YAML passes that test solidly. And I'm talking about its basic structure, not
> bash (or JSON, or etc) insertions.

-- 
Basil

Re: Should yaml-ts-mode inherit from prog-mode?

[Ongaro]

Daniel Fleischer <danflscr@gmail.com> writes:

> I think yaml doesn't look at all like a programming language. There are
> no keywords, only free text.

If something is considered a "programming language" isn't defined by
whether it has keywords. But we can ignore that, since yaml has quite a
few keywords (e.g. see yaml-font-lock-keywords or others which are not
even implemented in the current yaml-mode, like explicit tags
(https://yaml.org/spec/1.2.2/#example-various-explicit-tags)).

> There's no need for parenthesis or oven quotation marks, it's very
> lax.

Also not true. If you happen to use some of the keywords like "yes", you
have to mind to quote it if you actually meant the string "yes" not a
boolean value of true. Of course that is exactly where having a mode
which highlights these keywords helps.

> It looks much more like free text than a structured language.

It has a pretty strict specification, but it's true that some
implementations may be lax.

In any case, I'm not arguing that YAML is a "programming" language. But
it's definitely not free-form text. And if there is no viable
alternative between text-mode and prod-mode than it should be prog-mode.

Re: Should yaml-ts-mode inherit from prog-mode?

[Daniel Fleischer]

Ongaro [2023-03-11 Sat 18:14] wrote:

> If something is considered a "programming language" isn't defined by
> whether it has keywords. But we can ignore that, since yaml has quite a
> few keywords (e.g. see yaml-font-lock-keywords or others which are not...

If we look at font lock definitions, org-mode also highlights various
constructs; it doesn't mean it's a programming language. 

> Also not true. If you happen to use some of the keywords like "yes", you
> have to mind to quote it if you actually meant the string "yes" not a
> boolean value of true. Of course that is exactly where having a mode
> which highlights these keywords helps.

Please point out where in the YAML spec "yes" is used as a boolean. And
again, usefulness of highlighting different objects does not mean it's a
programming language. 

--
Daniel

Re: Should yaml-ts-mode inherit from prog-mode?

[Rudolf Schlatte]

Daniel Fleischer <danflscr@gmail.com> writes:

> Ongaro [2023-03-11 Sat 18:14] wrote:
>
>> Also not true. If you happen to use some of the keywords like "yes", you
>> have to mind to quote it if you actually meant the string "yes" not a
>> boolean value of true. Of course that is exactly where having a mode
>> which highlights these keywords helps.
>
> Please point out where in the YAML spec "yes" is used as a boolean. And
> again, usefulness of highlighting different objects does not mean it's a
> programming language. 

This is the "Norway Problem".  A YAML block like this:

    geoblock_regions:
      - dk
      - fi
      - is
      - no
      - se

evaluates to the equivalent JSON:

    {"geoblock_regions": ["dk", "fi", "is", false, "se"]}

Example taken from this page, which has many more examples where
semantic highlighting of YAML would avoid a LOT of footguns:

https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-from-hell

Re: Should yaml-ts-mode inherit from prog-mode?

[Yuri Khan]

On Sun, 12 Mar 2023 at 16:50, Daniel Fleischer <danflscr@gmail.com> wrote:

> Please point out where in the YAML spec "yes" is used as a boolean. And
> again, usefulness of highlighting different objects does not mean it's a
> programming language.

<https://yaml.org/type/bool.html>, referenced from
https://yaml.org/spec/1.1/ § 2.4.

The latest version of YAML is 1.2.2, and it no longer specifies ‘y’,
‘n’, ‘yes’, or ‘no’ as booleans. However, existing parser
implementations are not all up to date and still implement 1.0 or 1.1
even in presence of the ‘%YAML 1.2’ tag in the input.

Re: Should yaml-ts-mode inherit from prog-mode?

[David Ongaro]

Daniel Fleischer <danflscr@gmail.com> writes:

> If we look at font lock definitions, org-mode also highlights various
> constructs; it doesn't mean it's a programming language. 

But that's what I mean, the classification as programming language is
independent of if something has keywords. It's neither a necessary nor
sufficient condition (even if we see it as an indicator).

> Please point out where in the YAML spec "yes" is used as a boolean.

I probably can skip that since others did this already.

> And again, usefulness of highlighting different objects does not mean
> it's a programming language.

Again, I don't think anyone argues that YAML is a programming language,
(even if one can easily produce documents much larger than the original
code via anchors and refs and even if one can do pretty crazy things via
tags*). The question is if prog-mode may be a better fit for yaml-mode.
Since JSON is prog-mode and YAML is a superset of JSON the logical
answer appears to be 'yes'.


* E.g. try this in Python

import yaml
y = b'!!python/object/apply:time.sleep [10]'
yaml.unsafe_load(y)

-- 

PS: Unrelated to the topic, but since this header is appearing in every
mail I probably could ask this question anywhere:

> Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/304358>

Isn't the gmane.org domain down since years now? I suppose this header
may be included due to an outdated Mailman configuration? In any case my
actual question is: what is the easiest way to get the corresponding
https://lists.gnu.org/archive/ permalink of an article? Is there a magic
key-combination in Gnus?

Re: Should yaml-ts-mode inherit from prog-mode?

[Matthias Meulien]

[-- Attachment #1: Type: text/plain, Size: 1880 bytes --]

YAML is a data serialization format. Like JSON.

As a user of ELPA's yaml-mode, I was surprised too it isn't coherent with
lisp-data-mode, js-json-mode, json-ts-mode who all inherit from prog-mode.
And I had to disable the input method I usually enable in text mode
buffers, re-enable flymake and my shortcut for
flymake-show-diagnostics-buffer...

Since yaml-ts-mode is new it'd be a good thing to be coherent.

But thinking twice is there any concrete gains of inheriting from prog-mode
rather than text-mode for the end-user?
Don't we already face that dilemna with nxml since XML is used to serialize
data?


Le mar. 28 févr. 2023 à 15:14, Romanos Skiadas <rom.skiad@gmail.com> a
écrit :

> Hello,
>
> I was using yaml-ts-mode in Emacs 30, and to my surprise I found it
> inherits from text-mode.
>
> Now, a case can be made for that for sure. The existing rather widely used
> mode uses text-mode, although that seems like an arbitrary decision by a
> single maintainer:
> https://github.com/yoshiki/yaml-mode/issues/17
> https://github.com/yoshiki/yaml-mode/issues/27
> https://github.com/yoshiki/yaml-mode/issues/52
> https://github.com/yoshiki/yaml-mode/pull/85
>
> (I'm not sure Emacs maintainers are beholden to a decision made some jumps
> away and whether this is a strong enough argument to keep yaml-ts-mode as
> is.)
>
> However, and here is where things kind of break down a bit: I would
> personally argue that YAML has a code feel to it, what with all the bash
> scripts that end up in it. Personally, I feel that YAML is more of a prog
> language than JSON (whatever that means), and according to the spec it is a
> superset of JSON. In fact, the JSON mode shipped with Emacs inherits from
> prog-mode!
>
> So, what do people think? Should yaml-ts-mode be changed?
>
> Best,
> rski
>
>
>
>

-- 
Matthias

[-- Attachment #2: Type: text/html, Size: 2998 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Basil Contovounesios]

Matthias Meulien [2023-03-01 08:46 +0100] wrote:

> But thinking twice is there any concrete gains of inheriting from prog-mode rather than text-mode for the end-user?
> Don't we already face that dilemna with nxml since XML is used to serialize data?

Precisely because there exists a general dilemma (see also mhtml-mode
etc.), and these things are hard to impossible to change after the fact,
I agree that it is more pragmatic for now to stick to discussing the
concrete case of:

> Since yaml-ts-mode is new it'd be a good thing to be coherent.

But because by tautology I do as I do and not as I say I will also take
the liberty of pointing out that while Emacs 27 introduced a
conf-toml-mode based on conf-mode, and the third party toml-mode is also
based on conf-mode, Emacs 29 introduces a toml-ts-mode based on
text-mode.  I'm not sure how intentional/desirable that is, so at risk
of blowing this thread out of proportion I'm copying in Jostein as well.

Thanks,

-- 
Basil

Re: Should yaml-ts-mode inherit from prog-mode?

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

Matthias Meulien [2023-03-01 08:46 +0100] wrote:
> Emacs 29 introduces a toml-ts-mode based on
> text-mode.  I'm not sure how intentional/desirable that is, so at risk
> of blowing this thread out of proportion I'm copying in Jostein as well.
>
> Thanks,
>
Hey.

Author of toml-ts-mode here.

I originally implemented it as a prog-mode derived mode (because for the 
same reasons as reiterated above, that made sense to me).

I was however asked by the more senior Emacs maintainers to make it 
text-mode instead.

So while it was not my decision, I'd just like to point out that there 
/was/ a decision-making process involved when text-mode was decided.

If someone were to change this back to prog-mode before we ship Emacs 
29, I would have no objections to that, but I have no authority to 
mandate such a change either :)

--
Jostein

[-- Attachment #2: Type: text/html, Size: 1624 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Matthias Meulien]

[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]

Simple precision: I am not the author of the quoted text.

Matthias

Le mar. 7 mars 2023, 12:27, Jostein Kjønigsen <jostein@secure.kjonigsen.net>
a écrit :

> Matthias Meulien [2023-03-01 08:46 +0100] wrote:
>
>  Emacs 29 introduces a toml-ts-mode based on
>
> text-mode.  I'm not sure how intentional/desirable that is, so at risk
> of blowing this thread out of proportion I'm copying in Jostein as well.
>
> Thanks,
>
>
> Hey.
>
> Author of toml-ts-mode here.
>
> I originally implemented it as a prog-mode derived mode (because for the
> same reasons as reiterated above, that made sense to me).
>
> I was however asked by the more senior Emacs maintainers to make it
> text-mode instead.
>
> So while it was not my decision, I'd just like to point out that there
> *was* a decision-making process involved when text-mode was decided.
>
> If someone were to change this back to prog-mode before we ship Emacs 29,
> I would have no objections to that, but I have no authority to mandate such
> a change either :)
>
> --
> Jostein
>

[-- Attachment #2: Type: text/html, Size: 1967 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Romanos Skiadas]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

>
> Deriving from prog-mode or from text-mode doesn't mean much in practice.
> I see it as less of a formal taxonomy of programming languages, and more
> of a convenience measure to make changes that (vaguely speaking) "all
> programming modes" or "all text-related modes" have in common.  E.g. I
> have this in my init.el:
>
> I think this is kind of what it boils down to. There's one discussion to
be had about whether YAML is a programming language and one whether
yaml-ts-mode inheriting from prog mode just makes it nicer in practice.

I haven't seen anyone argue that YAML is a programming language per se, and
I don't believe that either. However, to me there is a prevalent feeling
that it would be nice and consistent with the other elpa shipped modes of
similar languages inherit from prog mod.

As to me calling yaml-mode inheriting from text-mode arbitrary; I mean that
in the sense that it was not discussed. It was done with a reason and as a
conscious choice, but it was a singular maintainer's choice in a non-GNU
elpa package.

Best,
Romanos

[-- Attachment #2: Type: text/html, Size: 1349 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I haven't seen anyone argue that YAML is a programming language per se, and
  > I don't believe that either. However, to me there is a prevalent feeling
  > that it would be nice and consistent with the other elpa shipped modes of
  > similar languages inherit from prog mod.

Practically speaking, will users who customize prog-mode prefer for those
customizations to affect the YAML mode?  Or will they be happier
if it does not?

This is what determines whether YAML mode should inherit from
prog-mode.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Should yaml-ts-mode inherit from prog-mode?

[Romanos Skiadas]

[-- Attachment #1: Type: text/plain, Size: 1273 bytes --]

By this standard, I have more than a couple of voices in favour of it.
Should it happen, given the stage the emacs29 branch is in?

(Apologies for the cc if you are not the right person, or if you are not
interested, Eli)

On Fri, 3 Mar 2023 at 06:23, Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > I haven't seen anyone argue that YAML is a programming language per
> se, and
>   > I don't believe that either. However, to me there is a prevalent
> feeling
>   > that it would be nice and consistent with the other elpa shipped modes
> of
>   > similar languages inherit from prog mod.
>
> Practically speaking, will users who customize prog-mode prefer for those
> customizations to affect the YAML mode?  Or will they be happier
> if it does not?
>
> This is what determines whether YAML mode should inherit from
> prog-mode.
>
> --
> Dr Richard Stallman (https://stallman.org)
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)
>
>
>

[-- Attachment #2: Type: text/html, Size: 1962 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Randy Taylor]

[-- Attachment #1: Type: text/plain, Size: 1518 bytes --]

Eli, do you have any comments?

The general consensus seems to be that yaml-ts-mode should inherit from prog-mode instead of text-mode.
The arguments in favour make sense to me.

>On Saturday, March 4th, 2023 at 13:24, Romanos Skiadas <rom.skiad@gmail.com> wrote:
>
>By this standard, I have more than a couple of voices in favour of it. Should it happen, given the stage the emacs29 branch is in?
>
>(Apologies for the cc if you are not the right person, or if you are not interested, Eli)
>
>>On Fri, 3 Mar 2023 at 06:23, Richard Stallman <rms@gnu.org> wrote:
>>[[[ To any NSA and FBI agents reading my email: please consider ]]]
>>[[[ whether defending the US Constitution against all enemies, ]]]
>>[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>>
>>> I haven't seen anyone argue that YAML is a programming language per se, and
>>> I don't believe that either. However, to me there is a prevalent feeling
>>> that it would be nice and consistent with the other elpa shipped modes of
>>> similar languages inherit from prog mod.
>>
>>Practically speaking, will users who customize prog-mode prefer for those
>>customizations to affect the YAML mode? Or will they be happier
>>if it does not?
>>
>>This is what determines whether YAML mode should inherit from
>>prog-mode.
>>
>>--
>>Dr Richard Stallman (https://stallman.org)
>>Chief GNUisance of the GNU Project (https://gnu.org)
>>Founder, Free Software Foundation (https://fsf.org)>>Internet Hall-of-Famer (https://internethalloffame.org)

[-- Attachment #2: Type: text/html, Size: 2874 bytes --]

Re: Should yaml-ts-mode inherit from prog-mode?

[Eli Zaretskii]

> Date: Mon, 20 Mar 2023 01:52:47 +0000
> From: Randy Taylor <dev@rjt.dev>
> Cc: rms@gnu.org, Romanos Skiadas <rom.skiad@gmail.com>, philipk@posteo.net, emacs-devel@gnu.org
> 
> Eli, do you have any comments?

Nothing substantial, no.

> The general consensus seems to be that yaml-ts-mode should inherit from prog-mode instead of
> text-mode.
> The arguments in favour make sense to me.

That's not my impression.  My impression is that there are arguments
both ways, and therefore the original decision of having it in
textmodes still stands.  The arguments to the contrary were not strong
enough to justify the move, IMO.

Re: Should yaml-ts-mode inherit from prog-mode?

[Rudolf Schlatte]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 20 Mar 2023 01:52:47 +0000
>> From: Randy Taylor <dev@rjt.dev>
>> Cc: rms@gnu.org, Romanos Skiadas <rom.skiad@gmail.com>, philipk@posteo.net, emacs-devel@gnu.org
>> 
>> Eli, do you have any comments?
>
> Nothing substantial, no.
>
>> The general consensus seems to be that yaml-ts-mode should inherit from prog-mode instead of
>> text-mode.
>> The arguments in favour make sense to me.
>
> That's not my impression.  My impression is that there are arguments
> both ways, and therefore the original decision of having it in
> textmodes still stands.  The arguments to the contrary were not strong
> enough to justify the move, IMO.

Let me try one last argument for making the switch to prog-mode.  Put
the following yaml fragment into a file and hit M-q ("fill-paragraph").
This destroys the yaml structure:

Before fill-paragraph:

    jobs:
      check:
        runs-on: ubuntu-latest
        strategy:
          fail-fast: false
          matrix:
            emacs_version:
              - 27.1
              - 27.2
              - 28.1
              - 28.2

After fill-paragraph:

    jobs: check: runs-on: ubuntu-latest strategy: fail-fast: false matrix:
      emacs_version: - 27.1 - 27.2 - 28.1 - 28.2

I'd argue that many commands offered by text-mode would have to be
audited or adapted to be meaningful in a yaml file; for example, what is
`forward-sentence' supposed to do in the context of yaml?

Most yaml files "in the wild" are configuration files (for Kubernetes,
ansible, and other "modern" tools), so in practice yaml-mode is much
closer to json-mode and lisp-data-mode, both of which derive from
prog-mode.

From memory, my (probably unfair) impression of the arguments against
the change were "I checked the spec, and it isn't a programming language
and has no keywords, so it's clearly text", with the arguments for the
change were more like "these are the things I'd have to adjust to make
it more like prog-mode"; in my case, this would be changing to a
fixed-width font, turning off flyspell, and unbinding (for safety) lots
of text-mode keybindings as I run afoul of them.

Re: Should yaml-ts-mode inherit from prog-mode?

[Brian Cully via Emacs development discussions.]

Rudolf Schlatte <rudi@constantly.at> writes:

> Most yaml files "in the wild" are configuration files (for 
> Kubernetes,
> ansible, and other "modern" tools), so in practice yaml-mode is 
> much
> closer to json-mode and lisp-data-mode, both of which derive 
> from
> prog-mode.

The distinction, for me, is that YAML is highly structured text, 
which will fail to parse intelligibly for everything that tries to 
use it if that structure is ignored or malformed. It is intended 
primarily for machine interpretation, not human. Contrast that to 
Org, which is obviously intended primarily for humans, with 
structural elements specifically chosen so that it is intelligible 
by humans in the absence of org-mode entirely, and is still quite 
usable even with malfored structure. Org is, primarily, for 
prose. YAML is for instructing a machine. If your job is telling a 
machine what to do, you're a prog-mode, in my book.

And count me in as someone who has to jump through a bunch of 
hoops to get yaml-mode working “correctly” due to it being based 
on text mode.

-bjc

Re: Should yaml-ts-mode inherit from prog-mode?

[Eli Zaretskii]

> From: Rudolf Schlatte <rudi@constantly.at>
> Date: Mon, 20 Mar 2023 15:37:05 +0100
> 
> Let me try one last argument for making the switch to prog-mode.  Put
> the following yaml fragment into a file and hit M-q ("fill-paragraph").
> This destroys the yaml structure:
> 
> Before fill-paragraph:
> 
>     jobs:
>       check:
>         runs-on: ubuntu-latest
>         strategy:
>           fail-fast: false
>           matrix:
>             emacs_version:
>               - 27.1
>               - 27.2
>               - 28.1
>               - 28.2
> 
> After fill-paragraph:
> 
>     jobs: check: runs-on: ubuntu-latest strategy: fail-fast: false matrix:
>       emacs_version: - 27.1 - 27.2 - 28.1 - 28.2

This just means that yaml-ts-mode should be fixed by defining an
appropriate fill-paragraph-function.  Many descendants of prog-mode
define such functions, so being a descendant of prog-mode doesn't
magically make M-q DTRT for a mode.

> I'd argue that many commands offered by text-mode would have to be
> audited or adapted to be meaningful in a yaml file; for example, what is
> `forward-sentence' supposed to do in the context of yaml?

What useful things does forward-sentence do in C mode or in Emacs Lisp
mode?

Anyway, this and other similar arguments were already brought up, I
believe.  And other arguments were brought up in defense of the
opposite view.  Which is why I think there's no strong justification
to make yaml-ts-mode a descendant of prog-mode, after we had it where
it is now for some time.

Re: Should yaml-ts-mode inherit from prog-mode?

[Jostein Kjønigsen]

On 3/20/23 15:37, Rudolf Schlatte wrote:
>
> Let me try one last argument for making the switch to prog-mode.  Put
> the following yaml fragment into a file and hit M-q ("fill-paragraph").
> This destroys the yaml structure:
>
> Before fill-paragraph:
>
>      jobs:
>        check:
>          runs-on: ubuntu-latest
>          strategy:
>            fail-fast: false
>            matrix:
>              emacs_version:
>                - 27.1
>                - 27.2
>                - 28.1
>                - 28.2
>
> After fill-paragraph:
>
>      jobs: check: runs-on: ubuntu-latest strategy: fail-fast: false matrix:
>        emacs_version: - 27.1 - 27.2 - 28.1 - 28.2
>
Following up on this exact use-case, due to my text-mode hooks, using 
yaml-ts-mode many times ended up with causing invalid YAML or breaking 
existing YAML-files.

I was pretty sure yaml-ts-mode itself was buggy and actually started 
using fundamental-mode to edit yaml-files instead, until I realized what 
was going on.

== Yes. The user-experience was literally that bad. ==

Lots of people are going to have lots of problems and breakage when 
using yaml-ts-mode as it is today, deriving from text-mode.

I plea from the bottom of my heart that we allow a change to prog-mode.

--
Jostein

Re: Should yaml-ts-mode inherit from prog-mode?

[Eli Zaretskii]

> Date: Tue, 21 Mar 2023 12:19:31 +0100
> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
> 
> On 3/20/23 15:37, Rudolf Schlatte wrote:
> >
> > Let me try one last argument for making the switch to prog-mode.  Put
> > the following yaml fragment into a file and hit M-q ("fill-paragraph").
> > This destroys the yaml structure:
> >
> > Before fill-paragraph:
> >
> >      jobs:
> >        check:
> >          runs-on: ubuntu-latest
> >          strategy:
> >            fail-fast: false
> >            matrix:
> >              emacs_version:
> >                - 27.1
> >                - 27.2
> >                - 28.1
> >                - 28.2
> >
> > After fill-paragraph:
> >
> >      jobs: check: runs-on: ubuntu-latest strategy: fail-fast: false matrix:
> >        emacs_version: - 27.1 - 27.2 - 28.1 - 28.2
> >
> Following up on this exact use-case, due to my text-mode hooks, using 
> yaml-ts-mode many times ended up with causing invalid YAML or breaking 
> existing YAML-files.
> 
> I was pretty sure yaml-ts-mode itself was buggy and actually started 
> using fundamental-mode to edit yaml-files instead, until I realized what 
> was going on.
> 
> == Yes. The user-experience was literally that bad. ==
> 
> Lots of people are going to have lots of problems and breakage when 
> using yaml-ts-mode as it is today, deriving from text-mode.
> 
> I plea from the bottom of my heart that we allow a change to prog-mode.

We should instead fix yaml-ts-mode by providing a proper
fill-paragraph-function for it.  No one said that the default
text-mode one will do TRT.

Re: Should yaml-ts-mode inherit from prog-mode?

[Rudolf Schlatte]

Eli Zaretskii <eliz@gnu.org> writes:

> We should instead fix yaml-ts-mode by providing a proper
> fill-paragraph-function for it.  No one said that the default
> text-mode one will do TRT.

Quoting yaml.org: “YAML is a human-friendly data serialization language
for all programming languages.”  (See https://yaml.org/spec/1.2.2/ for
all the gory details.)  Both whitespace and line breaks are semantically
meaningful.

One meaningful behavior for `fill-paragraph' could be to allow it only
within YAML flowed blocks and flow scalars (see
https://yaml-multiline.info):

    script: |
      # this block cannot be refilled.
      apt-get -y update
      apt-get -y install my-necessary-package
    description: >
      The above script updates the system and installs a necessary
      package.

      This is a second line, where we give more information We had to
      leave an empty line in the yaml file to get a line break.
    comment: "This is a double-quoted flow scalar.
      We have to escape line breaks like this:\
      All other line breaks in the string
      are read as spaces."

Re: Should yaml-ts-mode inherit from prog-mode?

[Yuri Khan]

On Tue, 21 Mar 2023 at 21:14, Rudolf Schlatte <rudi@constantly.at> wrote:

> One meaningful behavior for `fill-paragraph' could be to allow it only
> within YAML flowed blocks and flow scalars (see
> https://yaml-multiline.info):
>
>     script: |
>       # this block cannot be refilled.
>       apt-get -y update
>       apt-get -y install my-necessary-package

This is reasonable behavior for literal-styled YAML blocks containing
a sequence of shell commands, or formatted code in a programming
language, or other line-break-sensitive content such as an embedded
YAML document. Swiping an example from [1]:

    env1:
      nova:
        flavor: "30"
        nics:
          - net_id: db6e78cb-99fa-4223-9a6f-3b4a9b486d9b
        userdata: |
          #cloud-config
          disable_root: False
          ssh_pwauth: True
          chpasswd:
            list: |
              root: password

[1]: https://medium.com/@george.shuklin/embedding-yaml-into-yaml-3511246d7ea7

On the other hand, for example, OpenAPI specifications accept Markdown
in several of its properties. My personal preference for these is to
notate them as literal-styled blocks so that joining lines into a
single paragraph happens in the Markdown processor.

In these cases, some users might want to ‘fill-paragraph’ Markdown
paragraphs inside a literal-styled YAML block.

    description: |
      Create a new foo
      and return a 201 with `Location:` header set.

      If the input data is invalid,
      return a 400 with a detailed error report.

Without knowing the syntax for the YAML string value, there is not
enough information to allow or restrict ‘fill-paragraph’.

A sensible compromise could be to allow ‘fill-paragraph’ at user’s
discretion in all string literals, but recognize YAML’s string start
and end as paragraph boundaries.

Re: Should yaml-ts-mode inherit from prog-mode?

[Rudolf Schlatte]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> In these cases, some users might want to ‘fill-paragraph’ Markdown
> paragraphs inside a literal-styled YAML block.
> 
>     description: |
>       Create a new foo
>       and return a 201 with `Location:` header set.
> 
>       If the input data is invalid,
>       return a 400 with a detailed error report.
> 
> Without knowing the syntax for the YAML string value, there is not
> enough information to allow or restrict ‘fill-paragraph’.
>
> A sensible compromise could be to allow ‘fill-paragraph’ at user’s
> discretion in all string literals, but recognize YAML’s string start
> and end as paragraph boundaries.

Good points and convincing examples, thank you.  It's also easier to
describe the behavior that way: "`fill-paragraph' is enabled inside block
and flow scalars.  Text inside a block or flow scalar will be reflowed
as usual, with empty lines acting as paragraph boundaries."

I.e., your example would be reflowed like this (after `M-q' on the line
starting with "Create" or "and"), if I understood it correctly?

    description: |
      Create a new foo and return a 201 with `Location:` header set.

      If the input data is invalid,
      return a 400 with a detailed error report.

Re: Should yaml-ts-mode inherit from prog-mode?

[Yuri Khan]

On Wed, 22 Mar 2023 at 10:21, Rudolf Schlatte <rudi@constantly.at> wrote:

> >     description: |
> >       Create a new foo
> >       and return a 201 with `Location:` header set.

> I.e., your example would be reflowed like this (after `M-q' on the line
> starting with "Create" or "and"), if I understood it correctly?
>
>     description: |
>       Create a new foo and return a 201 with `Location:` header set.
>
>       If the input data is invalid,
>       return a 400 with a detailed error report.

Yes, with a suitable fill-column.

(Personally, I prefer not filling but practicing semantic line breaks
<https://sembr.org/>.)

Re: Should yaml-ts-mode inherit from prog-mode?

[David Ongaro]

Rudolf Schlatte <rudi@constantly.at> writes:

> From memory, my (probably unfair) impression of the arguments against
> the change were "I checked the spec, and it isn't a programming language
> and has no keywords, so it's clearly text"

Nobody is arguing whether it is a programming lanugage, and the keyword
argument was refuted. So what is left besides momentum? I can understand
the resistance to change something which is already working in the wild
(even if it's not ideal). But in contrast to yaml-mode it seems
yaml-ts-mode is quite new, so it's rather a chance than a risk to change
it.

Re: Should yaml-ts-mode inherit from prog-mode?

[Randy Taylor]

On Monday, March 20th, 2023 at 08:07, Eli Zaretskii <eliz@gnu.org> wrote:
> > Date: Mon, 20 Mar 2023 01:52:47 +0000
> 
> > From: Randy Taylor dev@rjt.dev
> > Cc: rms@gnu.org, Romanos Skiadas rom.skiad@gmail.com, philipk@posteo.net, emacs-devel@gnu.org
> > 
> > Eli, do you have any comments?
> 
> 
> Nothing substantial, no.
> 
> > The general consensus seems to be that yaml-ts-mode should inherit from prog-mode instead of
> > text-mode.
> > The arguments in favour make sense to me.
> 
> 
> That's not my impression. My impression is that there are arguments
> both ways, and therefore the original decision of having it in
> textmodes still stands. The arguments to the contrary were not strong
> enough to justify the move, IMO.

Fair enough.

I originally made yaml-ts-mode inherit from text-mode because the already existing yaml-mode package in MELPA does, and because I was under the impression that prog-mode was meant only for programming languages (as the docstring states). It has since come to my attention that there is not such a strict requirement for deriving from prog-mode, but rather that folks who customize prog-mode would want those customizations applying to the derived mode as well. As many others have stated (for various reasons), I think yaml-ts-mode makes more sense deriving from prog-mode than text-mode (assuming there are no negative consequences of doing so, but I can't think of any).