nxml-mode: Derive from prog-mode instead of text-mode

nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 1191 bytes --]

Hey everyone.

One thing I find unintuitive about nxml-mode is how it derives from
text-mode.
The reason this surprises me is that when I edit XML-files, I almost
exclusively (to the point of not being able to remember an exception) am
working with programming-related tasks.
Therefore I expect my prog-mode-hook to have fired, and my prog-mode
settings and key-bindings to be in effect (compile, projectile-
navigation, commenting/uncommenting, etc etc).
But no. I have my text-mode settings instead (spell-checking, etc),
which often conflict with what I want to get done.
Could we consider changing nxml-mode to derive from prog-mode instead?

While it can be considered a "breaking"-change, I think this would make
much more sense. And such a change is not entirely unprecedented either:
css-mode made the same transistion back in 2015. [1]
I can provide the patch for it, but first I thought I'd first check what
people's opinion on such a change would be.
[1] https://github.com/emacs-mirror/emacs/commit/f21d9b6e4461719cece6ad4adc85ea84ddc5a4e0
--
 Yours truly
Jostein Kjønigsen

jostein@kjonigsen.net 🍵 jostein@gmail.com
https://jostein.kjonigsen.net


[-- Attachment #2: Type: text/html, Size: 1960 bytes --]

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Yuri Khan]

On Wed, May 10, 2017 at 5:35 PM, Jostein Kjønigsen
<jostein@secure.kjonigsen.net> wrote:

> One thing I find unintuitive about nxml-mode is how it derives from text-mode.
>
> The reason this surprises me is that when I edit XML-files, I almost exclusively (to the point of not being able to remember an exception) am working with programming-related tasks.

You never edit XHTML, or Docbook, or any other XML-based formats which
get transformed into documentation?

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 1536 bytes --]

On Wed, May 10, 2017, at 12:40 PM, Yuri Khan wrote:
> You never edit XHTML, or Docbook, or any other XML-based formats which> get transformed into documentation?

First and foremost: HTML counts as programming among the majority of the
population, so thank you for backing my argument :)With regard to XHTML, I use HTML-oriented major-modes for editing these
as I've found them to have much better support for HTML-specific use-
cases (valid element-names, valid attributes, etc), and I think this is
fairly common among most web-developers.
And to be honest, these days I almost exclusively stick to HTML5 anyway,
which has "thrown out" all the XMLness of XHTML with requirements to
close tags and all that stuff.
In that regard (with a risk of being a bit opinionated), I think
considering nxml-mode as a major-mode for general text-editing (and HTML-
editing especially) is looking backwards.
As for Docbook or used other XML-based formats for documentation, I'll
admit I haven't done that, but I can see how that can also be popular
use-cases too.
So nxml-mode may have to appease to different use-cases.

In that case, me saying nxml-mode is mostly used for programming-related
tasks, may be opinionated (and definitely not scientific by any
standards).
On the other hand I can't see how it is more opinionated or less
scientific than assuming the majority of XML-work to be non-programming-
related. But that's seemingly OK?
I'd love to hear other opinions on the matter.

--
Jostein Kjønigsen




[-- Attachment #2: Type: text/html, Size: 2200 bytes --]

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Yuri Khan]

On Wed, May 10, 2017 at 5:52 PM, Jostein Kjønigsen
<jostein@secure.kjonigsen.net> wrote:

> As for Docbook or used other XML-based formats for documentation, I'll admit
> I haven't done that, but I can see how that can also be popular use-cases
> too.
>
> So nxml-mode may have to appease to different use-cases.

Yes. “Textness” or “programmingness” of nxml-mode should probably be
dependent on the schema used by a particular file.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> One thing I find unintuitive about nxml-mode is how it derives from
> text-mode.

I don't think it should choose: it should derive from both.


        Stefan


PS: I don't think it can really be done, given the way text-mode and
    prog-mode are currently defined, but we can get closer, e.g. by
    inheriting from both keymaps running both mode hooks, ...

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Eli Zaretskii]

> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
> Date: Wed, 10 May 2017 12:35:09 +0200
> 
> Could we consider changing nxml-mode to derive from prog-mode instead?

What user-visible changes do you expect to see as result of this?

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 1867 bytes --]


On Wed, May 10, 2017, at 06:38 PM, Eli Zaretskii wrote:
>> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
>> Date: Wed, 10 May 2017 12:35:09 +0200
>> 
>> Could we consider changing nxml-mode to derive from prog-mode
>> instead?> 
> What user-visible changes do you expect to see as result of this?

In short: always  have my "programming-setup" available when I do
programming.
I think is is best illustrated with an example:

In prog-mode  I have F5 bound to the recompile-command, and the compile
command set via .dir-locals.el. That means in any "project" I work with,
I can always press F5 to build and see that everything works. It's a
good workflow.
I also rely on projectile for file-navigation, which is also enabled in
the prog-mode-hook.
Now consider I press F5 to start building and see that something fails
to due invalid data in a XML-file. I'll need to edit XML.
I use projectile to switch to the "bad" XML-file, and correct the
settings.  I have spell-checking everywhere telling me my XML-
elements are not valid English words. I disable flyspell (which I
have for text-mode).
By reflex I press F5 to build. It doesn't. Because my prog-mode hook
was not run.
A little bit annoyed, I try to switch back to a source-file to build
from (by reflex using projectile). I press "C-c p f". But there is
no projectile. Because my prog-mode hook never ran, thus projectile
is not on.
Yes, I know there are workarounds. Like redundantly binding the same
keys many places, or to manually run prog-mode from the nxml-mode hook,
and then cancel out whatever I set in text-mode.
I just honestly think it makes infinitely more sense to have nxml-mode
just invoke prog-mode directly.
To put it the other way way: What's the argument for nxml-mode
predominantly invoking text-mode-hooks?
--
Regards
Jostein Kjønigsen

[-- Attachment #2: Type: text/html, Size: 2629 bytes --]

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Eli Zaretskii]

> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
> Cc: emacs-devel@gnu.org
> Date: Wed, 10 May 2017 19:59:13 +0200
> 
> In prog-mode I have F5 bound to the recompile-command, and the compile command set via .dir-locals.el.
> That means in any "project" I work with, I can always press F5 to build and see that everything works. It's a
> good workflow.
> 
> I also rely on projectile for file-navigation, which is also enabled in the prog-mode-hook.
> 
> Now consider I press F5 to start building and see that something fails to due invalid data in a XML-file. I'll need
> to edit XML.
> 
> I use projectile to switch to the "bad" XML-file, and correct the settings. I have spell-checking everywhere
> telling me my XML-elements are not valid English words. I disable flyspell (which I have for text-mode).
> 
> By reflex I press F5 to build. It doesn't. Because my prog-mode hook was not run.
> 
> A little bit annoyed, I try to switch back to a source-file to build from (by reflex using projectile). I press "C-c p
> f". But there is no projectile. Because my prog-mode hook never ran, thus projectile is not on.

These seem all to stem from your personal setup, not from inherent
features of prog-mode that are absent from text-mode.  In fact,
prog-mode is exceedingly minimal: it only sets 3 variables, none of
them related to what you describe.

> I just honestly think it makes infinitely more sense to have nxml-mode just invoke prog-mode directly.
> 
> To put it the other way way: What's the argument for nxml-mode predominantly invoking text-mode-hooks?

History, I guess.  In the beginning there were SGML and HTML, the
latter is (or was back then) mostly text with markup.  XML just
naturally inherited from HTML.

I could understand an argument that nowadays XML and even HTML deviate
a lot from text with markup, but I don't see how prog-mode would be
more appropriate.  I tend to think that we should come up with a new
family of modes, which specifically caters to the likes of XML-based
coding.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Eli Zaretskii]

> Date: Wed, 10 May 2017 21:59:57 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > I have spell-checking everywhere
> > telling me my XML-elements are not valid English words. I disable flyspell (which I have for text-mode).

Btw, this ought to work, because ispell.el is supposed to skip markup
in XML mode.  Perhaps the problem is that ispell-buffer-local-parsing
doesn't recognize nxml-mode?  Did you try setting ispell-skip-html to
t?

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

On Wed, May 10, 2017, at 08:59 PM, Eli Zaretskii wrote:
> These seem all to stem from your personal setup, not from inherent
> features of prog-mode that are absent from text-mode.  In fact,
> prog-mode is exceedingly minimal: it only sets 3 variables, none of
> them related to what you describe.

Obviously. 

But prog-mode represents a API, a endpoint, for end-users and developers
to wire up anything and any customization they deem programming-related.
With prog-mode API-wise being a "success", shouldn't Emacs core honour
that API by using it where appropriate? That would IMO be the consistent
thing to do.
That Emacs *ships* with only 3 such customizations out of the box seems
to me irellevant.
> I could understand an argument that nowadays XML and even HTML deviate> a lot from text with markup, but I don't see how prog-mode would be
> more appropriate.  I tend to think that we should come up with a new
> family of modes, which specifically caters to the likes of XML-based
> coding.

Something like structured-text-mode ?  Which for instance nxml-mode, json-
mode, yaml-mode (etc etc) could derive from.
That could be another approach, and I'd be happy with that, if
done properly.
--
Regards
Jostein Kjønigsen

[-- Attachment #2: Type: text/html, Size: 1929 bytes --]

Re: nxml-mode: Derive from prog-mode instead of text-mode

[raman]

Note that the syntax table might matter in how nxml-mode functions, and
text-mode and prog-mode likely have slightly different syntax tables 
-- 

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Eli Zaretskii]

> From: raman <raman@google.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  jostein@kjonigsen.net,  emacs-devel@gnu.org
> Date: Thu, 11 May 2017 08:15:41 -0700
> 
> Note that the syntax table might matter in how nxml-mode functions, and
> text-mode and prog-mode likely have slightly different syntax tables 

True in general, but in this case, prog-mode doesn't define any syntax
table, AFAIK.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Eli Zaretskii]

> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
> Cc: emacs-devel@gnu.org
> Date: Thu, 11 May 2017 09:29:59 +0200
> 
> But prog-mode represents a API, a endpoint, for end-users and developers to wire up anything and any
> customization they deem programming-related.
> 
> With prog-mode API-wise being a "success", shouldn't Emacs core honour that API by using it where
> appropriate? That would IMO be the consistent thing to do.
> 
> That Emacs ships with only 3 such customizations out of the box seems to me irellevant.

My point is, given how little prog-mode customizes, there's no
particular reason why your customizations should start from prog-mode.
They could start from any other major mode, since you need to invent
most of them from scratch anyway; prog-mode doesn't help you make
fewer customizations.

>  I could understand an argument that nowadays XML and even HTML deviate
>  a lot from text with markup, but I don't see how prog-mode would be
>  more appropriate. I tend to think that we should come up with a new
>  family of modes, which specifically caters to the likes of XML-based
>  coding.
> 
> Something like structured-text-mode ? Which for instance nxml-mode, json-mode, yaml-mode (etc etc) could
> derive from.

Yes.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]

Jostein Kjønigsen <jostein@secure.kjonigsen.net> schrieb am Do., 11. Mai
2017 um 09:30 Uhr:

> On Wed, May 10, 2017, at 08:59 PM, Eli Zaretskii wrote:
>
> These seem all to stem from your personal setup, not from inherent
> features of prog-mode that are absent from text-mode.  In fact,
> prog-mode is exceedingly minimal: it only sets 3 variables, none of
> them related to what you describe.
>
>
> Obviously.
>
> But prog-mode represents a API, a endpoint, for end-users and developers
> to wire up anything and any customization they deem programming-related.
>
> With prog-mode API-wise being a "success", shouldn't Emacs core honour
> that API by using it where appropriate? That would IMO be the consistent
> thing to do.
>
>
prog-mode is more of an implementation detail than an API. See e.g. the
discussion in
http://cc-mode-help.narkive.com/KcrSl9QI/standalone-cc-mode-doesn-t-run-prog-mode-hook:
"The fact that many programming lanuguage modes are derived from prog-mode
is an internal implementation detail, not a user level feature, and users
taking advantage of such things are liable to get into trouble."

I think it also represents a false dichotomy: computer languages can't be
arbitrarily classified in "programming" and "text"; it's rather a
continuum. If HTML is a "programming language", why not reStructuredText or
Markdown? All three are markup languages originally intended for documents,
with well-defined grammars. If Markdown is also a "programming language",
why not plain text files that contain similar ad-hoc markup?

[-- Attachment #2: Type: text/html, Size: 2275 bytes --]

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Dmitry Gutov]

On 14.05.2017 22:28, Philipp Stephani wrote:

> prog-mode is more of an implementation detail than an API. See e.g. the 
> discussion in 
> http://cc-mode-help.narkive.com/KcrSl9QI/standalone-cc-mode-doesn-t-run-prog-mode-hook: 
> "The fact that many programming lanuguage modes are derived from 
> prog-mode is an internal implementation detail, not a user level 
> feature, and users taking advantage of such things are liable to get 
> into trouble."

That is Alan's opinion, not gospel.

> I think it also represents a false dichotomy: computer languages can't 
> be arbitrarily classified in "programming" and "text"; it's rather a 
> continuum. If HTML is a "programming language", why not reStructuredText 
> or Markdown? All three are markup languages originally intended for 
> documents, with well-defined grammars. If Markdown is also a 
> "programming language", why not plain text files that contain similar 
> ad-hoc markup?

In the end, we have to consider whether an average user would consider a 
major mode to be "programming mode". Or we might choose some other 
criterion.

IIRC Stefan mentioned one such criterion: a mode has an indentation 
function that works significantly differently from indent-relative.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Tom Tromey]

>>>>> "Philipp" == Philipp Stephani <p.stephani2@gmail.com> writes:

Philipp> prog-mode is more of an implementation detail than an API.

I don't think that's correct.  For example, the emacs manual recommends
it to users:

      In particular, if you want to apply a hook function to any
    programming language mode, add it to ‘prog-mode-hook’; Prog mode is a
    major mode that does little else than to let other major modes inherit
    from it, exactly for this purpose.

Tom

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 2198 bytes --]

On Mon, May 15, 2017, at 05:05 AM, Tom Tromey wrote:
>>>>>> "Philipp" == Philipp Stephani <p.stephani2@gmail.com> writes:
> 
> Philipp> prog-mode is more of an implementation detail than an API.
> 
> I don't think that's correct.  For example, the emacs manual
> recommends> it to users:
> 
>       In particular, if you want to apply a hook function to any
>     programming language mode, add it to ‘prog-mode-hook’; Prog mode
>     is a>     major mode that does little else than to let other major modes
>     inherit
>     from it, exactly for this purpose.
> 
> Tom

Thanks for the feedback everyone.

Much of the response I have received has been arguments for why one
should *not *use prog-mode and how it is *wrong* to assume prog-mode
represents a specific behaviour.
This had me quite surprised. It seems very counter-intuitive, and 100%
opposite to how I think it's perceived and used in the community.
Given the amount of "negative" type replies I've gotten, I'm tempted to
post a counter-challenge: If this isn't the appropriate use of prog-
mode... What would you consider the proper uses of it to be (both as a
developer or user)?
Apart from the "it's an implementation detail" (which from what I can
tell is *not *a consensus-opinion), are there any other opinions?
If nobody can think of anything other use-cases... What use it? Like
Alan points out: it represents absolutely minimal amount of code-
reuse internally with only 3 bindings, so that's clearly not a
convincing argument.
So before anything else, I would like to assess if we can have a
majority agreement that prog-mode is NOT an implementation detail, but
rather an API both users and developers are encouraged to use for programming-
related activities and customizations.
If we can agree on that, I think there are options for moving forward.

If not, and we take the "implementation detail" claim at face value, I
think it would be a quite interesting exercise to remove prog-mode from
Emacs and see just how many packages and configs out there were stopped
working, vs how many were built on the same assumption. Anyone want to
take bets? :)
--
Regards
Jostein

[-- Attachment #2: Type: text/html, Size: 3106 bytes --]

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Yuri Khan]

On Tue, May 16, 2017 at 5:34 PM, Jostein Kjønigsen
<jostein@secure.kjonigsen.net> wrote:

> Given the amount of "negative" type replies I've gotten, I'm tempted to post
> a counter-challenge: If this isn't the appropriate use of prog-mode... What
> would you consider the proper uses of it to be (both as a developer or
> user)?

I have two functions in my current Emacs configuration that run on
'prog-mode-hook.

* One binds 'newline-and-indent to RET and 'newline to C-j. I suppose
it is a leftover from the times before electric-indent-mode. I should
get rid of it.

* Another changes the value of 'whitespace-style. In text modes, I
only visualize spaces, tabs and newlines; in programming modes, I also
mark all kinds of coding style violations such as overly long lines,
mixing tabs and spaces, and using an indentation character that
disagrees with the value of 'indent-tabs-mode specific to major mode.
This probably could be unified, too. The big difference here is modes
that are neither text nor programming — I call them application modes
— such as dired, calendar, or compilation — because they don’t need
even whitespace visualization.


As for your cases, you mention compilation and project management. I
think they are not actually programming-dependent. I bind 'recompile
with a compile-command of "make" (actually a variation on that) to
<f9> globally, and I use it in contexts that I don’t consider
programming — e.g. converting images between formats.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> This probably could be unified, too. The big difference here is modes
> that are neither text nor programming — I call them application modes
> — such as dired, calendar, or compilation — because they don’t need

These should supposedly derive from special-mode.


        Stefan

RE: nxml-mode: Derive from prog-mode instead of text-mode

[Drew Adams]

> > This probably could be unified, too. The big difference here is modes
> > that are neither text nor programming — I call them application modes
> > — such as dired, calendar, or compilation — because they don’t need
> 
> These should supposedly derive from special-mode.

Absolutely not, for Dired.  It should not derive from
special-mode, tabulated-list-mode, or anything else.

That's almost like saying that Emacs should derive
from special-mode. ;-)

(But Stefan has admitted that he rarely uses Dired.)

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Eric Abrahamsen]

Drew Adams <drew.adams@oracle.com> writes:

>> > This probably could be unified, too. The big difference here is modes
>> > that are neither text nor programming — I call them application modes
>> > — such as dired, calendar, or compilation — because they don’t need
>> 
>> These should supposedly derive from special-mode.
>
> Absolutely not, for Dired.  It should not derive from
> special-mode, tabulated-list-mode, or anything else.

I'm curious why not special-mode. My observation of special-mode is that
it handles: 1) quit, 2) refresh, and 3) unbind everything else.

Honest question...

RE: nxml-mode: Derive from prog-mode instead of text-mode

[Drew Adams]

> >> > This probably could be unified, too. The big difference here is modes
> >> > that are neither text nor programming — I call them application modes
> >> > — such as dired, calendar, or compilation — because they don’t need
> >>
> >> These should supposedly derive from special-mode.
> >
> > Absolutely not, for Dired.  It should not derive from
> > special-mode, tabulated-list-mode, or anything else.
> 
> I'm curious why not special-mode. My observation of special-mode is
> that it handles: 1) quit, 2) refresh, and 3) unbind everything else.

Quit (`q') and refresh (`g') - that's all.  Oh, and `h' too - but
not `?'.

Like `?', all of the `special-mode' keys besides `q', `g', and `h'
are inappropriate - Dired uses them for other things.

Sure, Dired could override those other bindings, but it makes no
sense to derive from special mode just to get the mundane bindings
for `q', `g', and `h' - providing just `quit-window', `revert-buffer',
and `describe-mode' hardly makes for a "special" mode.

And Dired has its own hooks.  I see no reason for Dired to
respect/use `special-mode-hook'.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> Quit (`q') and refresh (`g') - that's all.  Oh, and `h' too - but
> not `?'.

> Like `?', all of the `special-mode' keys besides `q', `g', and `h'
> are inappropriate - Dired uses them for other things.

> Sure, Dired could override those other bindings, but it makes no
> sense to derive from special mode just to get the mundane bindings
> for `q', `g', and `h' - providing just `quit-window', `revert-buffer',
> and `describe-mode' hardly makes for a "special" mode.

> And Dired has its own hooks.  I see no reason for Dired to
> respect/use `special-mode-hook'.

So you don't see any use.  That's not a very strong argument against it,
it's just a lack of argument in favor of it.


        Stefan

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Yuri Khan]

On Wed, May 17, 2017 at 1:19 AM, Drew Adams <drew.adams@oracle.com> wrote:

> it makes no
> sense to derive from special mode just to get the mundane bindings
> for `q', `g', and `h' - providing just `quit-window', `revert-buffer',
> and `describe-mode' hardly makes for a "special" mode.

Derivation is not only a mechanism to get functionality. It is
primarily a mechanism to express facts and relationships.

What makes special-mode special is that ordinary character keys are
divorced from their usual self-insert meaning, instead, they are all
available for commands. And that gives me an idea… does
key-translation-map work if made buffer-local?

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> available for commands. And that gives me an idea… does
> key-translation-map work if made buffer-local?

Depends.


        Stefan

RE: nxml-mode: Derive from prog-mode instead of text-mode

[Drew Adams]

> > it makes no sense to derive from special mode just to get the mundane
> > bindings for `q', `g', and `h' - providing just `quit-window',
> > `revert-buffer', and `describe-mode' hardly makes for a "special" mode.
> 
> Derivation is not only a mechanism to get functionality. It is
> primarily a mechanism to express facts and relationships.
> 
> What makes special-mode special is that ordinary character keys are
> divorced from their usual self-insert meaning, instead, they are all
> available for commands.

TL;DR:

What you say "makes special-mode special" has nothing specially
to do with `special-mode'.  It's just `make-sparse-keymap'.

There are "special" modes (which include Dired) and there
is `special-mode' (which probably should have been given a
different name, since "special mode" has another meaning).

---

Your characterization of "makes special-mode special" is
elegant in the abstract, perhaps.

In reality, to accomplish that we already have a better, more
exact means, and it is what is used in practice:
`make-sparse-keymap'.

In the delivered Lisp sources for GNU Emacs 24.5 there are 46
instances of deriving from `special-mode'.  And there are 829
instances of invoking `make-sparse-keymap'.

The fact that `special-mode' is _not_ used as much as your
"what makes special-mode special" would suggest suggests that
removing self-insertion from char keys is _not_ "what makes
special-mode special".

There are 875 (829 + 46) places where a sparse keymap is used.
These are generally modes where "ordinary character keys are
divorced from their usual self-insert meaning".  (Maybe a few
of those involve self-insertion, but probably very few.)

If existing uses of both `special-mode' and `make-sparse-keymap'
are well thought out then cases where "ordinary character keys
are divorced from their usual self-insert meaning" are 19 times
more common than cases where `special-mode' is used.  That's a
lot.

But let's allow for some cases that were not well thought out
or that are just old cruft - cases where, if well thought out
today would in fact call for using the `special-mode' keys and
not just a sparse keymap - i.e., not just "ordinary character
keys [being] divorced from their usual self-insert meaning".

How much such potential is there, would you suppose?

Or do you think that every mode that uses a sparse keymap
should instead be derived from `special-mode' and then
override any `special-mode' keys that it doesn't like/need?
I think not.

Modes that want the `special-mode' keys are already derived
from `special-mode'.  Those that don't aren't.  Modulo perhaps
some minor potential for retinkering.

Since not many use cases for a sparse keymap seem to call for
use of `special-mode' (its keys and hook - which is what it
offers), do you think we perhaps need another, more bare-bones,
not-so-special mode that just provides an empty sparse keymap
and a mode hook?  (Fundamental mode provides a hook but no map.)

Should every mode that needs a sparse keymap and doesn't
need/want the `special-mode' keys or doesn't want to be
affected by its hook inherit from such a new, purer mode?
I think not.

`make-sparse-keymap', not `special-mode', is the beast that
corresponds to your "ordinary character keys are divorced
from their usual self-insert meaning".  That divorce is not
"what makes special-mode special".

To come back to the start of this detour from the main thread:
Dired and similar modes already create sparse keymaps.  They
already know enough to use  `make-sparse-keymap'.  They use
your abstraction, without using `special-mode'.

And there is another thing special about `special-mode',
which is also not covered by your "what makes special-mode
special": It is a "special" mode, in the sense described in
(elisp) `Major Mode Conventions' (this property has been
applied to symbol `special-mode').

This kind of specialness is not something new.  It's been
around since Day One, but not as a predefined mode
(`special-mode' was added in Emacs 23).  Here it is:

  If [a] mode is appropriate only for specially-prepared text
  produced by the mode itself (rather than by the user typing
  at the keyboard or by an external file), then the major mode
  command symbol should have a property named ‘mode-class’ with
  value ‘special’, put on as follows:

    (put 'funny-mode 'mode-class 'special)

  This tells Emacs that new buffers created while the current
  buffer is in Funny mode should not be put in Funny mode, even
  though the default value of ‘major-mode’ is ‘nil’.  By default,
  the value of ‘nil’ for ‘major-mode’ means to use the current
  buffer’s major mode when creating new buffers (*note Auto Major
  Mode::), but with such ‘special’ modes, Fundamental mode is
  used instead.  Modes such as Dired, Rmail, and Buffer List use
  this feature.

  The function ‘view-buffer’ does not enable View mode in buffers
  whose mode-class is special, because such modes usually provide
  their own View-like bindings.

  The ‘define-derived-mode’ macro automatically marks the derived
  mode as special if the parent mode is special.  Special mode
  is a convenient parent for such modes to inherit from; *Note
  Basic Major Modes::.

[Note that such "special" modes make use of Fundamental mode,
not `special-mode', for such new buffers.  Note too that the
use of  "Special mode" in the last sentence does not refer to
`special-mode'.  That's unfortunate/confusing, but it is the
fault of naming the new (in Emacs 23) mode `special-mode'.]

Dunno how many of the many uses of `make-sparse-keymap' (most,
if not all, of which correspond to your "what makes special-mode
special") have property `mode-class' = `special'.  Easy enough
to find out, if someone wants to bother.

Dired is one.  It is a "special" mode in the Day One sense
quoted above.  But it is not derived from `special-mode'.
It makes use of `make-sparse-keymap' in its own way, as it
should.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Yuri Khan]

On Wed, May 17, 2017 at 11:02 PM, Drew Adams <drew.adams@oracle.com> wrote:

> In the delivered Lisp sources for GNU Emacs 24.5 there are 46
> instances of deriving from `special-mode'.  And there are 829
> instances of invoking `make-sparse-keymap'.

How many of these 829 are for minor modes, transient uses and menus?

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

>> What makes special-mode special is that ordinary character keys are
>> divorced from their usual self-insert meaning, instead, they are all
>> available for commands.
[...]
> What you say "makes special-mode special" has nothing specially
> to do with `special-mode'.  It's just `make-sparse-keymap'.

Huh?  make-sparse-keymap doesn't "divorced [ordinary characters] from
their usual self-insert meaning".
Are you maybe thinking of `suppress-keymap`?


        Stefan

RE: nxml-mode: Derive from prog-mode instead of text-mode

[Drew Adams]

> Huh?  make-sparse-keymap doesn't "divorced [ordinary characters] from
> their usual self-insert meaning". Are you maybe thinking of
> `suppress-keymap`?

Yes, sorry.  (There are 91 occurrences of `suppress-keymap'.)

And I didn't notice that `dired-mode-map' now sets its parent
to `special-mode-map' (which does `suppress-keymap', which
`dired-mode-map' used to do).

Since Emacs 24, Dired already essentially inherits from
`special-mode', without doing so explicitly.  It gets the
map, but not the hook, and it overwrites most of the map.
It won't be any more useless for it to just inherit from
`special-mode'.  Sorry for the noise.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> Since Emacs 24, Dired already essentially inherits from
> `special-mode', without doing so explicitly.

Indeed, that's because the `dired-mode` function currently can't just
use define-derived-mode (because it takes a `dir` argument), and
I kept the needed refactoring as a todo-item (one of those "todo after
I retire from retirement").


        Stefan

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Johan Bockgård]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> Since Emacs 24, Dired already essentially inherits from
>> `special-mode', without doing so explicitly.
>
> Indeed, that's because the `dired-mode` function currently can't just
> use define-derived-mode (because it takes a `dir` argument), and
> I kept the needed refactoring as a todo-item (one of those "todo after
> I retire from retirement").

One disadvantage (currently) of deriving from special-mode is that
define-derived-mode makes `foo-mode' an interactive command. This is
practically never appropriate for "special" modes.

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> Absolutely not, for Dired.  It should not derive from
> special-mode,

Why not?


        Stefan

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> This had me quite surprised. It seems very counter-intuitive, and 100%
> opposite to how I think it's perceived and used in the community.

It's also opposite to the reason why I added it, FWIW.


        Stefan

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Eli Zaretskii]

> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
> Cc: jostein@kjonigsen.net, Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> Date: Tue, 16 May 2017 12:34:15 +0200
> 
> So before anything else, I would like to assess if we can have a majority agreement that prog-mode is NOT an
> implementation detail, but rather an API both users and developers are encouraged to use for
> programming-related activities and customizations.
> 
> If we can agree on that, I think there are options for moving forward.

Beware: you are shifting the focus of this discussion from nxml-mode
and its ilk, which was the original issue, to an almost completely
unrelated one.  There be a lot of unrelated bikeshedding (which
actually already began).

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 1817 bytes --]

On Tue, May 16, 2017, at 08:54 PM, Eli Zaretskii wrote:
>> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
>> Cc: jostein@kjonigsen.net, Eli Zaretskii <eliz@gnu.org>, emacs-
>> devel@gnu.org>> Date: Tue, 16 May 2017 12:34:15 +0200
>> 
>> So before anything else, I would like to assess if we can have a
>> majority agreement that prog-mode is NOT an>> implementation detail, but rather an API both users and developers
>> are encouraged to use for>> programming-related activities and customizations.
>> 
>> If we can agree on that, I think there are options for moving
>> forward.> 
> Beware: you are shifting the focus of this discussion from nxml-mode
> and its ilk, which was the original issue, to an almost completely
> unrelated one.  There be a lot of unrelated bikeshedding (which
> actually already began).

Thanks Eli.

That's a  good point. A point well taken.

Trying to get things back to where we started, with a minimal of
bikeshedding, I think there's 3 obvious choices for what direction
we can go:
 * Keep everything as is. Keep nxml-mode derived from text-mode (and
   annoy whoever finds that unexpected)
 * Derive nxml-mode from prog-mode instead (and annoy whoever finds
   *that* unexpected)
 * Create a new intermediate mode for deriving (structured-text-mode,
   or similar) and use this for all structured and textual data-
   format files (and thus, nxml-mode). Let people make new hooks as
   they see fit.
Personally I feel like the third option may be the most "proper" one,
but if people for some reason should be opposed to that, I still think
the second option makes more sense than the first one.
So... Is there any obvious counter-arguments to the third option? Any
reasons we should definitely *not* do that?
--
 RegardsJostein




[-- Attachment #2: Type: text/html, Size: 2801 bytes --]

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]

On Tue, May 16, 2017, at 09:02 PM, Jostein Kjønigsen wrote:
> On Tue, May 16, 2017, at 08:54 PM, Eli Zaretskii wrote:
>> you are shifting the focus of this discussion from nxml-mode
>> and its ilk, which was the original issue, to an almost completely
>> unrelated one.  There be a lot of unrelated bikeshedding (which
>> actually already began).
> 
> Thanks Eli.
> 
> That's a  good point. A point well taken.
> 
> Trying to get things back to where we started, with a minimal of
> bikeshedding, I think there's 3 obvious choices for what direction
> we can go:>
>  * Keep everything as is. Keep nxml-mode derived from text-mode (and
>    annoy whoever finds that unexpected)
>  * Derive nxml-mode from prog-mode instead (and annoy whoever finds
>    *that* unexpected)
>  * Create a new intermediate mode for deriving (structured-text-mode,
>    or similar) and use this for all structured and textual data-
>    format files (and thus, nxml-mode). Let people make new hooks as
>    they see fit.> 
> Personally I feel like the third option may be the most "proper" one,
> but if people for some reason should be opposed to that, I still think
> the second option makes more sense than the first one.> 
> So... Is there any obvious counter-arguments to the third option? Any
> reasons we should definitely *not* do that?> 
> --
> Regards
> Jostein

Any more opinions on this since last time around?

--
Regards
Jostein


[-- Attachment #2: Type: text/html, Size: 2129 bytes --]

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

> I think the best option is to create a new structured-text-mode which
> inherits from both text-mode and prog-mode (and then have nxml-mode
> inherit from that mode).
>
> Those users who want to enable some functionality in text-mode but not
> in structured-text-mode can always use (derived-mode-p
> 'structured-text-mode) in the hook function.
> 
> 
>         Stefan

Would this mean that structured-text-mode would activate all
customization and hooks which users may have setup for both text-mode
and prog-mode?

I can't speak for the "officially" intended uses of these hooks
ofcourse, but for me these two are in direct conflict.

With this setup, I would have to use structured-text-mode-hook to
explicitly cancel out text-mode hook and potentially re-set
prog-mode-hook settings.

While I'm all OK with structured-text-mode as a concept, I think
deriving it from both text-mode and prog-mode would be an error.

At least not without a "different" text-mode-hook for "regular" text
(literal-text-mode?), so that we can avoid these kinds of conflicts.

So I just tried to sketch out what that would look like in a
inheritance-tree, and it did honestly end up a bit messy. Basically if
we're to go that far, IMO not deriving in the first place might
actually be both simpler and better.

--
Jostein

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> Would this mean that structured-text-mode would activate all
> customization and hooks which users may have setup for both text-mode
> and prog-mode?

It would run both hooks, and inherit from both keymaps, yes.

> I can't speak for the "officially" intended uses of these hooks
> ofcourse, but for me these two are in direct conflict.

Can you give us some examples of conflicts?

> With this setup, I would have to use structured-text-mode-hook to
> explicitly cancel out text-mode hook and potentially re-set
> prog-mode-hook settings.

You can also test (derived-mode-p 'prog-mode) in your text-mode-hook in
order to only run some code for the text-only-mode (and vice-versa).

> At least not without a "different" text-mode-hook for "regular" text
> (literal-text-mode?), so that we can avoid these kinds of conflicts.

I think that would be over-engineered.


        Stefan

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Jostein Kjønigsen]

> > I can't speak for the "officially" intended uses of these hooks
> > ofcourse, but for me these two are in direct conflict.
> 
> Can you give us some examples of conflicts?

One type of conflict is mode-state: For text-based modes I want to
flyspell enabled, but for programming-related modes I want it disabled.

Another type of conflict is keybindings: I like to plug in languagetool-
related functionality on keys consistent with what I typically have in
programming-modes. That means that targets of these bindings for these
keys differ in text-mode and prog-mode.

> > With this setup, I would have to use structured-text-mode-hook to
> > explicitly cancel out text-mode hook and potentially re-set
> > prog-mode-hook settings.
> 
> You can also test (derived-mode-p 'prog-mode) in your text-mode-hook in
> order to only run some code for the text-only-mode (and vice-versa).

That's an option too, and wouldn't really complicate my setup that much.

Good idea!

> I think that would be over-engineered.

In which case we agree.

--
Jostein

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> One type of conflict is mode-state: For text-based modes I want to
> flyspell enabled, but for programming-related modes I want it disabled.
>
> Another type of conflict is keybindings: I like to plug in languagetool-
> related functionality on keys consistent with what I typically have in
> programming-modes. That means that targets of these bindings for these
> keys differ in text-mode and prog-mode.

It seems to me that those conflicts aren't a real problem for the
combined text+prog mode: the same problems exist with the current setup
when you want prog-mode and you get text-mode (or vice versa), so using
a combined text+prog mode wouldn't make things really worse.

And I don't think there's a really better solution to this problem: you
currently want A for text-modes and B for prog-modes, so for modes which
sit halfway you're going to have to tweak your ~/.emacs to tell Emacs
which of A or B (or yet something else, maybe) you want, no matter how
we make these halfway modes behave.


        Stefan

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Stefan Monnier]

> Beware: you are shifting the focus of this discussion from nxml-mode
> and its ilk, which was the original issue, to an almost completely
> unrelated one.  There be a lot of unrelated bikeshedding (which
> actually already began).

I think the best option is to create a new structured-text-mode which
inherits from both text-mode and prog-mode (and then have nxml-mode
inherit from that mode).

Those users who want to enable some functionality in text-mode but not
in structured-text-mode can always use (derived-mode-p
'structured-text-mode) in the hook function.


        Stefan

Re: nxml-mode: Derive from prog-mode instead of text-mode

[Rolf Ade]

Jostein Kjønigsen <jostein@secure.kjonigsen.net> writes:
> One thing I find unintuitive about nxml-mode is how it derives from
> text-mode.
> [...]
> Could we consider changing nxml-mode to derive from prog-mode instead?

FWIW and as an anecdotical data point: I'm working since almost two
decades in this field using eamcs and I support this.

It's not only that it doesn't run prog-mode-hook, but it also does run
text-mode-hook, which is, what me disturbs more.