electric-indent-mode: abolition of `newline' function is not the Right Thing.

electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Alan Mackenzie]

Hi, Emacs.

`electric-indent-post-self-insert-function' effectively redefines
`newline' to `newline-and-indent'.  So, whilst `electric-indent-mode' is
enabled, there is no easy way to invoke the traditional functionality of
`newline'.  This is a Bad Thing.

I discovered this whilst trying `electric-indent-mode' in Text Mode.
I have a command which starts a new paragraph thusly:
(i) It calls `newline' and inserts, e.g. "(vi) ".
(ii) It calls `fill-paragraph' for the text in the new paragraph.

Unfortunately, `newline' indents the "(vi) " to the level of the
old paragraph, so that the following `fill-paragraph' joins the old and
new paragraphs together.  This is bad.

`newline-and-indent' exists as its own command, traditionally bound to
C-j.  A better way for `electric-indent-mode' to achieve the effect it
wants is to get users to rebind <CR> to `newline-and-indent' or suggest
the use of C-j.

`newline' must continue to exist, both as a command and as a lisp
function.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Stefan Monnier]

> I discovered this whilst trying `electric-indent-mode' in Text Mode.
> I have a command which starts a new paragraph thusly:
> (i) It calls `newline' and inserts, e.g. "(vi) ".

Use (insert "\n") rather than (newline).


        Stefan

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Alan Mackenzie]

'morning, Stefan!

On Sun, Oct 13, 2013 at 09:23:49AM -0400, Stefan Monnier wrote:
> > I discovered this whilst trying `electric-indent-mode' in Text Mode.
> > I have a command which starts a new paragraph thusly:
> > (i) It calls `newline' and inserts, e.g. "(vi) ".

> Use (insert "\n") rather than (newline).

:-)  Of course there are workarounds, but that's not the point.  There
are approximately 198 calls to `newline' in the Emacs Lisp sources, and
an unknown further number in other Lisp files and users' .emacsen.  Some
of these calls are going to get broken by the effective change to
`newline'.

Redefining `newline' as `newline-and-indent' whilst
`electric-indent-mode' is active is not the Right Thing to do.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Stefan Monnier]

> :-)  Of course there are workarounds, but that's not the point.  There
> are approximately 198 calls to `newline' in the Emacs Lisp sources, and
> an unknown further number in other Lisp files and users' .emacsen.  Some
> of these calls are going to get broken by the effective change to
> `newline'.

Right.  But the problem is that this is not specific to
electric-indent-mode: newline calls self-insert-command, which runs
expand-abbrev and (since Emacs24) post-self-insert-hook.  Any caller
which only wants a \n and no extra doo-dads should not call `newline'.

electric-indent-mode just makes this problem more apparent.


        Stefan

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Alan Mackenzie]

Hello, Stefan.

On Sun, Oct 13, 2013 at 12:22:33PM -0400, Stefan Monnier wrote:
> > :-)  Of course there are workarounds, but that's not the point.
> > There are approximately 198 calls to `newline' in the Emacs Lisp
> > sources, and an unknown further number in other Lisp files and users'
> > .emacsen.  Some of these calls are going to get broken by the
> > effective change to `newline'.

> Right.  But the problem is that this is not specific to
> electric-indent-mode: newline calls self-insert-command, which runs
> expand-abbrev and (since Emacs24) post-self-insert-hook.  Any caller
> which only wants a \n and no extra doo-dads should not call `newline'.

> electric-indent-mode just makes this problem more apparent.

Sorry Stefan, but that is disingenuous.  If you look again at
`electric-indent-post-self-insert-function', you will see that that
function deliberately converts `newline' into `newline-and-indent'.  It
even says so in the comments.  _This_ is the problem.

What is the justification for this extension in functionality of
`newline' to do the same as `newline-and-indent'?

And people calling `newline' do not simply want a \n.  They want
abbreviations to be processed, they want auto-fill to be run, trailing
space to be removed from otherwise blank lines, and so on.  What they
_don't_ want is indentation inserted on the new line.

Please take this indent-newline functionality out of
e-i-post-s-i-function (or let me do it).

Incidentally, each call of e-i-post-s-i-function creates a new marker,
bound to the let variable `before'.  This is not set to point nowhere
before `before' becomes unbound.  I think it probably should.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Stefan Monnier]

> And people calling `newline' do not simply want a \n.  They want
> abbreviations to be processed, they want auto-fill to be run, trailing
> space to be removed from otherwise blank lines, and so on.  What they
> _don't_ want is indentation inserted on the new line.

Not sure who is your "they".  In my experience, what I've seen users
want is "auto-indentation" and by that they mean that the text should be
kept indented when they hit RET and a few other keys.
E.g. a question I've seen many times is

  So I tried this Emacs thingy which claims to do indentation for me,
  but all my text stays at the left-margin.  What am I doing wrong.

and the answer I generally see given is:

  You want to bind RET to newline-and-indent.

Are these the majority, users?  I don't know.  But so far, you're the
only person who's complained about ?\n being in the default value of
electric-indent-chars, so I see no evidence that your preference in this
regard is representative.

> Please take this indent-newline functionality out of
> e-i-post-s-i-function (or let me do it).

Very easy: remove ?\n from electric-indent-chars.
IOW (setq-default electric-indent-chars nil) should do that for you.

> Incidentally, each call of e-i-post-s-i-function creates a new marker,
> bound to the let variable `before'.  This is not set to point nowhere
> before `before' becomes unbound.  I think it probably should.

Let's see if/when this leads to a concrete performance problem, and at
that point we'll see how to fix it.  Making a marker point nowhere is an
annoyance and is itself a costly operation (O(N) for N markers), so
I don't much like using the workaround you suggest.


        Stefan

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Alan Mackenzie]

Hello, Stefan.

On Mon, Oct 14, 2013 at 08:56:25AM -0400, Stefan Monnier wrote:
> > And people calling `newline' do not simply want a \n.  They want
> > abbreviations to be processed, they want auto-fill to be run, trailing
> > space to be removed from otherwise blank lines, and so on.  What they
> > _don't_ want is indentation inserted on the new line.

> Not sure who is your "they".

The people who have written Emacs Lisp code which calls `newline' (as
contrasted with `newline-and-indent').

> In my experience, what I've seen users want is "auto-indentation" and
> by that they mean that the text should be kept indented when they hit
> RET and a few other keys.

That is a different topic from this thread's.  The question which should
be asked, should have been asked some while ago, is what is the best way
of giving these users what they want.  The answer is not, and can not be,
to have an Emacs without the traditional functionality of `newline'.

[ .... ]

> But so far, you're the only person who's complained about ?\n being in
> the default value of electric-indent-chars, so I see no evidence that
> your preference in this regard is representative.

Have I complained about \n being in electric-indent-chars?  I don't think
I have, but might be mistaken.  My complaint, and I've been perfectly
clear about it, is the abolition of the traditional functionality of the
`newline' function whenever electric-indent-mode is enabled.  You haven't
really addressed this issue in your past few posts.

> > Please take this indent-newline functionality out of
> > e-i-post-s-i-function (or let me do it).

> Very easy: remove ?\n from electric-indent-chars.
> IOW (setq-default electric-indent-chars nil) should do that for you.

Is that your solution to the ~198 uses of `newline' in Emacs and
countless others in other code and users' .emacsen?  Each caller of
`newline' should first ensure that \n is not in electric-indent-chars?

> > Incidentally, each call of e-i-post-s-i-function creates a new marker,
> > bound to the let variable `before'.  This is not set to point nowhere
> > before `before' becomes unbound.  I think it probably should.

> Let's see if/when this leads to a concrete performance problem, and at
> that point we'll see how to fix it.  Making a marker point nowhere is an
> annoyance and is itself a costly operation (O(N) for N markers), so
> I don't much like using the workaround you suggest.

Well, it's not really my suggestion, it's standard advice in the Elisp
manual on page "Overview of Markers".  Without doing this, a buffer's
performance is gradually going to worsen as a heap of useless markers
accumulates, until such time as garbage collection eventually gets rid of
them.  The cost of making a marker point nowhere, less than once per
keystroke, is hardly high.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[martin rudalics]

 > And people calling `newline' do not simply want a \n.  They want
 > abbreviations to be processed, they want auto-fill to be run, trailing
 > space to be removed from otherwise blank lines, and so on.  What they
 > _don't_ want is indentation inserted on the new line.

What I want is RET to always indent the new line.  And when I'm at the
beginning of a line I want that RET opens a new line here and indents
it.  And that's why I'm hopelessly lost when I have to work with emacs
-Q.

martin

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Alan Mackenzie]

Hello, Martin.

On Tue, Oct 15, 2013 at 08:28:22PM +0200, martin rudalics wrote:
>  > And people calling `newline' do not simply want a \n.  They want
>  > abbreviations to be processed, they want auto-fill to be run, trailing
>  > space to be removed from otherwise blank lines, and so on.  What they
>  > _don't_ want is indentation inserted on the new line.

> What I want is RET to always indent the new line.  And when I'm at the
> beginning of a line I want that RET opens a new line here and indents
> it.  And that's why I'm hopelessly lost when I have to work with emacs
> -Q.

With respect, that is not the topic of this thread.  That topic is that
with electric-indent-mode enabled, there is now no command, also no
elisp function which has the traditional functionality of `newline'.
That is a significant loss to Emacs.  The change in the functionality of
`newline' will break existing elisp code which calls `newline'.  Also,
there will be no way for a user to set up traditional Emacs behaviour
wrt <ret> and C-j.

For what you want, `newline-and-indent' exists, as you know.

> martin
> 

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Stefan Monnier]

> With respect, that is not the topic of this thread.  That topic is that
> with electric-indent-mode enabled, there is now no command, also no
> elisp function which has the traditional functionality of `newline'.

Quick, without looking at the source code of newline: what does
`newline' do that (insert "\n") or C-q C-j doesn't do?

> That is a significant loss to Emacs.

I don't know what that is supposed to mean.


        Stefan

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Alan Mackenzie]

Hello, Stefan.

On Wed, Oct 16, 2013 at 01:59:07PM -0400, Stefan Monnier wrote:
> > With respect, that is not the topic of this thread.  That topic is that
> > with electric-indent-mode enabled, there is now no command, also no
> > elisp function which has the traditional functionality of `newline'.

> Quick, without looking at the source code of newline: what does
> `newline' do that (insert "\n") or C-q C-j doesn't do?

The answer you're not looking for is "It inserts a newline, whilst doing
lots of Right Things on the previous line that I don't need to worry
about too much.", but it's my preferred answer.

Answering you literally, I think newline invokes filling, abbreviation
expansion and possibly other things, but (insert "\n") and C-q C-j
don't.

> > That is a significant loss to Emacs.

> I don't know what that is supposed to mean.

For one thing, it would prevent somebody using a mode with {e-i-m using
\n} and retaining the classical difference betweeen <ret> and C-j.

For another thing, it will stop lots of Emacs Lisp from working.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[chad]

On 16 Oct 2013, at 11:58, Alan Mackenzie <acm@muc.de> wrote:
> 
> For another thing, it will stop lots of Emacs Lisp from working.

What specific packages/libraries/etc break with the current
electric-indent-mode enabled?

~Chad

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Daniel Colascione]

On 10/16/13 1:55 PM, chad wrote:
> On 16 Oct 2013, at 11:58, Alan Mackenzie <acm@muc.de> wrote:
>>
>> For another thing, it will stop lots of Emacs Lisp from working.
>
> What specific packages/libraries/etc break with the current
> electric-indent-mode enabled?
>

FWIW, I've seen a lot of crufty site-specific elisp that barely works as 
it is. (Monkeypatching cc-mode seems perennially popular.) Can we please 
try not to break it further? I don't see why we have to change newline's 
behavior in order to make newline-and-indent behavior the default, which 
I agree we should do.

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Alan Mackenzie]

On Wed, Oct 16, 2013 at 01:55:07PM -0700, chad wrote:
> On 16 Oct 2013, at 11:58, Alan Mackenzie <acm@muc.de> wrote:

> > For another thing, it will stop lots of Emacs Lisp from working.

> What specific packages/libraries/etc break with the current
> electric-indent-mode enabled?

Haven't a clue.  Some will though.  One of my personal defuns breaks,
with the following sequence:

(newline)
(insert "(vi)")
(fill-paragraph)

Because of the unwanted indentation attached to `insert', which indents
the new "(vi)" to the same level as the previous line of text, the
`fill-paragraph' joins it to the previous paragraph.

> ~Chad

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Stephen J. Turnbull]

Stefan Monnier writes:
 > > With respect, that is not the topic of this thread.  That topic is that
 > > with electric-indent-mode enabled, there is now no command, also no
 > > elisp function which has the traditional functionality of `newline'.
 > 
 > Quick, without looking at the source code of newline: what does
 > `newline' do that (insert "\n") or C-q C-j doesn't do?

    `newline' is an interactive built-in compiled Lisp function
      -- loaded from "/playpen/src/XEmacs/xemacs/lisp/simple.elc"
    (newline &optional N)

    Documentation:
    Insert a newline, and move to left margin of the new line if it's blank.
    The newline is marked with the text-property `hard'.
    With optional arg N, insert that many newlines.
    In Auto Fill mode, if no numeric arg, break the preceding line if it's
    long.

Look ma, no code browsing!  (N.B. The Emacs docstring is worded
differently but it says the same thing, modulo being more accurate
about `use-hard-newlines' and auto-fill-mode.)

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[martin rudalics]

> For what you want, `newline-and-indent' exists, as you know.

I've been talking about the command run by RET and that I'm lost when I
run emacs -Q and try to do some editing.  Otherwise, I don't care about
the standard behavior because I obviously have RET run my own command.
But if I were new to Emacs, I certainly would want RET to indent the new
line out of the box.

martin

Default behaviour of RET.

[Alan Mackenzie]

Hi, Martin.

On Wed, Oct 16, 2013 at 08:34:56PM +0200, martin rudalics wrote:
> > For what you want, `newline-and-indent' exists, as you know.

> I've been talking about the command run by RET and that I'm lost when I
> run emacs -Q and try to do some editing.  Otherwise, I don't care about
> the standard behavior because I obviously have RET run my own command.
> But if I were new to Emacs, I certainly would want RET to indent the new
> line out of the box.

I might agree with you, at least for programming modes.  I'm not so sure
about things like Text Mode.  But there must also be a ready way of doing
what RET currently does, inserting a new line without indenting it.

> martin

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default behaviour of RET.

[Eli Zaretskii]

> Date: Wed, 16 Oct 2013 19:26:42 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: emacs-devel@gnu.org
> 
> But there must also be a ready way of doing what RET currently does,
> inserting a new line without indenting it.

What's wrong with "C-q C-j"?

Default behaviour of RET.

[Stephen J. Turnbull]

Alan Mackenzie writes:
 > On Wed, Oct 16, 2013 at 08:34:56PM +0200, martin rudalics wrote:

 > > I've been talking about the command run by RET

Which is easy enough to change.

 > I might agree with you, at least for programming modes.  I'm not so sure
 > about things like Text Mode.

I suspect I would find it really annoying, by analogy with the overly
aggressive quote-prefixing that is done by `filladapt'.

 > But there must also be a ready way of doing what RET currently
 > does, inserting a new line without indenting it.

You're enabling the confusaholics.  If the user doesn't like what RET
does, the the user can bind it to a different function (or, more
likely, invoke a mode that does that for her and perhaps customize the
mode).  Despite the subject, the interesting issue is "what should
`newline' do when invoked from code?"

The traditional docstring says that it moves to the left margin and
handles auto-filling.  Eli's suggestion of `(insert "\n")' doesn't do
that, and it's not what `newline' does when corrupted by
`electric-shock-mode'.  But I think it's useful behavior, and I think
programs should be able to rely on it (as opposed to users who can
modify the behavior of `One-Flew-Over-the-Cuckoos-Nest-mode' by
removing ?\n, or not invoke the mode in the first place).

Of course Stefan is correct that in Emacs, nothing is reliable.  I
don't understand why he thinks it's a virtue, though.

Re: Default behaviour of RET.

[Stefan Monnier]

> The traditional docstring says that it moves to the left margin and
> handles auto-filling.  Eli's suggestion of `(insert "\n")' doesn't do
> that, and it's not what `newline' does when corrupted by
> `electric-shock-mode'.  But I think it's useful behavior, and I think
> programs should be able to rely on it (as opposed to users who can
> modify the behavior of `One-Flew-Over-the-Cuckoos-Nest-mode' by
> removing ?\n, or not invoke the mode in the first place).

I can't remember ever seeing a piece of code which wants "the Emacs-23
newline behavior".  Usually it either wants to (insert "\n") or it wants
to simulate hitting RET.

This discussion would benefit from actual examples of code that
fall into neither "do whatever RET does" nor "insert \n".


        Stefan

Re: Default behaviour of RET.

[Alan Mackenzie]

Hello, Stefan.

On Wed, Oct 16, 2013 at 08:47:04PM -0400, Stefan Monnier wrote:
> > The traditional docstring says that it moves to the left margin and
> > handles auto-filling.  Eli's suggestion of `(insert "\n")' doesn't do
> > that, and it's not what `newline' does when corrupted by
> > `electric-shock-mode'.  But I think it's useful behavior, and I think
> > programs should be able to rely on it (as opposed to users who can
> > modify the behavior of `One-Flew-Over-the-Cuckoos-Nest-mode' by
> > removing ?\n, or not invoke the mode in the first place).

> I can't remember ever seeing a piece of code which wants "the Emacs-23
> newline behavior".  Usually it either wants to (insert "\n") or it wants
> to simulate hitting RET.

I gave an example of such code in the post which opened this thread.  To
recap:

(newline)
(insert "(vi)")
(fill-paragraph)

The `-and-indent' behaviour messes up the indentation of "(vi)" and
causes it to get re-attached to the previous paragraph.

> This discussion would benefit from actual examples of code that
> fall into neither "do whatever RET does" nor "insert \n".

See above.  Any code which is interested in filling (or, possibly, even
margins, if anything actually uses these) will get broken by the
`-and-indent'.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default behaviour of RET.

[Stefan Monnier]

>> > The traditional docstring says that it moves to the left margin and
>> > handles auto-filling.  Eli's suggestion of `(insert "\n")' doesn't do
>> > that, and it's not what `newline' does when corrupted by
>> > `electric-shock-mode'.  But I think it's useful behavior, and I think
>> > programs should be able to rely on it (as opposed to users who can
>> > modify the behavior of `One-Flew-Over-the-Cuckoos-Nest-mode' by
>> > removing ?\n, or not invoke the mode in the first place).
>> I can't remember ever seeing a piece of code which wants "the Emacs-23
>> newline behavior".  Usually it either wants to (insert "\n") or it wants
>> to simulate hitting RET.
> I gave an example of such code in the post which opened this thread.  To
> recap:

> (newline)
> (insert "(vi)")
> (fill-paragraph)

> The `-and-indent' behaviour messes up the indentation of "(vi)" and
> causes it to get re-attached to the previous paragraph.

I see why the above doesn't want the "simulate RET" behavior.
But I don't see why it wouldn't work just fine with the (insert
"\n") behavior?

>> This discussion would benefit from actual examples of code that
>> fall into neither "do whatever RET does" nor "insert \n".
> See above.  Any code which is interested in filling (or, possibly, even
> margins, if anything actually uses these) will get broken by the
> `-and-indent'.

"any code which..." is not concrete.


        Stefan

Re: Default behaviour of RET.

[Alan Mackenzie]

Hello, Stefan.

On Fri, Oct 18, 2013 at 03:52:22PM -0400, Stefan Monnier wrote:
> >> > The traditional docstring says that it moves to the left margin and
> >> > handles auto-filling.  Eli's suggestion of `(insert "\n")' doesn't do
> >> > that, and it's not what `newline' does when corrupted by
> >> > `electric-shock-mode'.  But I think it's useful behavior, and I think
> >> > programs should be able to rely on it (as opposed to users who can
> >> > modify the behavior of `One-Flew-Over-the-Cuckoos-Nest-mode' by
> >> > removing ?\n, or not invoke the mode in the first place).

> >> I can't remember ever seeing a piece of code which wants "the Emacs-23
> >> newline behavior".  Usually it either wants to (insert "\n") or it wants
> >> to simulate hitting RET.

> > I gave an example of such code in the post which opened this thread.  To
> > recap:

> > (newline)
> > (insert "(vi)")
> > (fill-paragraph)

> > The `-and-indent' behaviour messes up the indentation of "(vi)" and
> > causes it to get re-attached to the previous paragraph.

> I see why the above doesn't want the "simulate RET" behavior.
> But I don't see why it wouldn't work just fine with the (insert
> "\n") behavior?

The `newline' tidies up the previous paragraph, in particular it fills
it.  It might also cause abbreviations to be expanded.

I could easily adapt this piece of code to cope with the `-and-indent'
which has been attached to `newline', but that's not the point.

The point is that there are a near infinite number of invocations of
`newline' in the wild, and an unknown proportion of these, possibly
quite high, will depend on `newline' doing precisely what its doc string
says.  Their existence makes the utility of electric-indent-mode, at the
moment, questionable.

> >> This discussion would benefit from actual examples of code that
> >> fall into neither "do whatever RET does" nor "insert \n".

> > See above.  Any code which is interested in filling (or, possibly, even
> > margins, if anything actually uses these) will get broken by the
> > `-and-indent'.

> "any code which..." is not concrete.

I've given you a concrete example of such code, several times.  It's not
a particularly bizarre piece of code, either.

electric-indent-mode's indentation of a new line is broken.  Please take
this behaviour out.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default behaviour of RET.

[Stefan Monnier]

> The point is that there are a near infinite number of invocations of
> `newline' in the wild, and an unknown proportion of these, possibly
> quite high, will depend on `newline' doing precisely what its doc string

but also possibly quite low.

> says.  Their existence makes the utility of electric-indent-mode, at the
> moment, questionable.

As long as we're in the "unknown proportion", we can't tell.

>> >> This discussion would benefit from actual examples of code that
>> >> fall into neither "do whatever RET does" nor "insert \n".
>> > See above.  Any code which is interested in filling (or, possibly, even
>> > margins, if anything actually uses these) will get broken by the
>> > `-and-indent'.
>> "any code which..." is not concrete.
> I've given you a concrete example of such code, several times.

Right, but I said "examples" not "example".  So far I've seen one
concrete example, from your ~/.emacs (i.e. a problem that will affect
O(1) user).  That's not very compelling.


        Stefan

Re: Default behaviour of RET.

[Alan Mackenzie]

Hello, Stefan.

On Fri, Oct 18, 2013 at 09:59:54PM -0400, Stefan Monnier wrote:
> > The point is that there are a near infinite number of invocations of
> > `newline' in the wild, and an unknown proportion of these, possibly
> > quite high, will depend on `newline' doing precisely what its doc string

> but also possibly quite low.

Just how low does it have to be before we can ignore it?

> > says.  Their existence makes the utility of electric-indent-mode, at the
> > moment, questionable.

> As long as we're in the "unknown proportion", we can't tell.

> >> >> This discussion would benefit from actual examples of code that
> >> >> fall into neither "do whatever RET does" nor "insert \n".

> >> > See above.  Any code which is interested in filling (or, possibly, even
> >> > margins, if anything actually uses these) will get broken by the
> >> > `-and-indent'.

> >> "any code which..." is not concrete.

> > I've given you a concrete example of such code, several times.

> Right, but I said "examples" not "example".  So far I've seen one
> concrete example, from your ~/.emacs (i.e. a problem that will affect
> O(1) user).  That's not very compelling.

OK.  I've done a find-grep in .../lisp for all *.el matching
'^[^;\n]*(newline\([ )]\|$\)'.  There are 199 matches.

There are 27 occurences of the exact string "(or (bolp) (newline))",
which looks like making sure point is at BOL in an output file.  For
example, in uce.el, we have the following code at L359:

        ;; If message doesn't end with a newline, insert it.
	(goto-char (point-max))
	(or (bolp) (newline)))

. In org-datetree.el, we have this, at L83:

      (goto-char (point-max))
      (or (bolp) (newline))
      (org-datetree-insert-line year))

.  This same idiom appears two other times in this file alone.  I looked
at quite a few other occurences of "(or (bolp) (newline))" and all the
ones I could understand instantly were about ensuring point is at BOL, as
indeed was my own personal code.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

RE: Default behaviour of RET.

[Drew Adams]

> OK.  I've done a find-grep in .../lisp for all *.el matching
> '^[^;\n]*(newline\([ )]\|$\)'.  There are 199 matches.
> There are 27 occurences of the exact string "(or (bolp) (newline))",
> which looks like making sure point is at BOL in an output file.

I haven't been following this thread; sorry.  I don't have any calls
to `newline' in my code, FWIW.

Is the following a reasonable summary of the question & positions?

1. One position wants to change the behavior of `newline', so that
all uses of it automatically benefit from the new behavior.

2. The other position wants to keep `newline' as it is, perhaps for
some existing calls and in any case for some future calls.  This
position holds that the "old" behavior can be useful in some contexts.

Sounds like two different functions are in order, for the old and new
behaviors - au choix.  If so, it's not a big deal to update the Emacs
sources one way or the other.

But in case there is 3rd-party code that uses `newline', it seems
like the prudent choice would be to keep the existing name for the
existing behavior and give the new behavior a new name.  3rd-party
code that wants to "upgrade" to the new behavior can do that.  And
there would be no surprises.

Is there more to it that this?

Re: Default behaviour of RET.

[Alan Mackenzie]

[ personal mail ]

Hi, Drew.

On Sat, Oct 19, 2013 at 08:07:35AM -0700, Drew Adams wrote:
> > OK.  I've done a find-grep in .../lisp for all *.el matching
> > '^[^;\n]*(newline\([ )]\|$\)'.  There are 199 matches.
> > There are 27 occurences of the exact string "(or (bolp) (newline))",
> > which looks like making sure point is at BOL in an output file.

> I haven't been following this thread; sorry.  I don't have any calls
> to `newline' in my code, FWIW.

> Is the following a reasonable summary of the question & positions?

> 1. One position wants to change the behavior of `newline', so that
> all uses of it automatically benefit from the new behavior.

> 2. The other position wants to keep `newline' as it is, perhaps for
> some existing calls and in any case for some future calls.  This
> position holds that the "old" behavior can be useful in some contexts.

More or less.

> Sounds like two different functions are in order, for the old and new
> behaviors - au choix.  If so, it's not a big deal to update the Emacs
> sources one way or the other.

I think I've managed to persuade SM of that, now.  But I'm not convinced
his latest patch is the right thing.  It's probably the best I'll manage,
though.

> But in case there is 3rd-party code that uses `newline', it seems
> like the prudent choice would be to keep the existing name for the
> existing behavior and give the new behavior a new name.  3rd-party
> code that wants to "upgrade" to the new behavior can do that.  And
> there would be no surprises.

That command already exists, and is called `newline-and-indent'.

> Is there more to it that this?

'Fraid so!  The mechanism by which `newline' was changed was by adding a
function to `post-self-insert-hook' when electric-indent-mode is active,
and this function checked whether a newline had been added, and indented
the new line if it had.  My task, which seems to have been inordinately
difficult, was to persuade SM that this roundabout means of turning
`newline' into `newline-and-indent' is just as bad as the direct means.
It's been exhausting.  I hope the argument has now been won.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default behaviour of RET.

[Richard Stallman]

        [ To any NSA and FBI agents reading my email: please consider
        [ whether defending the US Constitution against all enemies,
        [ foreign or domestic, requires you to follow Snowden's example.

I don't think the default behavior of RET should be changed, even in
programming modes.  We have two commands, RET and C-j, which insert a newline.
C-j indents, and RET doesn't.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Default behaviour of RET.

[Eli Zaretskii]

> Date: Sun, 20 Oct 2013 18:26:31 -0400
> From: Richard Stallman <rms@gnu.org>
> Cc: rudalics@gmx.at, stephen@xemacs.org, monnier@iro.umontreal.ca,
> 	drew.adams@oracle.com, emacs-devel@gnu.org
> 
> I don't think the default behavior of RET should be changed, even in
> programming modes.  We have two commands, RET and C-j, which insert a newline.
> C-j indents, and RET doesn't.

The days when keyboards had an LFD key that produced C-j are long
gone.  Nowadays, keyboards have only the RET key, and users expect it
to indent in programming modes, at least by default.  As things are
now, I need to customize every programming mode I use to do that.  I
think it's time for a change.

Re: Default behaviour of RET.

[Stefan Monnier]

>> but also possibly quite low.
> Just how low does it have to be before we can ignore it?

That was not the question.

Anyway, it does look like there are indeed many incorrect uses of
newline (most of those (or (bolp) (newline)) will break sooner or later
because of unexpected filling and/or abbrev/skeleton expansion).

So I installed a patch which makes `newline' only run
post-self-insert-hook when called interactively, so as to try and
preserve compatibility with those dubious uses.


        Stefan

Re: Default behaviour of RET.

[Alan Mackenzie]

Hello, Stefan.

On Sat, Oct 19, 2013 at 06:20:24PM -0400, Stefan Monnier wrote:
> >> but also possibly quite low.
> > Just how low does it have to be before we can ignore it?

> That was not the question.

> Anyway, it does look like there are indeed many incorrect uses of
> newline (most of those (or (bolp) (newline)) will break sooner or later
> because of unexpected filling and/or abbrev/skeleton expansion).

We don't know that the filling, etc., is unexpected.  The crucial point
is that all these secondary actions of `newline' happen on the _previous_
line, leaving the main point of newline: "Insert a newline, and move to
left margin of the new line ...." unaffected.

> So I installed a patch which makes `newline' only run
> post-self-insert-hook when called interactively, so as to try and
> preserve compatibility with those dubious uses.

Thanks.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default behaviour of RET.

[Alan Mackenzie]

Hi, Stephen.

On Thu, Oct 17, 2013 at 08:17:01AM +0900, Stephen J. Turnbull wrote:
> Alan Mackenzie writes:
>  > On Wed, Oct 16, 2013 at 08:34:56PM +0200, martin rudalics wrote:

>  > > I've been talking about the command run by RET

> Which is easy enough to change.

>  > I might agree with you, at least for programming modes.  I'm not so sure
>  > about things like Text Mode.

> I suspect I would find it really annoying, by analogy with the overly
> aggressive quote-prefixing that is done by `filladapt'.

>  > But there must also be a ready way of doing what RET currently
>  > does, inserting a new line without indenting it.

> You're enabling the confusaholics.  If the user doesn't like what RET
> does, the the user can bind it to a different function (or, more
> likely, invoke a mode that does that for her and perhaps customize the
> mode).  Despite the subject, the interesting issue is "what should
> `newline' do when invoked from code?"

Indeed, that was in the title of the original post in the thread.  But
you seem to have misunderstood the situation: the current
electric-indent-mode effectively transforms `newline' into
`newline-and-indent', leaving no command for the user who doesn't like
this to bind RET to.

> The traditional docstring says that it moves to the left margin and
> handles auto-filling.  Eli's suggestion of `(insert "\n")' doesn't do
> that, and it's not what `newline' does when corrupted by
> `electric-shock-mode'.  But I think it's useful behavior, and I think
> programs should be able to rely on it (as opposed to users who can
> modify the behavior of `One-Flew-Over-the-Cuckoos-Nest-mode' by
> removing ?\n, or not invoke the mode in the first place).

Yes.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Matthias Meulien]

> (...) I discovered this whilst trying `electric-indent-mode' in 
> Text Mode. 

For the same reason it is impossible to write Python code with 
both `python-skeleton-autoinsert' set to t and Electric Indent 
mode enabled...

Say a buffer contains the following three lines:

class base(object): 
    pass 

-!-

Now you hit c l a s s SPC d e r i v e d RET b a s e RET RET
which results in:

class base(object): 
    pass 

    class derived(base):
        """-!-""" 

Unexpected isn't it?
-- 
Matthias Meulien

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Stefan Monnier]

> For the same reason it is impossible to write Python code with both
> python-skeleton-autoinsert' set to t and Electric Indent mode enabled...

> Say a buffer contains the following three lines:

> class base(object):  pass 

> -!-

> Now you hit c l a s s SPC d e r i v e d RET b a s e RET RET
> which results in:

> class base(object):  pass 

>    class derived(base):
>        """-!-""" 

> Unexpected isn't it?

I can't reproduce it on trunk, probably thanks to the new
electric-indent-inhibit, which is set in python-mode.


        Stefan

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Matthias Meulien]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

> (...) I can't reproduce it on trunk, probably thanks to the new
> electric-indent-inhibit, which is set in python-mode.

Good news! I encounter the problem I described with 24.3.1. Note that my
comment was misleading: It has nothing to do with
`python-skeleton-autoinsert'... And my post was badly formatted (I don't
know why): The `pass' keyword must be on its own indented line, like this:

class base(object):
>       pass
>

Sorry for the noise and many thanks!
-- 
Matthias

[-- Attachment #2: Type: text/html, Size: 763 bytes --]

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Stefan Monnier]

>> (...) I can't reproduce it on trunk, probably thanks to the new
>> electric-indent-inhibit, which is set in python-mode.

> Good news! I encounter the problem I described with 24.3.1. Note that my
> comment was misleading: It has nothing to do with
> `python-skeleton-autoinsert'... And my post was badly formatted (I don't
> know why): The `pass' keyword must be on its own indented line, like this:

> class base(object):
>> pass
>> 

Hmm, then I don't know if it happens in trunk or not, because I don't
really know how to reproduce it or what result would be correct or not
(I don't know/use Python).


        Stefan

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Davis Herring]

>> class base(object):
>>> pass
> 
> Hmm, then I don't know if it happens in trunk or not, because I don't
> really know how to reproduce it or what result would be correct or not
> (I don't know/use Python).

I can help with the basic idea: most colons require a following
statement, either on the same line (one statement) or on all the
following lines indented more than the line with the colon.  (We're only
concerned with the latter case, which Python calls a suite.)

"pass" is a no-op used in such a context where no (real) statement is
desired.  Therefore, it makes sense for an Emacs Python mode to notice a
"pass" (along with certain other choices like "return") and presume that
the user wants it to end the suite -- that is, to use one fewer level of
indentation on the next line.  (Sometimes the user wants even less
indentation, so they type p a s s C-j DEL or so.)

The bug (if it exists) is that newline/newline-and-indent/whatever are
not implementing this behavior properly, resulting in blocks that the
user "obviously" means to be siblings being nested instead (when no DEL
is used).

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Matthias Meulien]

>>> (...) I can't reproduce it on trunk, probably thanks to the 
>>> new electric-indent-inhibit, which is set in python-mode.

You are right: I've compiled the trunk (git revision: b670693) and it
works as expected!
-- 
Matthias

Re: electric-indent-mode: abolition of `newline' function is not the Right Thing.

[Stefan Monnier]

> You are right: I've compiled the trunk (git revision: b670693) and it
> works as expected!

Thank you for confirming,


        Stefan

RE: Default behaviour of RET.

[Drew Adams]

> The days when keyboards had an LFD key that produced C-j are long
> gone.  Nowadays, keyboards have only the RET key, and users expect
> it to indent in programming modes, at least by default.  As things
> are now, I need to customize every programming mode I use to do that.  

You do?  You can't use `C-j'?  Are you bemoaning the disappearance
of a key that did the same thing key `C-j' does?  And yet you feel
you need to make `RET' do that?  Not so clear.

If you still had a `LFD' key would you nevertheless feel the "need to
customize every programming mode I use" to make `RET' do what `LFD'
did?

And which users are you talking about, who have this expectation?
Emacs users?

Re: Default behaviour of RET.

[Rustom Mody]

On Mon, Oct 21, 2013 at 8:56 AM, Drew Adams <drew.adams@oracle.com> wrote:
>> The days when keyboards had an LFD key that produced C-j are long
>> gone.  Nowadays, keyboards have only the RET key, and users expect
>> it to indent in programming modes, at least by default.  As things
>> are now, I need to customize every programming mode I use to do that.
>
> You do?  You can't use `C-j'?

RET is one keystroke; C-j is two

Rusi
-- 
http://www.the-magus.in
http://blog.languager.org

Re: Default behaviour of RET.

[Richard Stallman]

        [ To any NSA and FBI agents reading my email: please consider
        [ whether defending the US Constitution against all enemies,
        [ foreign or domestic, requires you to follow Snowden's example.

    RET is one keystroke; C-j is two

Lots of Emacs commands are control keys.

It would do no harm to have a minor mode that effectively swaps RET
and C-j, in all modes.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Default behaviour of RET.

[Stephen J. Turnbull]

Richard Stallman writes:

 > It would do no harm to have a minor mode that effectively swaps RET
 > and C-j, in all modes.

Thank you, Richard!

The bikeshedding can stop now.

Re: Default behaviour of RET.

[Kenichi Handa]

In article <CAJ+TeoeW-FBJb0wGMGmFcko-PRP-dvKx4gPk1CdRTaBbbZQPPQ@mail.gmail.com>, Rustom Mody <rustompmody@gmail.com> writes:

> On Mon, Oct 21, 2013 at 8:56 AM, Drew Adams <drew.adams@oracle.com> wrote:
>>> The days when keyboards had an LFD key that produced C-j are long
>>> gone.  Nowadays, keyboards have only the RET key, and users expect
>>> it to indent in programming modes, at least by default.  As things
>>> are now, I need to customize every programming mode I use to do that.
> >
> > You do?  You can't use `C-j'?

> RET is one keystroke; C-j is two

To me, C-j is easier to type than RET.  I can type C-j
without moving my palms from the home position (my left
little finger can type Ctrl-key (next to 'A') without moving
the left palm.  But, as there are one or two keys between
':' (the home position of the right little finter) and RET
key, I have to move my right palm to type RET.

Of course, I can use C-m instead of RET, but still C-j is
easier.

By the way, users expecting RET to indent don't expect C-f
and M-f to move cursor.  They will keep on using arrow keys
or mouse to navigate a buffer.  But I believe C-f/M-f and
etc. are far more convenient, and recommend users to use
them.  How one feels about C-j may change when he/she is
accustomed to, for instance, all the basic key bindings
shown in TUTORIAL.

---
Kenichi Handa
handa@gnu.org

Re: Default behaviour of RET.

[Eli Zaretskii]

> Date: Sun, 20 Oct 2013 20:26:00 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: acm@muc.de, rudalics@gmx.at, stephen@xemacs.org, monnier@iro.umontreal.ca,
>         emacs-devel@gnu.org
> 
> > I need to customize every programming mode I use to do that.  
> 
> You do?  You can't use `C-j'?

I can, but it's much less convenient than to use RET.

> Are you bemoaning the disappearance of a key that did the same thing
> key `C-j' does?

Not in particular.  And what would be the use? it's not like keyboard
designers will listen.

> And yet you feel you need to make `RET' do that?  Not so clear.

What's not clear?  I want a key that is easily reached and that
performs newline-and-indent.  Every modern IDE does that.

> If you still had a `LFD' key would you nevertheless feel the "need to
> customize every programming mode I use" to make `RET' do what `LFD'
> did?

Maybe, I don't know.  Last time I used that key was 30 years ago.

> And which users are you talking about, who have this expectation?
> Emacs users?

Users of any modern IDE, which I think includes Emacs.

RE: Default behaviour of RET.

[Drew Adams]

> > > I need to customize every programming mode I use to do that.
> >
> > You do?  You can't use `C-j'?
> 
> I can, but it's much less convenient than to use RET.

Much less convenient, really?  Do you find `C-n' as inconvenient as
`C-j', so you customize Emacs to replace `C-n' also?  `C-a'?  `C-e'?

> I want a key that is easily reached and that
> performs newline-and-indent.  Every modern IDE does that.

Emacs does that too, with `C-j'.  Easily reached by most, I think.

But any user is free to bind some other key for this.  If one finds
`<f9>' or `<kp-enter>' (or even `RET') better, then it's easy enough
to substitute it.

Sorry, I'm with Richard on this one.  You seem to be viewing a local
mole hill (if even that) from a foreign vantage point and seeing a
mountain.  `C-j' is a good choice for Emacs, here.  Let's keep `RET'
for what it does and `C-j' for what it does.

Re: Default behaviour of RET.

[Dmitry Gutov]

Drew Adams <drew.adams@oracle.com> writes:

>> I want a key that is easily reached and that
>> performs newline-and-indent.  Every modern IDE does that.
>
> Emacs does that too, with `C-j'.  Easily reached by most, I think.

Every IDE does that when the user presses RET.

> But any user is free to bind some other key for this.  If one finds
> `<f9>' or `<kp-enter>' (or even `RET') better, then it's easy enough
> to substitute it.

Yes, we've heard this argument loads of times already. Then'll also be
able to rebind RET back to `newline', or disable `electric-indent-mode',
or whatever. The question is what would be the sane default.

> Sorry, I'm with Richard on this one.  You seem to be viewing a local
> mole hill (if even that) from a foreign vantage point and seeing a
                                  ^^^^^^^^^^^^^^^^^^^^^
> mountain.

You seem to be under impression that Eli is somehow new to using Emacs.

> `C-j' is a good choice for Emacs, here.  Let's keep `RET'
> for what it does and `C-j' for what it does.

I really have to wonder when anyone would wish to use RET bound to
newline. Why? Does some popular major mode provide inadequate
indentation function, so that you have to pick whether to indent the
next line automatically or not?

RE: Default behaviour of RET.

[Drew Adams]

> >> I want a key that is easily reached and that
> >> performs newline-and-indent.  Every modern IDE does that.
> >
> > Emacs does that too, with `C-j'.  Easily reached by most, I think.
> 
> Every IDE does that when the user presses RET.

Not if you count Emacs as an IDE. ;-)

> The question is what would be the sane default.

The default Emacs behavior for this is quite sane, and has been so
for almost 40 years now.

That it is not the same as the default behavior of this or that
other application does not make the Emacs default behavior insane.

> > Sorry, I'm with Richard on this one.  You seem to be viewing a
> > local mole hill (if even that) from a foreign vantage point and
>                                         ^^^^^^^^^^^^^^^^^^^^^
> > seeing a mountain.
>
> You seem to be under impression that Eli is somehow new to using
> Emacs.

You seem to be fantasizing.  The "foreign vantage point" is an
outside view, nothing more.  That is the argument, no?  "All the
other guys are doing it another way."

If you view Emacs `C-j' from the point of view of "every IDE" then
I guess it is possible to find it "much less convenient" than `RET'.
Otherwise, it is not.

Maybe it is a very teeny tiny bit less convenient - only a mole
hill, at most.  Certainly not "much less" convenient.  If it were
then Emacs would never have adopted `C-j' for this often-used
command.  Do you think those who did so felt that it was "much
less convenient" but went ahead and made it the default anyway?
Or do you think that they were insensitive to user convenience?

There is nothing new that makes the difference in convenience
between `C-j' and `RET' any greater now than it has been at any
point in the past.  Exactly the same difference: same mole hill.

For Emacs, `C-j' has been considered convenient for this behavior
for a very long time.  And I, for one, still find it convenient.
It doesn't get much more convenient than `C-j'.  Circulez ; il
n'y a rien a voir.

Re: Default behaviour of RET.

[Dmitry Gutov]

On 22.10.2013 00:53, Drew Adams wrote:
>> Every IDE does that when the user presses RET.
>
> Not if you count Emacs as an IDE. ;-)

Only if you like splitting hairs.
Counting Emacs among points of reference for proposed change in Emacs 
behavior is useless.

>> The question is what would be the sane default.
>
> The default Emacs behavior for this is quite sane, and has been so
> for almost 40 years now.

It is sane because it is old? That's a bad argument.

> That it is not the same as the default behavior of this or that
> other application does not make the Emacs default behavior insane.

The Emacs default settings are considered outdated by many of its users 
(here's one tiny example: 
https://github.com/technomancy/better-defaults). So discussing changes 
in long-established behaviors can be useful.

>> You seem to be under impression that Eli is somehow new to using
>> Emacs.
>
> You seem to be fantasizing.  The "foreign vantage point" is an
> outside view, nothing more.

Yeah, and who's outside? One of the most prolific Emacs contributors?

> That is the argument, no?  "All the
> other guys are doing it another way."

Read it this way: "won't somebody think of the poor new users?"

And: "quite often standard behavior emerges for a reason".

> If you view Emacs `C-j' from the point of view of "every IDE" then
> I guess it is possible to find it "much less convenient" than `RET'.
> Otherwise, it is not.

It's ~twice as inconvenient, for obvious reasons.

> There is nothing new that makes the difference in convenience
> between `C-j' and `RET' any greater now than it has been at any
> point in the past.  Exactly the same difference: same mole hill.

It's not the greatest of problems one might find with Emacs, true.

> For Emacs, `C-j' has been considered convenient for this behavior
> for a very long time.  And I, for one, still find it convenient.
> It doesn't get much more convenient than `C-j'.  Circulez ; il
> n'y a rien a voir.

You still haven't answered why you want RET to be bound to `newline'.

Re: Default behaviour of RET.

[chad]

On 21 Oct 2013, at 13:53, Drew Adams <drew.adams@oracle.com> wrote:
> 
> For Emacs, `C-j' has been considered convenient for this behavior
> for a very long time. 

This claim is made with no more support than Eli's counterclaim,
and it is stated as if it were a fact. We've already seen multiple
claims to the contrary (some because you repeated the question, and
some because other people also disagreed with you). The people
making those claims attempted to provide support, which you nitpicked
(as expected) and then discounted.

> Certainly not "much less" convenient.


Either we're going to poll the users, or we aren't and instead are
going to rely on developer opinion. In both cases, positive statements
like "I find it convenient." are valuable. In neither case does it
seem valuable or worthwhile to disparage or discount the opinions
of other people simply because you disagree.

If there's a vote to be made, I have helped hundreds of users (mostly
MIT students) who wanted RET to do newline-and-indent by default
in programming modes. It's easy enough to change if it's not the
default, but I certainly think it's a better default.

I hope that helps,
~Chad

Re: Default behaviour of RET.

[Daniel Colascione]

On 10/21/13 3:03 PM, chad wrote:
> If there's a vote to be made, I have helped hundreds of users (mostly
> MIT students) who wanted RET to do newline-and-indent by default
> in programming modes. It's easy enough to change if it's not the
> default, but I certainly think it's a better default.

I agree. It's not by coincidence that every other popular editing 
environment indents on RET. Why would you _want_ to insert a new line 
and not indent it?

While we're at it, can we bind C-l and C-o to more generally useful 
commands?

RE: Default behaviour of RET.

[Drew Adams]

> While we're at it, can we bind C-l and C-o to more generally useful
> commands?

C'est parti.  Here we go...

Re: Default behaviour of RET.

[Lars Brinkhoff]

Daniel Colascione <dancol@dancol.org> writes:
> While we're at it, can we bind C-l and C-o to more generally useful
> commands?

I'm an Emacs user, and I use those all the time.  It would be
especially annoying if C-l started doing something else.

But then, eval-expression moved from M-ESC to M-:, and I got over it.
Eventually.

Re: Default behaviour of RET.

[Alan Mackenzie]

Hi, Daniel.

On Mon, Oct 21, 2013 at 03:12:49PM -0700, Daniel Colascione wrote:
> I agree. It's not by coincidence that every other popular editing 
> environment indents on RET. Why would you _want_ to insert a new line 
> and not indent it?

If you're in text mode, or you're in a programming mode and want, for
example, some comments to start at column zero.

> While we're at it, can we bind C-l and C-o to more generally useful 
> commands?

NO!!  I use these commands a lot.  :-)

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default behaviour of RET.

[Davis Herring]

> If there's a vote to be made, I have helped hundreds of users (mostly
> MIT students) who wanted RET to do newline-and-indent by default
> in programming modes. It's easy enough to change if it's not the
> default, but I certainly think it's a better default.

The question of RET's default behavior is not independent of the
question of that of C-j.  It is manifestly more useful to expose
different functionality on different keys.  (Even though users can
customize one of them away, the defaults have their usual importance
when you're not at your home machine/account/whatever.)

Is the proposal to make RET and C-j synonymous despite this?  In the
abstract, it would actually make sense to interchange them: then C-j
could just be a self-inserting character.  It would even make sense for
lisp-interaction-mode's current binding: you rarely would want to eval
the preceding sexp and add its value into an enclosing sexp (i.e., where
indentation would make a difference).

For my own use case, which involves a lot of Python, I like having a
non-indenting newline available because it saves having to dedent
manually with DEL (or M-\) when I know I want to return to top-level.
(This detail is of course not important in {} or () languages, where a
trailing ) can be considered or an electric } can dedent for you.)

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

RE: Default behaviour of RET.

[Drew Adams]

> The question of RET's default behavior is not independent of the
> question of that of C-j.  It is manifestly more useful to expose
> different functionality on different keys.  (Even though users can
> customize one of them away, the defaults have their usual importance
> when you're not at your home machine/account/whatever.)

Exactly.  And that's the main point Richard was making, IMO.  The
only other thing he said was that he personally did not think that
the default behavior of RET should be changed.  The more important
point is to keep two behaviors easily accessible.
 
> Is the proposal to make RET and C-j synonymous despite this?

That is unclear to me too, but I too got that impression, and that's
why I responded.  We should keep both behaviors, IMO.

> In the abstract, it would actually make sense to interchange them:
> then C-j could just be a self-inserting character.

FWIW, that works for me.  (I already have C-j self-insert elsewhere, 
including in the minibuffer.)

> It would even make sense for lisp-interaction-mode's current
> binding: you rarely would want to eval the preceding sexp and
> add its value into an enclosing sexp (i.e., where
> indentation would make a difference).

There too, I'm with you.  (In fact, I put *scratch* into Emacs-Lisp
mode because I hate having RET evaluate a sexp.)

> For my own use case, which involves a lot of Python, I like having
> a non-indenting newline available because it saves having to dedent
> manually with DEL (or M-\) when I know I want to return to top-
> level.

I too use RET sometimes, but I can't say offhand when it is that I do.

Re: Default behaviour of RET.

[Jorgen Schaefer]

On Mon, 21 Oct 2013 15:03:10 -0700
chad <yandros@MIT.EDU> wrote:

> Either we're going to poll the users, or we aren't and instead are
> going to rely on developer opinion. In both cases, positive statements
> like "I find it convenient." are valuable. In neither case does it
> seem valuable or worthwhile to disparage or discount the opinions
> of other people simply because you disagree.

Thank you for this very sensible point.

To add my .2 cents:

If the default binding of RET changes, please do make sure that there
is a simple and easily accessible way of toggling this, to enable
correct pasting of multi-line text in tty versions of Emacs.

To me, this side effect is the one understandable reason for RET's
current behavior (outside of "I'd have to rewire my muscle memory!").
But it's certainly not something we couldn't work around. We just
shouldn't forget to. :-)

Regards,
	-- Jorgen

Re: Default behaviour of RET.

[Stefan Monnier]

> If the default binding of RET changes, please do make sure that there
> is a simple and easily accessible way of toggling this, to enable
> correct pasting of multi-line text in tty versions of Emacs.

Indeed, very good point.
So we have the following issues if we want to enable electric-indent-mode:
- C-j's default binding becomes useless.
- need for a new tty-paste command.
I recently wrote a tty-paste command (the main purpose of which was to
speed up the process rather than to avoid RET's indentation), but it
would need some polishing.  Even better would be if we could auto-detect
a "tty paste", tho this would inevitably be unreliable since the only
difference is timing.


        Stefan

Re: Default behaviour of RET.

[Richard Stallman]

        [ To any NSA and FBI agents reading my email: please consider
        [ whether defending the US Constitution against all enemies,
        [ foreign or domestic, requires you to follow Snowden's example.

    So we have the following issues if we want to enable electric-indent-mode:
    - C-j's default binding becomes useless.

Why not make the mode swap RET and C-j?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Default behaviour of RET.

[Josh]

On Tue, Oct 22, 2013 at 7:02 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> If the default binding of RET changes, please do make sure that there
>> is a simple and easily accessible way of toggling this, to enable
>> correct pasting of multi-line text in tty versions of Emacs.
>
> Indeed, very good point.
> So we have the following issues if we want to enable electric-indent-mode:
> - C-j's default binding becomes useless.
> - need for a new tty-paste command.

I'm not sure that follows.  In visual-line-mode, the problem of how
to specialize functions' interactive behavior while leaving their
programmatic behavior unchanged is solved thusly:

  (defvar visual-line-mode-map
    (let ((map (make-sparse-keymap)))
      (define-key map [remap kill-line] 'kill-visual-line)
      (define-key map [remap move-beginning-of-line] 'beginning-of-visual-line)
      ...
      map))

Is there some reason why electric-mode could not employ a similar
approach using a new electric-mode-map, for example by removing ?\n
from electric-indent-chars and remapping newline to
newline-and-indent?  Alternatively, electric-mode could abandon use
of post-self-insert-hook entirely, instead remapping
self-insert-command to a new self-insert-and-reindent-command that
would first perform the insertion as usual and then reindent iff
electric-indent-chars contained the inserted character.  Both of these
approaches would perform electric reindentation after interactive
newline insertion while leaving the newline function's programmatic
behavior unchanged.  If electric-mode were enabled by default for
programming modes, both of these approaches would also result in
the "modern" reindentation behavior advocated by many here without
the necessity of any changes to default key bindings.

Josh

Re: Default behaviour of RET.

[Stefan Monnier]

>>> If the default binding of RET changes, please do make sure that there
>>> is a simple and easily accessible way of toggling this, to enable
>>> correct pasting of multi-line text in tty versions of Emacs.
>> Indeed, very good point.
>> So we have the following issues if we want to enable electric-indent-mode:
>> - C-j's default binding becomes useless.
>> - need for a new tty-paste command.
> I'm not sure that follows.

I can't see the relation between what you say and what you quoted.


        Stefan

Re: Default behaviour of RET.

[Josh]

On Wed, Oct 23, 2013 at 5:29 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>>>> If the default binding of RET changes, please do make sure that there
>>>> is a simple and easily accessible way of toggling this, to enable
>>>> correct pasting of multi-line text in tty versions of Emacs.
>>> Indeed, very good point.
>>> So we have the following issues if we want to enable electric-indent-mode:
>>> - C-j's default binding becomes useless.
>>> - need for a new tty-paste command.
>> I'm not sure that follows.
>
> I can't see the relation between what you say and what you quoted.

Sorry if I was unclear; I'll try again.  Several[0][1][2][3] people here have
expressed their preference for the continued availability of distinct
newline and newline-and-indent commands.  This distinction is lost
under the current electric-mode implementation, which effectively
merges the two.  This is presumably why you believe that C-j's default
binding of newline-and-indent would become useless, but it seems to me
that a keymap-based electric-mode implementation could remap those
commands such that hitting return (->newline-and-indent) would trigger
reindentation while keeping non-indenting newline functionality available
on C-j (->newline).

Secondly, this discussion was launched by a complaint[4] about
electric-mode's effect on newline's behavior when called
programmatically.  As Alan mentioned[5], there are hundreds of
existing calls to newline and such a change to its behavior would have
ripple effects on and perhaps breakage of an unknown number of those
callers.  In contrast to the present electric-mode implementation, a
keymap-based approach would leave newline's behavior in those
contexts unchanged.

[0] http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00552.html
[1] http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00685.html
[2] http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00639.html
[3] http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00690.html
[4] http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00407.html
[5] http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00413.html

Re: Default behaviour of RET.

[Stefan Monnier]

> binding of newline-and-indent would become useless, but it seems to me
> that a keymap-based electric-mode implementation could remap those
> commands such that hitting return (->newline-and-indent) would trigger
> reindentation while keeping non-indenting newline functionality available
> on C-j (->newline).

Yes, the plan would be indeed to make C-j perform
a newline-without-indent when electric-indent-mode is enabled.

> electric-mode's effect on newline's behavior when called
> programmatically.

That's been fixed already.


        Stefan

RE: Default behaviour of RET.

[Drew Adams]

> Either we're going to poll the users, or we aren't and instead are
> going to rely on developer opinion. In both cases, positive
> statements like "I find it convenient." are valuable.

Precisely.  And that's what I've given: my opinion that I find
the current state convenient.  Just one opinion.

> In neither case does it seem valuable or worthwhile to disparage
> or discount the opinions of other people simply because you
> disagree.

I couldn't agree more.

> If there's a vote to be made, I have helped hundreds of users
> (mostly MIT students) who wanted RET to do newline-and-indent
> by default in programming modes. It's easy enough to change if
> it's not the default, but I certainly think it's a better default.
> 
> I hope that helps,

Good, so we have some more opinions in favor of a change.

Re: Default behaviour of RET.

[Jarek Czekalski]

Some of the problems seem to be agreed, like retaining 2 different 
behaviours. But whether to choose a more popular solution - it requires 
a philosophical background.

You need to choose whether Emacs should remain a weird editor for geeks 
who do everything in the opposite way or whether it should be better 
approachable for new users. When I gave a shot at Emacs 2 years ago, I 
quit in a matter of seconds. Not having shift-selection was enough for 
me to bury Emacs. If you don't get fresh users, the editor may gradually 
become less popular. Am I right guessing that the average age of Emacs 
users is greater than of any other editor? So it may be called "getting 
old". We don't like if a nation is getting old, so I think we should not 
like an editor getting old too. Users must be drawn to draw developers. 
New blood, alternation of generations.

So more generally I would stick to the following rule: do everything 
like others do, unless there are serious reasons to do it differently.

A small non-counting voice on the RET behaviour from Emacs newbie: don't 
connect it with the electric-mode. I don't like indentation changes at 
any other moment except: on demand, for newline. I want new RET behavior 
without electric-mode. I would simply edit all modes. Personally I use 
<ret> for newline-and-indent and <S-ret> for newline.

Jarek

Re: Default behaviour of RET.

[Rustom Mody]

On Tue, Oct 22, 2013 at 1:15 PM, Jarek Czekalski  wrote:
> A small non-counting voice on the RET behaviour from Emacs newbie:

As a newbie (to this list though not emacs) myself, I tend to agree
myself. Just remember that when you say:

> So more generally I would stick to the following rule: do everything like
> others do, unless there are serious reasons to do it differently.

Do you realize that "doing everything like others do" may require
heavy-duty rewrites?  Someone has to do that. That someone should be
motivated.

As a personal example, I am trying to put together something that will
make APL usable under emacs.  The code will at most be a few hundred
lines of elisp and maybe a few dozen of xmodmap if I can wrap my head
round that (if I do manage it).  If the expected scale was a hundred
thousand lines (or more), I would not even think about it.

So
1. Demands for change are good; just have a sense of the scale
2. Personally I would wish emacs to be more 'modern' -- yeah thats a
vague wishy-washy statement :-)


Rusi
-- 
http://www.the-magus.in
http://blog.languager.org

Re: Default behaviour of RET.

[Alan Mackenzie]

Hi, Dmitry.

On Tue, Oct 22, 2013 at 12:04:32AM +0400, Dmitry Gutov wrote:

> I really have to wonder when anyone would wish to use RET bound to
> newline. Why? Does some popular major mode provide inadequate
> indentation function, so that you have to pick whether to indent the
> next line automatically or not?

`newline' is the Right Thing to do in non-programming modes like Text
Mode, at least a lot of the time.

For example, it is if you have paragraphs indented like this one, where
    you use auto-fill-mode to calculate a non-null fill prefix to indent
    subsequent lines of the paragraph and RET to start a new paragraph at
    column zero.

Even in programming modes, you might want to start a whole-line comment
at column zero, even where (or especially where) the code is deeply
indented.

My personal position is that I'm quite happy with RET doing `newline' and
C-j doing `newline-and-indent', but (despite being a traditionalist) I
wouldn't be that bothered if those bindings were exchanged.  I would be
most unhappy if the `newline' functionality were to be obliterated, even
in restricted circumstances like `electric-indent-mode' being enabled and
\n being in `electric-indent-chars'.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default behaviour of RET.

[Stephen J. Turnbull]

Alan Mackenzie writes:
 > On Tue, Oct 22, 2013 at 12:04:32AM +0400, Dmitry Gutov wrote:

 >> I really have to wonder when anyone would wish to use RET bound to
 >> newline.

Because it doesn't autosense style, it is especially useful where
style is inconsistent.  For example, I sometimes write patches in
sequences where the last patch just fixes up the whitespace, and
previous patches do the actual work but avoid whitespace changes for
ease of review.  (Sure the reviewer can use --ignore-space-change, but
it's an annoyance.  And review tools should know how to ignore ws
changes, but not everybody uses them.)

 >> Why? Does some popular major mode provide inadequate indentation
 >> function, so that you have to pick whether to indent the next line
 >> automatically or not?

In the above use case, they all do.

 > `newline' is the Right Thing to do in non-programming modes like Text
 > Mode, at least a lot of the time.
 > 
 > For example, it is if you have paragraphs indented like this one, where
 >     you use auto-fill-mode to calculate a non-null fill prefix to indent
 >     subsequent lines of the paragraph and RET to start a new paragraph at
 >     column zero.

To be accurate, at the current left margin, which is usually zero.

 > Even in programming modes, you might want to start a whole-line comment
 > at column zero, even where (or especially where) the code is deeply
 > indented.

Sure, but all of these use cases are relatively infrequent (though not
rare).  I think you should concentrate on the consistency of semantics:

 > I would be most unhappy if the `newline' functionality were to be
 > obliterated, even in restricted circumstances like
 > `electric-indent-mode' being enabled and \n being in
 > `electric-indent-chars'.

Indeed, that change is horrible.  `newline' has well-defined
semantics, which do not include indentation.  Programs should be able
to rely on them, given that it's trivial to define a minor mode which
does nothing but install a space keymap that swaps the command
bindings of C-j and RET.

It might even be reasonable to make { RET: newline-and-indent, C-j:
newline } the default for the sake of the youngsters.

Re: Default behaviour of RET.

[Dmitry Gutov]

Hi Alan,

On 24.10.2013 00:18, Alan Mackenzie wrote:
>> I really have to wonder when anyone would wish to use RET bound to
>> newline. Why? Does some popular major mode provide inadequate
>> indentation function, so that you have to pick whether to indent the
>> next line automatically or not?
>
> `newline' is the Right Thing to do in non-programming modes like Text
> Mode, at least a lot of the time.
>
> For example, it is if you have paragraphs indented like this one, where
>      you use auto-fill-mode to calculate a non-null fill prefix to indent
>      subsequent lines of the paragraph and RET to start a new paragraph at
>      column zero.
>
> Even in programming modes, you might want to start a whole-line comment
> at column zero, even where (or especially where) the code is deeply
> indented.

I see your point.

> My personal position is that I'm quite happy with RET doing `newline' and
> C-j doing `newline-and-indent', but (despite being a traditionalist) I
> wouldn't be that bothered if those bindings were exchanged.  I would be
> most unhappy if the `newline' functionality were to be obliterated, even
> in restricted circumstances like `electric-indent-mode' being enabled and
> \n being in `electric-indent-chars'.

This seems like the best solution to me: either just a change of default 
bindings, or a new by-default-on minor mode.
And removal of `\n' from `electric-indent-chars' everywhere.

The alternative approach of remapping `C-j' to `newline' when 
`electric-indent-mode' is on seems less consistent to me, because 
there's no hard guarantee, AFAICT, that `electric-indent-chars' will 
include \n in all situations (modes and user configurations).

Re: Default behaviour of RET.

[Xue Fuqiao]

On Tue, Oct 22, 2013 at 1:01 AM, Drew Adams <drew.adams@oracle.com> wrote:
>> > > I need to customize every programming mode I use to do that.
>> >
>> > You do?  You can't use `C-j'?
>>
>> I can, but it's much less convenient than to use RET.
>
> Much less convenient, really?  Do you find `C-n' as inconvenient as
> `C-j', so you customize Emacs to replace `C-n' also?  `C-a'?  `C-e'?

Actually, I've customized Emacs to replace ‘C-n’, ‘C-a’, and ‘C-e’,
because I'm an ErgoEmacs[fn:1] user.

Footnotes:

[fn:1] http://www.emacswiki.org/emacs/ErgoemacsKeybindings

-- 
Best regards, Xue Fuqiao.
http://www.gnu.org/software/emacs/

RE: Default behaviour of RET.

[Drew Adams]

> >> The question is what would be the sane default.
> >
> > The default Emacs behavior for this is quite sane, and has been so
> > for almost 40 years now.
> 
> It is sane because it is old? That's a bad argument.

That's not the argument.  I gave my opinion that the Emacs behavior
is sane.  And that it has been so for 40 years.  No one said that
_because_ the behavior is old it must be sane.  The behavior is sane,
_and_ it is longstanding behavior, used by thousands for a long time.

But yes, one could make the argument that that longevity provides
_some_ measure of support for an assertion that the behavior is not
insane.  Some measure of support does not mean old-therefore-sane.

> > That it is not the same as the default behavior of this or that
> > other application does not make the Emacs default behavior insane.
> 
> The Emacs default settings are considered outdated by many of its
> users (here's one tiny example:
> https://github.com/technomancy/better-defaults). So discussing
> changes in long-established behaviors can be useful.

Of course it can be useful.  And change is sometimes good.  That
doesn't mean that all change is good.  It doesn't mean that this
particular change would be the right thing to do.

You are making sweeping generalizations that do not help advance
reasons why this change should be made.

Yes, long-established behaviors can always be questioned.  And the
questions and answers _here_ are?  So far, we have as arguments only
that (a) IDEs use RET instead of C-j, and (b) RET is, in some minds,
more convenient than C-j.

Emacs offers both keys/behaviors, which is good (IMHO).  Perhaps your
argument should be that the keys should be reversed?  That would be
a more reasonable request (IMO) than just making RET also do what C-j
does.

> >> You seem to be under impression that Eli is somehow new to using
> >> Emacs.
> >
> > You seem to be fantasizing.  The "foreign vantage point" is an
> > outside view, nothing more.
> 
> Yeah, and who's outside? One of the most prolific Emacs contributors?

No one is outside.  You keep repeating that strawman.  RET does outside
what C-j does inside.  From the point of view outside, RET is the right
key and C-j is the wrong key.  For Emacs, the opposite is the case -
so far.

> > That is the argument, no?  "All the other guys are doing it another
> > way."
> 
> Read it this way: "won't somebody think of the poor new users?"

We all are doing so, no?  It's not like someone is forcing a contorted
key combination here.  C-j is a simple key sequence, as simple as C-f.

Or else it's not.  Either it is convenient, as has apparently been
thought for a long time by many users, or it is not and this is finally
being realized.

And inherently not, not just because someone might have a habit
developed elsewhere - that is a separate argument from the relative
difficulty of typing C-j.

IOW, there are two separate arguments that those proposing to make
RET do what C-j does have advanced:

1. C-j is "much less convenient" to type than RET.
2. RET is what the "poor new users" are used to using.

I've argued against #1 - I'm not convinced that C-j is "much less
convenient".

But especially, I am in favor of keeping _both_ behaviors.  The keys
could be swapped, if people think that is the way to go.

Against #2 I have less to say.  I've argued that if "others are doing
it" is the _only_ argument then that is pretty lame.  If that is it,
then we are essentially back to the discussion about turning on CUA
mode by default: same argument exactly.  I would hope for a better
argument.

> And: "quite often standard behavior emerges for a reason".

Yet another platitude - yap, yap.  (And what/whom are you quoting?)

> > There is nothing new that makes the difference in convenience
> > between `C-j' and `RET' any greater now than it has been at any
> > point in the past.  Exactly the same difference: same mole hill.
> 
> It's not the greatest of problems one might find with Emacs, true.
> 
> > For Emacs, `C-j' has been considered convenient for this behavior
> > for a very long time.  And I, for one, still find it convenient.
> > It doesn't get much more convenient than `C-j'.  Circulez ; il
> > n'y a rien a voir.
> 
> You still haven't answered why you want RET to be bound to
> `newline'.

I like having both keys, for the two different behaviors.  I use both.
I can't tell you just when I use RET, but I do.

I don't really care much which key provides which behavior.  I don't
mind using either - I already use both.

I would put it exactly the way Richard put it: (a) "I don't think the
default behavior of RET should be changed" and (b) "We have two
commands, RET and C-j", with two different behaviors.

For me, (b) is more important: keep the two behaviors.  For (a), I'm
adding my voice to agree that "I don't think the default behavior of
RET should be changed."  That's all.

Re: Default behaviour of RET.

[Dmitry Gutov]

Drew Adams <drew.adams@oracle.com> writes:

> But especially, I am in favor of keeping _both_ behaviors.  The keys
> could be swapped, if people think that is the way to go.

Swapping wouldn't be the worst idea, but there's also a related default
`M-j' keybinding, which invokes a related, but even fancier command.

Have `C-j' insert newline, but still have `M-j' do lots more additional
stuff would make it less consistent.