bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Andreas Röhler]

Hi,

have an auto-fill-function set in a mode.

When testing, noticed that  this auto-fill-function was called by  
`newline` even if  no auto-fill-mode.

See definition of newline from simple.el, line 22:

(auto-fill-function (if arg nil auto-fill-function))

IMO there is also a docu bug, as this auto-filling is switched of by any arg

whilst docu says: With ARG, insert that many newlines -- which sounds 
unrelated if filling.

Herewith a test-command: inserts "foo" in a temp-buffer by just calling 
newline

(defun auto-fill-test()
   (interactive)
   (let ((auto-fill-function (lambda ()(insert "foo"))))
     (with-temp-buffer
       (switch-to-buffer (current-buffer))
       (newline))))

GNU Emacs 27.0.50 (build 1, i686-pc-linux-gnu, GTK+ Version 3.14.5) of 
2019-06-04

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> have an auto-fill-function set in a mode.
>
> When testing, noticed that  this auto-fill-function was called by  `newline`
> even if  no auto-fill-mode.

Making auto-fill-function non-nil is effectively the same as enabling
auto-fill-mode.

> See definition of newline from simple.el, line 22:
>
> (auto-fill-function (if arg nil auto-fill-function))
>
> IMO there is also a docu bug, as this auto-filling is switched of by any arg
>
> whilst docu says: With ARG, insert that many newlines -- which sounds unrelated
> if filling.

I don't see a documentation bug, as the behaviour you see is documented
in both docstrings and the Elisp manual.

The docstring of newline says:

  Calls `auto-fill-function' if the current column number is greater
  than the value of `fill-column' and ARG is nil.

The docstring of auto-fill-function says:

  It is called after self-inserting any character specified in
  the `auto-fill-chars' table.

The docstring of self-insert-command says:

  After insertion, `internal-auto-fill' is called if
  `auto-fill-function' is non-nil and if the `auto-fill-chars' table has
  a non-nil value for the inserted character.

So I don't see a bug here.

Thanks,

-- 
Basil

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Andreas Röhler]

On 17.07.19 12:47, Basil L. Contovounesios wrote:
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> have an auto-fill-function set in a mode.
>>
>> When testing, noticed that  this auto-fill-function was called by  `newline`
>> even if  no auto-fill-mode.
> Making auto-fill-function non-nil is effectively the same as enabling
> auto-fill-mode.
>
>> See definition of newline from simple.el, line 22:
>>
>> (auto-fill-function (if arg nil auto-fill-function))
>>
>> IMO there is also a docu bug, as this auto-filling is switched of by any arg
>>
>> whilst docu says: With ARG, insert that many newlines -- which sounds unrelated
>> if filling.
> I don't see a documentation bug, as the behaviour you see is documented
> in both docstrings and the Elisp manual.
>
> The docstring of newline says:
>
>    Calls `auto-fill-function' if the current column number is greater
>    than the value of `fill-column' and ARG is nil.

Did you try the test-function delivered?

It inserts a newline in an empty buffer.

May that condition being matched then?

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

[-- Attachment #1: Type: text/plain, Size: 1077 bytes --]

severity 36702 minor
found 36702 26.2.90
tags 36702 + patch
quit

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> On 17.07.19 12:47, Basil L. Contovounesios wrote:
>
>> The docstring of newline says:
>>
>>    Calls `auto-fill-function' if the current column number is greater
>>    than the value of `fill-column' and ARG is nil.
>
> Did you try the test-function delivered?

Yes.

> It inserts a newline in an empty buffer.
>
> May that condition being matched then?

Ah, are you referring to the condition that the current column number be
greater than fill-column?  This condition is enforced by the default
value of normal-auto-fill-function when auto-fill-mode is enabled,
namely do-auto-fill.  A different auto-fill-function mightn't enforce
it.

In general, there are multiple conditions guarding calls to
auto-fill-function (see e.g. internal-auto-fill and
internal_self_insert), and I'm not sure the docstring of newline is the
right place to discuss these things.  How's the following clarification
for emacs-26 instead?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Clarify-docs-on-newline-and-auto-fill-mode.patch --]
[-- Type: text/x-diff, Size: 3488 bytes --]

From 2ed60518cb5d29c0d8c7cc944740c0fecb1713ef Mon Sep 17 00:00:00 2001
From: "Basil L. Contovounesios" <contovob@tcd.ie>
Date: Wed, 17 Jul 2019 13:05:42 +0100
Subject: [PATCH] Clarify docs on newline and auto-fill-mode

* doc/lispref/text.texi (Commands for Insertion):
* lisp/simple.el (newline): Do not mention conditions specific to
'do-auto-fill' under documentation of 'newline' (bug#36702).
---
 doc/lispref/text.texi | 17 ++++++++++-------
 lisp/simple.el        |  9 +++++----
 2 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi
index f44659c622..dad6fa2280 100644
--- a/doc/lispref/text.texi
+++ b/doc/lispref/text.texi
@@ -542,25 +542,28 @@ Commands for Insertion
 loop handles this function specially.
 @end deffn
 
-@deffn Command newline &optional number-of-newlines
+@deffn Command newline &optional number-of-newlines interactive
 This command inserts newlines into the current buffer before point.
 If @var{number-of-newlines} is supplied, that many newline characters
-are inserted.
+are inserted.  In an interactive call, @var{number-of-newlines} is the
+numeric prefix argument.
 
 @cindex newline and Auto Fill mode
-This function calls @code{auto-fill-function} if the current column
-number is greater than the value of @code{fill-column} and
-@var{number-of-newlines} is @code{nil}.  Typically what
+This command calls @code{self-insert-command} to insert newlines,
+which may subsequently break the preceding line by calling
+@code{auto-fill-function} (@pxref{Auto Filling}).  Typically what
 @code{auto-fill-function} does is insert a newline; thus, the overall
 result in this case is to insert two newlines at different places: one
 at point, and another earlier in the line.  @code{newline} does not
 auto-fill if @var{number-of-newlines} is non-@code{nil}.
 
+This command does not run the hook @code{post-self-insert-hook} unless
+called interactively or @var{interactive} is non-@code{nil}.
+
 This command indents to the left margin if that is not zero.
 @xref{Margins}.
 
-The value returned is @code{nil}.  In an interactive call, @var{count}
-is the numeric prefix argument.
+The value returned is @code{nil}.
 @end deffn
 
 @defvar overwrite-mode
diff --git a/lisp/simple.el b/lisp/simple.el
index a0f2da7152..d524d3f0ce 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -427,17 +427,18 @@ newline
 that it adds, and reindents the preceding line.  To just insert
 a newline, use \\[electric-indent-just-newline].
 
-Calls `auto-fill-function' if the current column number is greater
-than the value of `fill-column' and ARG is nil.
+If `auto-fill-mode' is enabled, this may cause automatic line
+breaking of the preceding line.  A non-nil ARG inhibits this.
+
 A non-nil INTERACTIVE argument means to run the `post-self-insert-hook'."
   (interactive "*P\np")
   (barf-if-buffer-read-only)
-  ;; Call self-insert so that auto-fill, abbrev expansion etc. happens.
+  ;; Call self-insert so that auto-fill, abbrev expansion etc. happen.
   ;; Set last-command-event to tell self-insert what to insert.
   (let* ((was-page-start (and (bolp) (looking-at page-delimiter)))
          (beforepos (point))
          (last-command-event ?\n)
-         ;; Don't auto-fill if we have a numeric argument.
+         ;; Don't auto-fill if we have a prefix argument.
          (auto-fill-function (if arg nil auto-fill-function))
          (arg (prefix-numeric-value arg))
          (postproc
-- 
2.20.1


[-- Attachment #3: Type: text/plain, Size: 20 bytes --]


Thanks,

-- 
Basil

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Andreas Röhler]

On 17.07.19 14:21, Basil L. Contovounesios wrote:
> severity 36702 minor
> found 36702 26.2.90
> tags 36702 + patch
> quit
>
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> On 17.07.19 12:47, Basil L. Contovounesios wrote:
>>
>>> The docstring of newline says:
>>>
>>>     Calls `auto-fill-function' if the current column number is greater
>>>     than the value of `fill-column' and ARG is nil.
>> Did you try the test-function delivered?
> Yes.
>
>> It inserts a newline in an empty buffer.
>>
>> May that condition being matched then?
> Ah, are you referring to the condition that the current column number be
> greater than fill-column?  This condition is enforced by the default
> value of normal-auto-fill-function when auto-fill-mode is enabled,
> namely do-auto-fill.  A different auto-fill-function mightn't enforce
> it.
>
> In general, there are multiple conditions guarding calls to
> auto-fill-function (see e.g. internal-auto-fill and
> internal_self_insert), and I'm not sure the docstring of newline is the
> right place to discuss these things.

 From my point of view it's not that much about the docu but about the 
design.

Users should not be forced to add an extra argument to prevent auto-fill 
when it is about a newline.


>   How's the following clarification
> for emacs-26 instead?
>
>

See above.

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> From my point of view it's not that much about the docu but about the design.
>
> Users should not be forced to add an extra argument to prevent auto-fill when it
> is about a newline.

Users who do not want auto-filling should not enable auto-fill-mode,
either directly or by setting auto-fill-function.

-- 
Basil

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> From my point of view it's not that much about the docu but about the design.
>>
>> Users should not be forced to add an extra argument to prevent auto-fill when it
>> is about a newline.
>
> Users who do not want auto-filling should not enable auto-fill-mode,
> either directly or by setting auto-fill-function.

Alternatively, users who want auto-fill-mode to ignore newlines can do
the following:

  (aset auto-fill-chars ?\C-j nil)

This is documented under (info "(elisp) Commands for Insertion") and
(info "(elisp) Auto Filling").

-- 
Basil

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Andreas Röhler]

On 17.07.19 16:38, Basil L. Contovounesios wrote:
> "Basil L. Contovounesios" <contovob@tcd.ie> writes:
>
>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>
>>>  From my point of view it's not that much about the docu but about the design.
>>>
>>> Users should not be forced to add an extra argument to prevent auto-fill when it
>>> is about a newline.
>> Users who do not want auto-filling should not enable auto-fill-mode,
>> either directly or by setting auto-fill-function.
> Alternatively, users who want auto-fill-mode to ignore newlines can do
> the following:
>
>    (aset auto-fill-chars ?\C-j nil)
>
> This is documented under (info "(elisp) Commands for Insertion") and
> (info "(elisp) Auto Filling").
>
Okay, commonly there many ways in Emacs to work around an issue, which 
is great. But that doesn't solve it at source. By its semantic newline 
is unrelated to filling, from there that behavior is not to expect, 
idiosyncratic.

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> On 17.07.19 16:38, Basil L. Contovounesios wrote:
>> "Basil L. Contovounesios" <contovob@tcd.ie> writes:
>>
>>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>>
>>>>  From my point of view it's not that much about the docu but about the design.
>>>>
>>>> Users should not be forced to add an extra argument to prevent auto-fill when it
>>>> is about a newline.
>>> Users who do not want auto-filling should not enable auto-fill-mode,
>>> either directly or by setting auto-fill-function.
>> Alternatively, users who want auto-fill-mode to ignore newlines can do
>> the following:
>>
>>    (aset auto-fill-chars ?\C-j nil)
>>
>> This is documented under (info "(elisp) Commands for Insertion") and
>> (info "(elisp) Auto Filling").
>>
> Okay, commonly there many ways in Emacs to work around an issue, which is
> great. But that doesn't solve it at source. By its semantic newline is unrelated
> to filling, from there that behavior is not to expect, idiosyncratic.

Unless you or someone else has a concrete proposal for how to fix said
idiosyncratic behaviour in a backward-compatible way, I think that ship
has sailed.

-- 
Basil

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Andreas Röhler]

On 17.07.19 19:47, Basil L. Contovounesios wrote:
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> On 17.07.19 16:38, Basil L. Contovounesios wrote:
>>> "Basil L. Contovounesios" <contovob@tcd.ie> writes:
>>>
>>>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>>>
>>>>>   From my point of view it's not that much about the docu but about the design.
>>>>>
>>>>> Users should not be forced to add an extra argument to prevent auto-fill when it
>>>>> is about a newline.
>>>> Users who do not want auto-filling should not enable auto-fill-mode,
>>>> either directly or by setting auto-fill-function.
>>> Alternatively, users who want auto-fill-mode to ignore newlines can do
>>> the following:
>>>
>>>     (aset auto-fill-chars ?\C-j nil)
>>>
>>> This is documented under (info "(elisp) Commands for Insertion") and
>>> (info "(elisp) Auto Filling").
>>>
>> Okay, commonly there many ways in Emacs to work around an issue, which is
>> great. But that doesn't solve it at source. By its semantic newline is unrelated
>> to filling, from there that behavior is not to expect, idiosyncratic.
> Unless you or someone else has a concrete proposal for how to fix said
> idiosyncratic behaviour in a backward-compatible way, I think that ship
> has sailed.
>
Decoupling it should not break anything. When fixed, just setting an 
auto-fill-function will no longer trigger auto-fill in case 
auto-fill-mode is off.

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> On 17.07.19 19:47, Basil L. Contovounesios wrote:
>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>
>>> Okay, commonly there many ways in Emacs to work around an issue, which is
>>> great. But that doesn't solve it at source. By its semantic newline is unrelated
>>> to filling, from there that behavior is not to expect, idiosyncratic.
>> Unless you or someone else has a concrete proposal for how to fix said
>> idiosyncratic behaviour in a backward-compatible way, I think that ship
>> has sailed.
>>
> Decoupling it should not break anything. When fixed, just setting an
> auto-fill-function will no longer trigger auto-fill in case auto-fill-mode is
> off.

But auto-fill-function is effectively synonymous with auto-fill-mode.
If the former is nil, the latter is disabled.  If the former is non-nil,
the latter is enabled.  There is no variable auto-fill-mode, only the
buffer-local variable auto-fill-function.

Users and modes that do not want to enable auto-fill-mode should not
make auto-fill-function non-nil.

Modes that want to provide a custom auto-fill function without enabling
auto-fill-mode should set normal-auto-fill-function buffer-locally
instead of auto-fill-function.

-- 
Basil

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Andreas Röhler]

On 17.07.19 21:07, Basil L. Contovounesios wrote:
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> On 17.07.19 19:47, Basil L. Contovounesios wrote:
>>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>>
>>>> Okay, commonly there many ways in Emacs to work around an issue, which is
>>>> great. But that doesn't solve it at source. By its semantic newline is unrelated
>>>> to filling, from there that behavior is not to expect, idiosyncratic.
>>> Unless you or someone else has a concrete proposal for how to fix said
>>> idiosyncratic behaviour in a backward-compatible way, I think that ship
>>> has sailed.
>>>
>> Decoupling it should not break anything. When fixed, just setting an
>> auto-fill-function will no longer trigger auto-fill in case auto-fill-mode is
>> off.
> But auto-fill-function is effectively synonymous with auto-fill-mode.

Consider that a design flaw. Just setting a var should not trigger a mode.


> If the former is nil, the latter is disabled.  If the former is non-nil,
> the latter is enabled.  There is no variable auto-fill-mode, only the
> buffer-local variable auto-fill-function.
>
> Users and modes that do not want to enable auto-fill-mode should not
> make auto-fill-function non-nil.
>
> Modes that want to provide a custom auto-fill function without enabling
> auto-fill-mode should set normal-auto-fill-function buffer-locally
> instead of auto-fill-function.

Thanks for the hint. Will use it as a workaround.

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> On 17.07.19 21:07, Basil L. Contovounesios wrote:
>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>
>>> Decoupling it should not break anything. When fixed, just setting an
>>> auto-fill-function will no longer trigger auto-fill in case auto-fill-mode is
>>> off.
>> But auto-fill-function is effectively synonymous with auto-fill-mode.
>
> Consider that a design flaw. Just setting a var should not trigger a mode.

Depends on how you set the variable:

- Most minor modes can be enabled by customising their mode variable.
- The mode variable of auto-fill-mode is auto-fill-function.
- Calling auto-fill-mode directly does little more than set
  auto-fill-function.
- In this sense it's similar to a glorified indent-tabs-mode.
- Setting the mode variable of auto-fill-mode and being surprised when
  auto-fill ensues in certain cases is a pilot error.

For better or worse, this is all long-standing and documented behaviour,
so unless you have concrete proposals for how to improve on said design
flaw in a backward-compatible way, I'm not sure there's much to be done
except for clarify the documentation where possible.

Thanks,

-- 
Basil

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Lars Ingebrigtsen]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> * doc/lispref/text.texi (Commands for Insertion):
> * lisp/simple.el (newline): Do not mention conditions specific to
> 'do-auto-fill' under documentation of 'newline' (bug#36702).

[etc]

I think this doc clarification makes sense, but it was not applied?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> "Basil L. Contovounesios" <contovob@tcd.ie> writes:
>
>> * doc/lispref/text.texi (Commands for Insertion):
>> * lisp/simple.el (newline): Do not mention conditions specific to
>> 'do-auto-fill' under documentation of 'newline' (bug#36702).
>
> [etc]
>
> I think this doc clarification makes sense, but it was not applied?

Basil, I think the right thing to do here is to apply the patch and
close the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> "Basil L. Contovounesios" <contovob@tcd.ie> writes:
>>
>>> * doc/lispref/text.texi (Commands for Insertion):
>>> * lisp/simple.el (newline): Do not mention conditions specific to
>>> 'do-auto-fill' under documentation of 'newline' (bug#36702).
>>
>> [etc]
>>
>> I think this doc clarification makes sense, but it was not applied?
>
> Basil, I think the right thing to do here is to apply the patch and
> close the bug report.

Oops, sorry for neglecting this.  Are we still pushing doc fixes to
emacs-26, or should this[1] go to master?  Eli?

[1]: https://debbugs.gnu.org/36702#14

Thanks,

-- 
Basil

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Eli Zaretskii]

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Date: Mon, 07 Oct 2019 19:27:53 +0100
> Cc: 36702@debbugs.gnu.org
> 
> Oops, sorry for neglecting this.  Are we still pushing doc fixes to
> emacs-26, or should this[1] go to master?  Eli?

Please push to master.  The emacs-26 branch is for all practical
purposes a thing of past at this point.

bug#36702: 27.0.50; newline: don't auto-fill regardless of auto-fill-mode

[Basil L. Contovounesios]

tags 36702 fixed
close 36702 27.1
quit

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Basil L. Contovounesios" <contovob@tcd.ie>
>> Date: Mon, 07 Oct 2019 19:27:53 +0100
>> Cc: 36702@debbugs.gnu.org
>> 
>> Oops, sorry for neglecting this.  Are we still pushing doc fixes to
>> emacs-26, or should this[1] go to master?  Eli?
>
> Please push to master.

Done[1], and I'm therefore closing this report.

[1]: Clarify docs on newline and auto-fill-mode
373ffc5931 2019-10-08 00:10:19 +0100
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=373ffc59317cb96e253a0b0939b921e9172b44ff

> The emacs-26 branch is for all practical purposes a thing of past at
> this point.

That was my impression, but I wanted to check since emacs-26 was pushed
to only yesterday.

Thanks,

-- 
Basil