bug#7751: 24.0.50; `fill-paragraph' on doc string with colons

bug#7751: 24.0.50; `fill-paragraph' on doc string with colons

[Drew Adams]

This is a regression introduced in Emacs 22.1.  There is no such problem
in Emacs 20 or 21.
 
emacs -Q
 
Put this in *scratch* or an Emacs-Lisp mode buffer:
 
(defun foo ()
  "WHAT'S WITH THIS?
 
The usual menu keywords are allowed: :enable, :visible, :keys, :filter,...."
 
  42)


Put the cursor on the second sentence of the doc string and hit `M-q'.
The sentence gets wrapped this way:
 
The usual menu keywords are
allowed: :enable, :visible, :keys, :filter,...."
 
The line should instead be broken between ":keys" and ":filter", since
there is enough space for "... allowed: :enable, :visible, :keys,".
 
Remove the colon before ":enable" or add a space to get ": enable" and
the line gets broken as one would expect.  But with the colons as they
are, and no matter how much space is between "allowed:" and ":enable",
the line is broken incorrectly.
 
The same problem occurs if you change "allowed:" to "allowed.",
"allowed!", "allowed?", "allowed,", or just "allowed", and no matter how
much space you leave after it.  The same problem occurs if you remove
"allowed:" altogether, i.e., with "The usual menu keywords are :enable,
...".
 
 
 
In GNU Emacs 24.0.50.1 (i386-mingw-nt5.1.2600)
 of 2010-12-20 on 3249CTO
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (4.4) --no-opt --cflags
-Ic:/imagesupport/include'
 

bug#7751: 24.0.50; `fill-paragraph' on doc string with colons

[Lawrence Mitchell]

Drew Adams wrote:
> This is a regression introduced in Emacs 22.1.  There is no such problem
> in Emacs 20 or 21.

[...]

> The same problem occurs if you change "allowed:" to "allowed.",
> "allowed!", "allowed?", "allowed,", or just "allowed", and no matter how
> much space you leave after it.  The same problem occurs if you remove
> "allowed:" altogether, i.e., with "The usual menu keywords are :enable,
> ...".

Here's an even more minimal test-case:

emacs -Q

(setq emacs-lisp-docstring-fill-column 30)
(insert "\"a word, another word and more :bar\"")

M-q

gives a break of:

| a word, another word and
| more :bar

whereas one would expect

| a word, another word and more
| :bar


The problem appears to be in the regexp used to define
paragraph-start in lisp-fill-paragraph:

...
      (let ((paragraph-start (concat paragraph-start
				     "\\|\\s-*\\([(;:\"]\\|`(\\|#'(\\)"))
...

Removing the colon leads to "expected" line breaking.

FWIW, the colon was introduced by this commit:

commit cd3f5084abdd59a8392cedee90ddbaa096d61d55
Author: Dave Love <fx@gnu.org>
Date:   Wed Mar 18 16:02:37 1998 +0000

    (lisp-fill-paragraph): Adjust
    paragraph-start in default filling case so that filling doc
    strings works.

Cheers,

Lawrence
-- 
Lawrence Mitchell <wence@gmx.li>

bug#7751: 24.0.50; `fill-paragraph' on doc string with colons

[Lawrence Mitchell]

Following up open bugs...

Lawrence Mitchell wrote:

[...]

> Here's an even more minimal test-case:

> emacs -Q

> (setq emacs-lisp-docstring-fill-column 30)
> (insert "\"a word, another word and more :bar\"")

> M-q

> gives a break of:

> | a word, another word and
> | more :bar

> whereas one would expect

> | a word, another word and more
> | :bar

> The problem appears to be in the regexp used to define
> paragraph-start in lisp-fill-paragraph:

> ...
>       (let ((paragraph-start (concat paragraph-start
> 				     "\\|\\s-*\\([(;:\"]\\|`(\\|#'(\\)"))
> ...

> Removing the colon leads to "expected" line breaking.

[...]

Would a patch to do this be welcome?

Lawrence

bug#7751: 24.0.50; `fill-paragraph' on doc string with colons

[Chong Yidong]

Lawrence Mitchell <wence@gmx.li> writes:

>> The problem appears to be in the regexp used to define
>> paragraph-start in lisp-fill-paragraph:
>
>>       (let ((paragraph-start (concat paragraph-start
>> 				     "\\|\\s-*\\([(;:\"]\\|`(\\|#'(\\)"))
>
>> Removing the colon leads to "expected" line breaking.
>
> Would a patch to do this be welcome?

As explained in the comments in that file, the colon is there so that
forms like

(defcustom foo nil
  :bar 3)

Don't get refilled to

(defcustom foo nil :bar 3)

when you type M-q.  The effect on docstrings containing colons is
unfortunate, but simply removing the colon from the regexp would be
incorrect.

bug#7751: 24.0.50; `fill-paragraph' on doc string with colons

[Drew Adams]

> the colon is there so that forms like
> (defcustom foo nil
>   :bar 3)
> 
> Don't get refilled to
> (defcustom foo nil :bar 3)
> 
> when you type M-q.

And just why is that important?

If there is room for it given the value of `fill-column', such filling makes
sense - a good thing, not a bad thing.

Sure, it would be best if a keyword and its value were always (`fill-column'
permitting) on the same line.  But that's a nice-to-have - it should not be the
first priority or the reason for mangling filling of doc strings.

Anyway, the #7751 bug report is not about that, at all.

It's about using `M-q' on a _doc string_.  It's not about using `M-q' on a sexp
such as you present above (which has no strings at all).  Let's not take a
detour.  If the current code does not distinguish the doc-string case, maybe it
should.

bug#7751: 24.0.50; `fill-paragraph' on doc string with colons

[Drew Adams]

ping

bug#7751: 24.0.50; `fill-paragraph' on doc string with colons

[Lars Ingebrigtsen]

Lawrence Mitchell <wence@gmx.li> writes:

> The problem appears to be in the regexp used to define
> paragraph-start in lisp-fill-paragraph:
>
> ...
>       (let ((paragraph-start (concat paragraph-start
> 				     "\\|\\s-*\\([(;:\"]\\|`(\\|#'(\\)"))
> ...
>
> Removing the colon leads to "expected" line breaking.

I've now removed the colon from that regexp when we're doing `M-q'
inside a string.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no