bug#21254: 24.5; python.el: Docstring fill-paragraph not working with style pep-257-nn

bug#21254: 24.5; python.el: Docstring fill-paragraph not working with style pep-257-nn

[Holger Schmidt]

I write a docstring of a class method in a buffer with Python mode.
python-fill-docstring-style is set to 'pep-257-nn.  The
cursor is at the end of the docstring after the three quotes.
fill-paragraph does nothing, but it should format the docstring.

I move the cursor inside the docstring.  fill-paragraph formats the
docstring with four errors: There is no newline after the first
sentence, the first line is longer (72) than fill-column (70), the
second line and following lines are not indented (but start at column 0),
there is no newline at the end.  A second fill-paragraph adds the
missing newline at the end.

Example (the docstring is one long line):

class MyClass(object):
    def __init__(self):
        """Class initialization.  According to PEP 257 there should be a newline after the first sentence.  Following sentences are indented by 8 spaces in this case.  With python-fill-docstring-style set to 'pep-257-nn there should be a newline at the end."""
        pass

This should be formatted like this:

class MyClass(object):
    def __init__(self):
        """Class initialization.

        According to PEP 257 there should be a newline after the first
        sentence.  Following sentences are indented by 8 spaces in
        this case.  With python-fill-docstring-style set to
        'pep-257-nn there should be a newline at the end.
        """
        pass

But the result is this (fill-paragraph (M-q) inside docstring):

class MyClass(object):
    def __init__(self):
        """Class initialization.  According to PEP 257 there should be a
newline after the first sentence.  Following sentences are indented by
8 spaces in this case.  With python-fill-docstring-style set to
'pep-257-nn there should be a newline at the end."""
        pass


In GNU Emacs 24.5.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.10.4)
 of 2015-08-13 on l53
Windowing system distributor `The X.Org Foundation', version 11.0.70101000
System Description:	CentOS release 5.11 (Final)

Configured using:
 `configure --prefix=/local/emacs-24.5'

Important settings:
  value of $LC_ALL: en_US.UTF-8
  value of $LANG: C
  value of $XMODIFIERS: @im=none
  locale-coding-system: utf-8-unix

Major mode: Python

Minor modes in effect:
  flyspell-mode: t
  show-paren-mode: t
  fci-mode: t
  flymake-mode: t
  tabbar-mwheel-mode: t
  tabbar-mode: t
  which-function-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
Starting new Ispell process /usr/bin/aspell with default dictionary...
9: E901 TokenError: EOF in multi-line string [5 times]
Mark set
Auto-saving...
9: E901 TokenError: EOF in multi-line string [9 times]
End of buffer
Saving file /home/schmidth/abc.py...
Wrote /home/schmidth/abc.py
80: E501 line too long (260 > 79 characters) [3 times]
Undo! [2 times]

Load-path shadows:
/home/schmidth/main/trunk/emacs/site-lisp/css-mode hides /local/emacs-24.5/share/emacs/24.5/lisp/textmodes/css-mode
/home/schmidth/main/trunk/emacs/site-lisp/verilog-mode hides /local/emacs-24.5/share/emacs/24.5/lisp/progmodes/verilog-mode
/home/schmidth/main/trunk/emacs/site-lisp/flymake hides /local/emacs-24.5/share/emacs/24.5/lisp/progmodes/flymake

Features:
(shadow sort mail-extr emacsbug message format-spec rfc822 mailabbrev
gmm-utils mailheader sendmail add-log vc-dispatcher vc-svn flyspell
ispell python json disp-table paren cus-start cus-load
fill-column-indicator edmacro kmacro flymake compile comint ansi-color
mediawiki derived url-cache cl-macs cl gv ring mm-url gnus gnus-ems
nnheader mail-utils wid-edit mml easymenu mml-sec mm-decode mm-bodies
mm-encode url-http tls url url-proxy url-privacy url-expand url-methods
url-history mailcap url-auth mail-parse rfc2231 rfc2047 rfc2045
ietf-drums url-cookie url-domsuf url-util url-parse auth-source eieio
byte-opt bytecomp byte-compile cl-extra cl-loaddefs cl-lib cconv
eieio-core gnus-util mm-util help-fns mail-prsvr password-cache url-gw
url-vars tabbar easy-mmode redo+ which-func imenu autorevert filenotify
delsel xemacs-theme time-date tooltip electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)

Memory information:
((conses 16 156269 7595)
 (symbols 48 25223 0)
 (miscs 40 340 817)
 (strings 32 29661 3447)
 (string-bytes 1 881207)
 (vectors 16 15256)
 (vector-slots 8 452487 8536)
 (floats 8 105 248)
 (intervals 56 377 31)
 (buffers 960 14)
 (heap 1024 423219 13302))

bug#21254: 24.5; python.el: Docstring fill-paragraph not working with style pep-257-nn

[Stefan Kangas]

tags 21254 confirmed
found 21254 26.1
quit

Holger Schmidt <Holger.Schmidt@zmdi.com> writes:

> I write a docstring of a class method in a buffer with Python mode.
> python-fill-docstring-style is set to 'pep-257-nn.  The
> cursor is at the end of the docstring after the three quotes.
> fill-paragraph does nothing, but it should format the docstring.
>
> I move the cursor inside the docstring.  fill-paragraph formats the
> docstring with four errors: There is no newline after the first
> sentence, the first line is longer (72) than fill-column (70), the
> second line and following lines are not indented (but start at column 0),
> there is no newline at the end.  A second fill-paragraph adds the
> missing newline at the end.
>
> Example (the docstring is one long line):
>
> class MyClass(object):
>     def __init__(self):
>         """Class initialization.  According to PEP 257 there should be a newline after the first sentence.  Following sentences are indented by 8 spaces in this case.  With python-fill-docstring-style set to 'pep-257-nn there should be a newline at the end."""
>         pass
>
> This should be formatted like this:
>
> class MyClass(object):
>     def __init__(self):
>         """Class initialization.
>
>         According to PEP 257 there should be a newline after the first
>         sentence.  Following sentences are indented by 8 spaces in
>         this case.  With python-fill-docstring-style set to
>         'pep-257-nn there should be a newline at the end.
>         """
>         pass
>
> But the result is this (fill-paragraph (M-q) inside docstring):
>
> class MyClass(object):
>     def __init__(self):
>         """Class initialization.  According to PEP 257 there should be a
> newline after the first sentence.  Following sentences are indented by
> 8 spaces in this case.  With python-fill-docstring-style set to
> 'pep-257-nn there should be a newline at the end."""
>         pass

I can reproduce this on Emacs 26.1 and current master.

Best regards,
Stefan Kangas

bug#21254: 24.5; python.el: Docstring fill-paragraph not working with style pep-257-nn

[Daniel Martín via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Holger Schmidt <Holger.Schmidt@zmdi.com> writes:

> I write a docstring of a class method in a buffer with Python mode.
> python-fill-docstring-style is set to 'pep-257-nn.  The
> cursor is at the end of the docstring after the three quotes.
> fill-paragraph does nothing, but it should format the docstring.
>
> I move the cursor inside the docstring.  fill-paragraph formats the
> docstring with four errors: There is no newline after the first
> sentence, the first line is longer (72) than fill-column (70), the
> second line and following lines are not indented (but start at column 0),
> there is no newline at the end.  A second fill-paragraph adds the
> missing newline at the end.
>

I tested this in Emacs 29 and I get the following filled docstring:

class MyClass(object):
    def __init__(self):
        """Class initialization.  According to PEP 257 there should be
        a newline after the first sentence.  Following sentences are
        indented by 8 spaces in this case.  With
        python-fill-docstring-style set to 'pep-257-nn there should be
        a newline at the end."""
        pass

It seems like it still misses the newline after the first sentence and
the newline at the end.  Looking at the code, the missing newline after
the first sentence happens because there is no code to do that.  The
missing newline at the end is a consequence of handling the docstring as
a single-line docstring.  I think it will be fixed if we add the newline
after the first sentence.

I'm not familiar with Python or the PEP, but does anyone see any issues
if we add a new line after the first sentence of a docstring, if there
are more than one?