bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Geoffrey Teale]

[-- Attachment #1: Type: text/plain, Size: 9928 bytes --]

In paragraphs.el there is a variable  'sentence-end-doulbe-space` that
indicates where we should expect sentences to end with one or two spaces
after a period. In the same file a command is provided called
"repunctuate-sentences", which steadfastly ignores the value of this
variable and expands all single-space endings to two-space endings.

It strikes me that the naming of this command does not imply that it will
insist on two-spaces (even if the docs say so), and it would be more
sensible and offer more utility for it to respect the users wishes.  As the
variable default to `t, changing this function would have zero impact on
the default behaviour.

In GNU Emacs 30.0.50 (build 1, aarch64-apple-darwin22.5.0, NS
 appkit-2299.60 Version 13.4 (Build 22F66)) of 2023-05-23 built on
 UP-Geoffrey.fritz.box
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.4.1

Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/opt/homebrew/share/emacs/site-lisp
 --infodir=/opt/homebrew/Cellar/emacs-plus@30/30.0.50/share/info/emacs
 --prefix=/opt/homebrew/Cellar/emacs-plus@30/30.0.50 --with-xml2
 --with-gnutls --with-native-compilation --without-compress-install
 --without-dbus --without-imagemagick --with-modules --with-rsvg
 --with-ns --disable-ns-self-contained 'CFLAGS=-Os -w -pipe
 -mmacosx-version-min=13
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk
 -DFD_SETSIZE=10000 -DDARWIN_UNLIMITED_SELECT'
 'CPPFLAGS=-I/opt/homebrew/opt/zlib/include
 -I/opt/homebrew/opt/jpeg/include -I/opt/homebrew/opt/icu4c/include
 -I/opt/homebrew/opt/openssl@1.1/include -isystem/opt/homebrew/include
 -F/opt/homebrew/Frameworks
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk'
 'LDFLAGS=-L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/jpeg/lib
 -L/opt/homebrew/opt/icu4c/lib -L/opt/homebrew/opt/openssl@1.1/lib
 -L/opt/homebrew/lib -F/opt/homebrew/Frameworks
 -Wl,-headerpad_max_install_names
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk
 -L/opt/homebrew/opt/libgccjit/lib''

Configured features:
ACL GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY KQUEUE NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM ZLIB

Important settings:
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  global-jinx-mode: t
  jinx-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  shell-dirtrack-mode: t
  spaceline-info-mode: t
  ns-auto-titlebar-mode: t
  global-corfu-mode: t
  corfu-mode: t
  vertico-mode: t
  global-treesit-auto-mode: t
  gcmh-mode: t
  elpaca-use-package-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t

Load-path shadows:
/Users/geoffrey/.config/emacs/elpaca/builds/transient/transient hides
/opt/homebrew/Cellar/emacs-plus@30
/30.0.50/share/emacs/30.0.50/lisp/transient
/Users/geoffrey/.config/emacs/elpaca/builds/seq/seq hides
/opt/homebrew/Cellar/emacs-plus@30
/30.0.50/share/emacs/30.0.50/lisp/emacs-lisp/seq

Features:
(shadow sort mail-extr dabbrev emacsbug misearch multi-isearch shortdoc
help-fns radix-tree tramp-cmds vc-git vc-dispatcher jinx yasnippet
com-css-sort restclient rust-ts-mode c-ts-common rust-utils
rust-prog-mode rust-mode rust-playpen rust-compile rust-cargo
rust-common rust-rustfmt go-ts-mode gotest f langtool markdown-mode
noutline outline racket-mode racket-bug-report cus-edit cus-start
cus-load wid-edit racket-lisp-mode racket-collection tq
racket-repl-buffer-name racket-stepper racket-shell term disp-table
ehelp racket-logger racket-profile racket-smart-open racket-xp
racket-xp-complete racket-imenu racket-edit racket-repl
semantic/symref/grep grep semantic/symref semantic/util-modes
semantic/util semantic semantic/tag semantic/lex semantic/fw mode-local
cedet racket-show face-remap racket-eldoc racket-company-doc
racket-describe racket-scribble racket-visit shr pixel-fill kinsoku
url-file svg dom racket-complete racket-common racket-parens
racket-indent racket-font-lock racket-ppss racket-keywords-and-builtins
racket-doc racket-browse-url racket-cmd racket-back-end racket-util
racket-custom sh-script smie executable paredit python python-mode
info-look info org-loaddefs tramp-sh tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat parse-time iso8601 ls-lisp
hideshow hippie-exp cc-cmds cc-engine cc-vars cc-defs magit-submodule
magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull
magit-fetch magit-clone magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit package
url-handlers magit-repos magit-apply magit-wip magit-log which-func
magit-diff smerge-mode diff diff-mode git-commit log-edit message
sendmail yank-media puny rfc822 mml mml-sec epa derived epg rfc6068
epg-config gnus-util time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev
mail-utils gmm-utils mailheader pcvs-util add-log magit-core
magit-autorevert autorevert magit-margin magit-transient magit-process
with-editor shell pcomplete server magit-mode browse-url magit-git
magit-base magit-section cursor-sensor crm eldoc-box emoji-labels emoji
multisession sqlite ac-emoji ac-emoji-data auto-complete popup let-alist
spaceline-all-the-icons spaceline-all-the-icons-separators
spaceline-all-the-icons-segments all-the-icons all-the-icons-faces
data-material data-weathericons data-octicons data-fileicons
data-faicons data-alltheicons memoize fancy-battery battery dbus xml
spaceline-config spaceline-segments s spaceline powerline comp comp-cstr
rx advice powerline-separators powerline-themes
sanityinc-tomorrow-blue-theme color-theme-sanityinc-tomorrow color
ns-auto-titlebar eshell-git-prompt dash casual-dired
casual-dired-settings casual-dired-version casual-dired-sort-by
casual-dired-variables transient format-spec edmacro kmacro image-dired
image-dired-tags image-dired-external image-dired-util xdg image-mode
exif wdired dired-x dired dired-loaddefs async corfu orderless vertico
treesit-auto treesit completion eglot external-completion xref
flymake-proc flymake thingatpt project seq seq-25 ert pp ewoc debug
backtrace find-func filenotify compile text-property-search comint
ansi-osc ring imenu jsonrpc warnings icons gcmh no-littering compat
yasnippet-autoloads com-css-sort-autoloads restclient-autoloads
rust-mode-autoloads gotest-autoloads f-autoloads langtool-autoloads
markdown-mode-autoloads racket-mode-autoloads paredit-autoloads
python-mode-autoloads magit-autoloads pcase git-commit-autoloads
magit-section-autoloads transient-autoloads with-editor-autoloads
eldoc-box-autoloads ac-emoji-autoloads auto-complete-autoloads
popup-autoloads spaceline-all-the-icons-autoloads
all-the-icons-autoloads memoize-autoloads fancy-battery-autoloads
spaceline-autoloads powerline-autoloads s-autoloads
color-theme-sanityinc-tomorrow-autoloads ns-auto-titlebar-autoloads
eshell-git-prompt-autoloads dash-autoloads jinx-autoloads
casual-dired-autoloads async-autoloads corfu-autoloads
orderless-autoloads vertico-autoloads treesit-auto-autoloads
gcmh-autoloads no-littering-autoloads compat-autoloads seq-autoloads
loadhist tealeg--nxml tealeg--yas tealeg--css tealeg--rest tealeg--rust
tealeg--go tealeg--languagetool tealeg--markdown tealeg--racket
tealeg--python tealeg--magit tealeg--eldoc tealeg--emoji
tealeg--appearance tealeg--compilation ansi-color tealeg--eshell
tealeg--spelling tealeg--dired tealeg--completion tealeg--tree-sitter
tealeg--bell tealeg--performance tealeg--litter cl-extra help-mode
elpaca-use-package use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key easy-mmode
use-package-core elpaca-use-package-autoloads elpaca-log elpaca-ui url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util url-parse auth-source eieio
eieio-core cl-macs password-cache json subr-x map byte-opt gv bytecomp
byte-compile url-vars mailcap tealeg--elpaca cl-seq elpaca
elpaca-process cl-loaddefs cl-lib elpaca-autoloads rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/ns-win ns-win ucs-normalize
mule-util term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice simple cl-generic indonesian philippine cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads kqueue cocoa ns lcms2 multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 809464 818668)
 (symbols 48 45071 6)
 (strings 32 280258 43041)
 (string-bytes 1 6127372)
 (vectors 16 71007)
 (vector-slots 8 1410775 2669962)
 (floats 8 1212 8068)
 (intervals 56 2052 1041)
 (buffers 984 20))

[-- Attachment #2: Type: text/html, Size: 10667 bytes --]

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Eli Zaretskii]

severity 70832 wishlist
thanks

> From: Geoffrey Teale <tealeg@gmail.com>
> Date: Wed, 8 May 2024 12:21:50 +0200
> 
> In paragraphs.el there is a variable  'sentence-end-doulbe-space` that indicates where we should expect
> sentences to end with one or two spaces after a period. In the same file a command is provided called
> "repunctuate-sentences", which steadfastly ignores the value of this variable and expands all single-space
> endings to two-space endings. 
> 
> It strikes me that the naming of this command does not imply that it will insist on two-spaces (even if the docs
> say so), and it would be more sensible and offer more utility for it to respect the users wishes.  As the variable
> default to `t, changing this function would have zero impact on the default behaviour. 

Patches to allow repunctuate-sentences heed the value of
sentence-end-double-space are welcome.

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Juri Linkov]

>> In paragraphs.el there is a variable  'sentence-end-doulbe-space` that indicates where we should expect
>> sentences to end with one or two spaces after a period. In the same file a command is provided called
>> "repunctuate-sentences", which steadfastly ignores the value of this variable and expands all single-space
>> endings to two-space endings.
>>
>> It strikes me that the naming of this command does not imply that it will insist on two-spaces (even if the docs
>> say so), and it would be more sensible and offer more utility for it to respect the users wishes.  As the variable
>> default to `t, changing this function would have zero impact on the default behaviour.
>
> Patches to allow repunctuate-sentences heed the value of
> sentence-end-double-space are welcome.

The purpose of repunctuate-sentences is
to put two spaces at the end of sentences.
What should it do when sentence-end-double-space is nil?

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: Geoffrey Teale <tealeg@gmail.com>,  70832@debbugs.gnu.org
> Date: Sun, 12 May 2024 09:58:05 +0300
> 
> >> In paragraphs.el there is a variable  'sentence-end-doulbe-space` that indicates where we should expect
> >> sentences to end with one or two spaces after a period. In the same file a command is provided called
> >> "repunctuate-sentences", which steadfastly ignores the value of this variable and expands all single-space
> >> endings to two-space endings.
> >>
> >> It strikes me that the naming of this command does not imply that it will insist on two-spaces (even if the docs
> >> say so), and it would be more sensible and offer more utility for it to respect the users wishes.  As the variable
> >> default to `t, changing this function would have zero impact on the default behaviour.
> >
> > Patches to allow repunctuate-sentences heed the value of
> > sentence-end-double-space are welcome.
> 
> The purpose of repunctuate-sentences is
> to put two spaces at the end of sentences.
> What should it do when sentence-end-double-space is nil?

Leave just one space between sentences, I think.

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Juri Linkov]

>> >> In paragraphs.el there is a variable  'sentence-end-doulbe-space` that indicates where we should expect
>> >> sentences to end with one or two spaces after a period. In the same file a command is provided called
>> >> "repunctuate-sentences", which steadfastly ignores the value of this variable and expands all single-space
>> >> endings to two-space endings.
>> >>
>> >> It strikes me that the naming of this command does not imply that it will insist on two-spaces (even if the docs
>> >> say so), and it would be more sensible and offer more utility for it to respect the users wishes.  As the variable
>> >> default to `t, changing this function would have zero impact on the default behaviour.
>> >
>> > Patches to allow repunctuate-sentences heed the value of
>> > sentence-end-double-space are welcome.
>> 
>> The purpose of repunctuate-sentences is
>> to put two spaces at the end of sentences.
>> What should it do when sentence-end-double-space is nil?
>
> Leave just one space between sentences, I think.

A special command 'repunctuate-sentences' that uses 'query-replace'
is required for the value 'sentence-end-double-space' because
replacements from 1 space to 2 spaces are ambiguous (e.g.
such text as "e.g." still require 1 space after the period),
therefore the command should query every replacement.
OTOH, the reverse replacement from 2 to 1 space is quite
straightforward.

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: tealeg@gmail.com,  70832@debbugs.gnu.org
> Date: Sun, 12 May 2024 19:30:52 +0300
> 
> A special command 'repunctuate-sentences' that uses 'query-replace'
> is required for the value 'sentence-end-double-space' because
> replacements from 1 space to 2 spaces are ambiguous (e.g.
> such text as "e.g." still require 1 space after the period),
> therefore the command should query every replacement.
> OTOH, the reverse replacement from 2 to 1 space is quite
> straightforward.

So you are saying that query-replace already does the job?  Fine by
me; if no one else supports this bug report in a week or two, I will
close.

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Juri Linkov]

> So you are saying that query-replace already does the job?  Fine by
> me; if no one else supports this bug report in a week or two, I will
> close.

Actually there is another problem: whereas it's easy to rely on the
variable sentence-end-double-space as was asked by this request:

diff --git a/lisp/textmodes/paragraphs.el b/lisp/textmodes/paragraphs.el
index af99a96e045..974a83efdea 100644
--- a/lisp/textmodes/paragraphs.el
+++ b/lisp/textmodes/paragraphs.el
@@ -532,7 +532,10 @@ repunctuate-sentences
                              (end (use-region-end))))
   (interactive (list nil (use-region-beginning) (use-region-end)))
   (let ((regexp "\\([]\"')]?\\)\\([.?!]\\)\\([]\"')]?\\) +")
-        (to-string "\\1\\2\\3  "))
+        (to-string (concat "\\1\\2\\3 "
+                           (if sentence-end-double-space
+                               " "
+                             ""))))
     (if no-query
         (progn
           (when start (goto-char start))

But the problem is that when a user customized
sentence-end-double-space to nil, the user still
might want to put two spaces since the docstring says:

  "Put two spaces at the end of sentences"

So using one space will contradict the docstring.

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Geoffrey Teale]

[-- Attachment #1: Type: text/plain, Size: 2078 bytes --]

On Mon, 13 May 2024 at 08:58, Juri Linkov <juri@linkov.net> wrote:

> > So you are saying that query-replace already does the job?  Fine by
> > me; if no one else supports this bug report in a week or two, I will
> > close.
>
> Actually there is another problem: whereas it's easy to rely on the
> variable sentence-end-double-space as was asked by this request:
>
> diff --git a/lisp/textmodes/paragraphs.el b/lisp/textmodes/paragraphs.el
> index af99a96e045..974a83efdea 100644
> --- a/lisp/textmodes/paragraphs.el
> +++ b/lisp/textmodes/paragraphs.el
> @@ -532,7 +532,10 @@ repunctuate-sentences
>                               (end (use-region-end))))
>    (interactive (list nil (use-region-beginning) (use-region-end)))
>    (let ((regexp "\\([]\"')]?\\)\\([.?!]\\)\\([]\"')]?\\) +")
> -        (to-string "\\1\\2\\3  "))
> +        (to-string (concat "\\1\\2\\3 "
> +                           (if sentence-end-double-space
> +                               " "
> +                             ""))))
>      (if no-query
>          (progn
>            (when start (goto-char start))
>
> But the problem is that when a user customized
> sentence-end-double-space to nil, the user still
> might want to put two spaces since the docstring says:
>
>   "Put two spaces at the end of sentences"
>
> So using one space will contradict the docstring.
>

I was actually aware about the docstring when I raised the bug, but I
assumed that changing that docstring would be part of the fix.  If I get to
the essence of the issue: two spaces at the end of a sentence is no
longer considered correct in most prose-authoring contexts.  In that sense
what I really wanted is the ability to automatically remove any such
instances from an Org or Markdown file, for example.  When I went looking
for that in emacs this seemed like the correct tool for the job, and it is
a little bit glaring that there's such a variable that one can set, but
this function, defined in the same file, doesn't respect it.

That said, yes of course one could easily use query-replace as suggested
above.

--

[-- Attachment #2: Type: text/html, Size: 2757 bytes --]

bug#70832: 30.0.50; repunctuate-sentences should respect sentence-end-double-space

[Eli Zaretskii]

> From: Geoffrey Teale <tealeg@gmail.com>
> Date: Mon, 13 May 2024 09:39:52 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, 70832@debbugs.gnu.org
> 
>  But the problem is that when a user customized
>  sentence-end-double-space to nil, the user still
>  might want to put two spaces since the docstring says:
> 
>    "Put two spaces at the end of sentences"
> 
>  So using one space will contradict the docstring.
> 
> I was actually aware about the docstring when I raised the bug, but I assumed that changing that docstring
> would be part of the fix.  If I get to the essence of the issue: two spaces at the end of a sentence is no longer
> considered correct in most prose-authoring contexts.  In that sense what I really wanted is the ability to
> automatically remove any such instances from an Org or Markdown file, for example.  When I went looking for
> that in emacs this seemed like the correct tool for the job, and it is a little bit glaring that there's such a
> variable that one can set, but this function, defined in the same file, doesn't respect it.  
> 
> That said, yes of course one could easily use query-replace as suggested above. 

No further comments in 12 days, so I'm now closing this bug.