bug#67180: 30.0.50; 'pp-to-string' emits extra newline

bug#67180: 30.0.50; 'pp-to-string' emits extra newline

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Emacs 29 and earlier, with `-Q`, we have:

(pp-to-string "foo")
  => "\"foo\""

On master with `-Q`, we get an extra newline at the end of the string:

(pp-to-string "foo")
  => "\"foo\"
"

AFAICT this change in behavior breaks `agda2-mode` completely on master.
This happens because apparently `agda2-mode` uses `pp-to-string` to
quote strings before sending them to a REPL (that doesn't expect the
newline the middle of a command):

--8<---------------cut here---------------start------------->8---
(defun agda2-string-quote (s)
  "..."
  (let ((pp-escape-newlines t)
        (s2 (copy-sequence s)))
    (set-text-properties 0 (length s2) nil s2)
    (mapconcat 'agda2-char-quote (pp-to-string s2) "")))
                                 ^^^^^^^^^^^^^^^^^
--8<---------------cut here---------------end--------------->8---


In GNU Emacs 30.0.50 (build 24, x86_64-apple-darwin23.0.0, NS
 appkit-2487.00 Version 14.0 (Build 23A344)) of 2023-11-13

bug#67180: 30.0.50; 'pp-to-string' emits extra newline

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eshel Yaron writes:

> On Emacs 29 and earlier, with `-Q`, we have:
>
> (pp-to-string "foo")
>   => "\"foo\""
>
> On master with `-Q`, we get an extra newline at the end of the string:
>
> (pp-to-string "foo")
>   => "\"foo\"
> "
>
> AFAICT this change in behavior breaks `agda2-mode` completely on master...

I see that the Agda developers worked around[0] this in the development
version of `agda2-mode`, but of course more code out there may be
affected by this change.  Perhaps it's worth mentioning in NEWS?


Best,

Eshel


[0] https://github.com/agda/agda/pull/6954

bug#67180: 30.0.50; 'pp-to-string' emits extra newline

[Eli Zaretskii]

> Date: Tue, 14 Nov 2023 21:12:34 +0100
> From:  Eshel Yaron via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> 
> On Emacs 29 and earlier, with `-Q`, we have:
> 
> (pp-to-string "foo")
>   => "\"foo\""
> 
> On master with `-Q`, we get an extra newline at the end of the string:
> 
> (pp-to-string "foo")
>   => "\"foo\"
> "

Stefan, is this due to your changes in pp?

bug#67180: 30.0.50; 'pp-to-string' emits extra newline

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> On Emacs 29 and earlier, with `-Q`, we have:
>> 
>> (pp-to-string "foo")
>>   => "\"foo\""
>> 
>> On master with `-Q`, we get an extra newline at the end of the string:
>> 
>> (pp-to-string "foo")
>>   => "\"foo\"
>> "

Is that a problem?
I also see that the old `pp-to-string` added that same extra newline for
slightly more complex input:

    ELISP> (pp-to-string '("foo"))
    "(\"foo\")
    "
    ELISP> 

> Stefan, is this due to your changes in pp?

Yes.


        Stefan

bug#67180: 30.0.50; 'pp-to-string' emits extra newline

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> On Emacs 29 and earlier, with `-Q`, we have:
>>> 
>>> (pp-to-string "foo")
>>>   => "\"foo\""
>>> 
>>> On master with `-Q`, we get an extra newline at the end of the string:
>>> 
>>> (pp-to-string "foo")
>>>   => "\"foo\"
>>> "
>
> Is that a problem?

FWIW, I think that this change is for the better, but it is
incompatible, and sadly it broke `agda2-mode`.  (In some sense this
probably Agda's "fault", because I don't really understand why they're
using `pp-to-string` the way they do.)  My suggestion was simply to
explicitly mention this new behavior in NEWS or some such.

> I also see that the old `pp-to-string` added that same extra newline for
> slightly more complex input:
>
>     ELISP> (pp-to-string '("foo"))
>     "(\"foo\")
>     "
>     ELISP> 
>
>> Stefan, is this due to your changes in pp?
>
> Yes.
>
>
>         Stefan

bug#67180: 30.0.50; 'pp-to-string' emits extra newline

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eshel Yaron [2023-11-15 14:10:39] wrote:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>>> On Emacs 29 and earlier, with `-Q`, we have:
>>>> 
>>>> (pp-to-string "foo")
>>>>   => "\"foo\""
>>>> 
>>>> On master with `-Q`, we get an extra newline at the end of the string:
>>>> 
>>>> (pp-to-string "foo")
>>>>   => "\"foo\"
>>>> "
>>
>> Is that a problem?
>
> FWIW, I think that this change is for the better, but it is
> incompatible, and sadly it broke `agda2-mode`.  (In some sense this
> probably Agda's "fault", because I don't really understand why they're
> using `pp-to-string` the way they do.)  My suggestion was simply to
> explicitly mention this new behavior in NEWS or some such.

Like this?

    diff --git a/etc/NEWS b/etc/NEWS
    index 23f4a8b5311..2dcb2f5664e 100644
    --- a/etc/NEWS
    +++ b/etc/NEWS
    @@ -1099,6 +1099,9 @@ showcases all their customization options.
     \f
     * Incompatible Lisp Changes in Emacs 30.1
     
    +** 'pp' and 'pp-to-string' now always include a terminating newline.
    +In the past they included a terminating newline in most cases but not all.
    +
     ** 'buffer-match-p' and 'match-buffers' take '&rest args'.
     They used to take a single '&optional arg' and were documented to use
     an unreliable hack to try and support condition predicates that


-- Stefan

bug#67180: 30.0.50; 'pp-to-string' emits extra newline

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Eshel Yaron [2023-11-15 14:10:39] wrote:
>
>> ...My suggestion was simply to explicitly mention this new behavior
>> in NEWS or some such.
>
> Like this?
>
>     diff --git a/etc/NEWS b/etc/NEWS
>     index 23f4a8b5311..2dcb2f5664e 100644
>     --- a/etc/NEWS
>     +++ b/etc/NEWS
>     @@ -1099,6 +1099,9 @@ showcases all their customization options.
>      \f
>      * Incompatible Lisp Changes in Emacs 30.1
>      
>     +** 'pp' and 'pp-to-string' now always include a terminating newline.
>     +In the past they included a terminating newline in most cases but not all.
>     +
>      ** 'buffer-match-p' and 'match-buffers' take '&rest args'.
>      They used to take a single '&optional arg' and were documented to use
>      an unreliable hack to try and support condition predicates that
>

Exactly, yes.  Thanks!

Eshel

bug#67180: 30.0.50; 'pp-to-string' emits extra newline

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Like this?
>>
>>     diff --git a/etc/NEWS b/etc/NEWS
>>     index 23f4a8b5311..2dcb2f5664e 100644
>>     --- a/etc/NEWS
>>     +++ b/etc/NEWS
>>     @@ -1099,6 +1099,9 @@ showcases all their customization options.
>>      \f
>>      * Incompatible Lisp Changes in Emacs 30.1
>>      
>>     +** 'pp' and 'pp-to-string' now always include a terminating newline.
>>     +In the past they included a terminating newline in most cases but not all.
>>     +
>>      ** 'buffer-match-p' and 'match-buffers' take '&rest args'.
>>      They used to take a single '&optional arg' and were documented to use
>>      an unreliable hack to try and support condition predicates that
>>
>
> Exactly, yes.  Thanks!

Thanks, pushed to `master`.
Closing,


        Stefan