bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Alan Mackenzie]

Hello, Emacs.

In cl-print-to-string-with-limit appears the following binding:

    (print-length (cond
                    ((null limit) nil)
                    ((eq limit t) print-length)
                    (t (min limit 50))))
                        ^^^^^^^^^^^^

..  This has the effect of ignoring the parameter LIMIT (unless it is
very small) and instead truncating the printed size to 50.  There is a
similar mechanism to limit print-level to 8.

Although the doc string doesn't explicitly say it won't truncate like
this, it kind of implies that LIMIT is the size it will truncate to.
This excessive truncation is a bug.

cl-print-to-string-with-limit is used by backtrace--print-to-string with
a LIMIT of 5000 to print the first line of a debug buffer.  The error
message gets rudely truncated at 50 characters with an ellipsis, e.g.:

    Debugger entered--Lisp error: (error "Invalid call to `edebug-before'.  Is your debug sp...")

..  We shouldn't be doing this.  It is irritating to the user.  Correct
would be to print:

    Debugger entered--Lisp error: (error "Invalid call to `edebug-before'.  Is your debug spec correct?")

..  I propose fixing this bug by removing these limits on print-length and
print-level in cl-print-to-string-with-limit.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Alan Mackenzie]

The bug has been fixed.

On Fri, Sep 01, 2023 at 14:17:12 +0000, Alan Mackenzie wrote:
> Hello, Emacs.

> In cl-print-to-string-with-limit appears the following binding:

>     (print-length (cond
>                     ((null limit) nil)
>                     ((eq limit t) print-length)
>                     (t (min limit 50))))
>                         ^^^^^^^^^^^^

> ..  This has the effect of ignoring the parameter LIMIT (unless it is
> very small) and instead truncating the printed size to 50.  There is a
> similar mechanism to limit print-level to 8.

> Although the doc string doesn't explicitly say it won't truncate like
> this, it kind of implies that LIMIT is the size it will truncate to.
> This excessive truncation is a bug.

> cl-print-to-string-with-limit is used by backtrace--print-to-string with
> a LIMIT of 5000 to print the first line of a debug buffer.  The error
> message gets rudely truncated at 50 characters with an ellipsis, e.g.:

>     Debugger entered--Lisp error: (error "Invalid call to `edebug-before'.  Is your debug sp...")

> ..  We shouldn't be doing this.  It is irritating to the user.  Correct
> would be to print:

>     Debugger entered--Lisp error: (error "Invalid call to `edebug-before'.  Is your debug spec correct?")

> ..  I propose fixing this bug by removing these limits on print-length and
> print-level in cl-print-to-string-with-limit.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Alan Mackenzie [2023-09-01 14:17:12] wrote:

> Hello, Emacs.
>
> In cl-print-to-string-with-limit appears the following binding:
>
>     (print-length (cond
>                     ((null limit) nil)
>                     ((eq limit t) print-length)
>                     (t (min limit 50))))
>                         ^^^^^^^^^^^^
>
> ..  This has the effect of ignoring the parameter LIMIT (unless it is
> very small) and instead truncating the printed size to 50.  There is a
> similar mechanism to limit print-level to 8.
>
> Although the doc string doesn't explicitly say it won't truncate like
> this, it kind of implies that LIMIT is the size it will truncate to.
> This excessive truncation is a bug.

See also bug#34183.

> ..  I propose fixing this bug by removing these limits on print-length and
> print-level in cl-print-to-string-with-limit.

Sounds a bit drastic.  Strings can be obnoxiously long, so it's
important for cl-print to be able to truncate them.
[ IOW, I'm not happy with commit
  761f8901fffdb155cbcc7f3b5a2329161c2c1826.  ]


        Stefan

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Eli Zaretskii]

> Cc: 65680@debbugs.gnu.org
> Date: Thu, 21 Sep 2023 12:16:57 -0400
> From:  Stefan Monnier via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> > ..  I propose fixing this bug by removing these limits on print-length and
> > print-level in cl-print-to-string-with-limit.
> 
> Sounds a bit drastic.  Strings can be obnoxiously long, so it's
> important for cl-print to be able to truncate them.
> [ IOW, I'm not happy with commit
>   761f8901fffdb155cbcc7f3b5a2329161c2c1826.  ]

Then how about if we reinstate the limit, but with a larger limit for
strings, in a separate variable?  Do you have a suggestion for the
value of such a limit?

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Alan Mackenzie]

Hello, Stefan.

On Thu, Sep 21, 2023 at 12:16:57 -0400, Stefan Monnier wrote:
> Alan Mackenzie [2023-09-01 14:17:12] wrote:

> > Hello, Emacs.

> > In cl-print-to-string-with-limit appears the following binding:

> >     (print-length (cond
> >                     ((null limit) nil)
> >                     ((eq limit t) print-length)
> >                     (t (min limit 50))))
> >                         ^^^^^^^^^^^^

> > ..  This has the effect of ignoring the parameter LIMIT (unless it is
> > very small) and instead truncating the printed size to 50.  There is a
> > similar mechanism to limit print-level to 8.

> > Although the doc string doesn't explicitly say it won't truncate like
> > this, it kind of implies that LIMIT is the size it will truncate to.
> > This excessive truncation is a bug.

> See also bug#34183.

> > ..  I propose fixing this bug by removing these limits on print-length and
> > print-level in cl-print-to-string-with-limit.

> Sounds a bit drastic.  Strings can be obnoxiously long, so it's
> important for cl-print to be able to truncate them.

There is clearly no human-sized bound on string lengths, so they can
indeed be very long.  Most of the time they're not.  But they are very
frequently longer than 50 characters.

> [ IOW, I'm not happy with commit
>   761f8901fffdb155cbcc7f3b5a2329161c2c1826.  ]

Well I did post about it to emacs-devel on Sunday and Monday, asking if
anybody had any objection.  Nobody, not even you, responded.

I still believe that not truncating strings is better than truncating
them to the minute length of 50.  In fact, why truncate strings at all in
cl-prin1?  They're not truncated in prin1, etc.  The reason for
truncating lists and vectors is to prevent infinite printing when there's
a circular list or vector, something which cannot happen with a string.

If somebody doesn't want a string longer that 50 to get printed, then she
shouldn't call cl-prin1 with it.

The mistake in Emacs before my patch was a category error: wrongly
believing that print-length applies to a string length too.  It doesn't.
String lengths are a completely different kettle of fish from list
lengths.

To solve this problem properly, we need, as Eli has suggested, a separate
variable called something like print-string-length, to be set
independently of print-length (and print-level).  A sensible value for
this variable in printing backtraces might be, say, 500.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > ..  I propose fixing this bug by removing these limits on print-length and
>> > print-level in cl-print-to-string-with-limit.
>
>> Sounds a bit drastic.  Strings can be obnoxiously long, so it's
>> important for cl-print to be able to truncate them.
>
> There is clearly no human-sized bound on string lengths, so they can
> indeed be very long.  Most of the time they're not.  But they are very
> frequently longer than 50 characters.

Agreed.

>> [ IOW, I'm not happy with commit
>>   761f8901fffdb155cbcc7f3b5a2329161c2c1826.  ]
>
> Well I did post about it to emacs-devel on Sunday and Monday, asking if
> anybody had any objection.  Nobody, not even you, responded.

I'm not on `emacs-devel` nowadays :-)

> I still believe that not truncating strings is better than truncating
> them to the minute length of 50.  In fact, why truncate strings at all in
> cl-prin1?  They're not truncated in prin1, etc.

The main purpose of `prin1` is to print Sexp in a way that can be
read back.  I.e. for machine-consumption.
The main purpose of `cl-prin1` OTOH is for human consumption.
For this reason it started truncating strings while `prin1` doesn't
bother doing so.

> The reason for truncating lists and vectors is to prevent infinite
> printing when there's a circular list or vector, something which
> cannot happen with a string.

Not really, no.  It's rather for things like keeping the length of
backtrace lines under control.

> If somebody doesn't want a string longer that 50 to get printed, then she
> shouldn't call cl-prin1 with it.

You're arguing against the wrong guy: I filed bug#34183 because I found the
(current implementation of) string truncation annoying.  That doesn't
mean it's always annoying.

> The mistake in Emacs before my patch was a category error: wrongly
> believing that print-length applies to a string length too.

That's bug#34183, indeed.

> It doesn't.  String lengths are a completely different kettle of fish
> from list lengths.

Not completely: they're all concerned with truncating the output so the
human gets to see what comes afterwards, and to a large extent their
optimal value for any given string/list/vector is probably one that
corresponds more or less to the same output string length.

> To solve this problem properly, we need, as Eli has suggested, a separate
> variable called something like print-string-length, to be set
> independently of print-length (and print-level).

Sounds good.

> A sensible value for this variable in printing backtraces might be,
> say, 500.

500 is "damn long" for a line in a backtrace, IMO.  If I need to see
that much of the string, I won't mind having to click the "..." to
reveal it.  I'd vote for a default limit somewhere between "one or two
lines of 80 columns".

FWIW, after filing bug#34183 I hacked my local Emacs to use (*
8 print-length) for max string length (leading to 96 chars by default,
IIRC), thinking I'd tweak it later based on experience.
I never felt like tweaking it, tho, so it looks like it's a good
approximation of my ideal choice.

We could make its default value dynamically computed from
`print-length`, so things like `cl-print-to-string-with-limit` don't
need to be adjusted.


        Stefan

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Eli Zaretskii]

> Cc: acm@muc.de, 65680@debbugs.gnu.org
> Date: Thu, 21 Sep 2023 17:07:58 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> > [ IOW, I'm not happy with commit
> >   761f8901fffdb155cbcc7f3b5a2329161c2c1826.  ]
> 
> Well I did post about it to emacs-devel on Sunday and Monday, asking if
> anybody had any objection.  Nobody, not even you, responded.

TTTT, 2 days is too short a time to assume no one has anything to say
about a change.  I had a busy day and just couldn't get to it, before
I saw you have already installed it.

> To solve this problem properly, we need, as Eli has suggested, a separate
> variable called something like print-string-length, to be set
> independently of print-length (and print-level).  A sensible value for
> this variable in printing backtraces might be, say, 500.

500 is too large a value, my vote is for 200 at most.

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Drew Adams]

> > I still believe that not truncating strings is better than truncating
> > them to the minute length of 50.  In fact, why truncate strings at all
> > in cl-prin1?  They're not truncated in prin1, etc.
> 
> The main purpose of `prin1` is to print Sexp in a way that can be
> read back.  I.e. for machine-consumption.
> The main purpose of `cl-prin1` OTOH is for human consumption.
> For this reason it started truncating strings while `prin1` doesn't
> bother doing so.

Not to muddy the waters too much, but FWIW: `cl-prin1'
is a far cry from an emulation of Common Lisp `prin1'.

(And for Common Lisp `prin1' there is _no_ truncation.
That is, nothing in the standard allows for truncation.
But _implementations_ could provide an option that
allows for truncation, AFAICS.)

> > It doesn't.  String lengths are a completely different kettle of fish
> > from list lengths.
> 
> Not completely: they're all concerned with truncating the output so the
> human gets to see what comes afterwards, and to a large extent their
> optimal value for any given string/list/vector is probably one that
> corresponds more or less to the same output string length.

FWIW: Common Lisp treats strings very differently
from arrays, vectors, and lists.  It specifically
does not let variable `*print-length*' apply to
strings (and there is no other var that does so).

> > To solve this problem properly, we need, as Eli has
> > suggested, a separate variable called something like
> > print-string-length, to be set independently of
> > print-length (and print-level).
> 
> Sounds good.

Yes, that would be reasonable.

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Not to muddy the waters too much, but FWIW: `cl-prin1'
> is a far cry from an emulation of Common Lisp `prin1'.

Please don't discuss Common Lisp here.  This is a bug report bout Emacs Lisp.


        Stefan

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Alan Mackenzie]

Hello, Eli.

On Fri, Sep 29, 2023 at 15:39:44 +0300, Eli Zaretskii wrote:
> > Date: Wed, 27 Sep 2023 15:12:37 +0000
> > Cc: Eli Zaretskii <eliz@gnu.org>
> > From: Alan Mackenzie <acm@muc.de>

> > I've implemented cl-print-string-length in cl-print.el, but not as a
> > customisable variable (for consistency with print-{length,level}).  I
> > manipulate it in cl-print-string-with-limit by binding it to (- limit
> > 3), then multiplying it by ¾ at each successive trial printing.

> > I haven't yet made any amendment to the CL manual.

> Please do.

It seems that cl-print.el is entirely undocumented in the CL manual.
I've opened bug #66267 for this.  I intend to work on it in the coming
days.

> > Here's the patch I propose to commit:

> OK, modulo Stefan's comment.

I've committed it, and I'm closing bug #65680 with this post.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Eli Zaretskii]

> Date: Fri, 29 Sep 2023 16:49:57 +0000
> Cc: monnier@iro.umontreal.ca, 65680-done@debbugs.gnu.org, acm@muc.de
> From: Alan Mackenzie <acm@muc.de>
> 
> > > Here's the patch I propose to commit:
> 
> > OK, modulo Stefan's comment.
> 
> I've committed it, and I'm closing bug #65680 with this post.

Thanks.

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Michael Heerdegen]

Hello Alan,

in 01229fe0096 "Stop truncating strings too much in
cl-print-string-with-limit" there is an error: the first `cond'
condition will error when limit is nil (`zerop' expects a number):

| @@ -557,6 +572,11 @@ cl-print-to-string-with-limit
|                          ((null limit) nil)
|                          ((eq limit t) print-level)
|                          (t (min 8 (truncate (log limit))))))
| +         (cl-print-string-length
| +          (cond
| +           ((or (null limit) (zerop limit)) nil)
| +           ((eq limit t) cl-print-string-length)
| +           (t (max 0 (- limit 3)))))

TIA,

Michael.

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Alan Mackenzie]

Hello, Michael.

On Tue, Oct 03, 2023 at 04:11:16 +0200, Michael Heerdegen wrote:
> Hello Alan,

> in 01229fe0096 "Stop truncating strings too much in
> cl-print-string-with-limit" there is an error: the first `cond'
> condition will error when limit is nil (`zerop' expects a number):

I think you mean when limit is t.  When limit is nil, the (null limit)
inside the `or' will catch it.

Thanks for pointing out this bug.  I'll fix it.

> | @@ -557,6 +572,11 @@ cl-print-to-string-with-limit
> |                          ((null limit) nil)
> |                          ((eq limit t) print-level)
> |                          (t (min 8 (truncate (log limit))))))
> | +         (cl-print-string-length
> | +          (cond
> | +           ((or (null limit) (zerop limit)) nil)
> | +           ((eq limit t) cl-print-string-length)
> | +           (t (max 0 (- limit 3)))))

> TIA,

> Michael.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Alan Mackenzie]

Hello, Michael.

On Tue, Oct 03, 2023 at 04:11:16 +0200, Michael Heerdegen wrote:
> Hello Alan,

> in 01229fe0096 "Stop truncating strings too much in
> cl-print-string-with-limit" there is an error: the first `cond'
> condition will error when limit is nil (`zerop' expects a number):

> | @@ -557,6 +572,11 @@ cl-print-to-string-with-limit
> |                          ((null limit) nil)
> |                          ((eq limit t) print-level)
> |                          (t (min 8 (truncate (log limit))))))
> | +         (cl-print-string-length
> | +          (cond
> | +           ((or (null limit) (zerop limit)) nil)
> | +           ((eq limit t) cl-print-string-length)
> | +           (t (max 0 (- limit 3)))))

> TIA,

This should now be fixed on the master branch.

> Michael.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#65680: cl-print-to-string-with-limit erroneously imposes a maximum print-length of 50

[Michael Heerdegen]

Alan Mackenzie <acm@muc.de> writes:

> I think you mean when limit is t.

Yes of course - sorry for the typo.

> This should now be fixed on the master branch.

Thank you.


Michael.