bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[Eric Abrahamsen]

EIEIO class instances can either be represented using the read-only
#<class-name object string> style, or as vectors, dumping all the slot
information. For large or recursive objects, this can take some time, and
can also make for very large strings. In backtraces and edebug, in
particular, this can make it very difficult to see what's happening.

Objects have an `object-print' method which produces the #<foo>
representation, but this appears to only sometimes be used in edebug,
and never in backtraces.

It would be great if object-print could always be used, or if there were
an option to toggle this.

In GNU Emacs 26.0.50.9 (x86_64-unknown-linux-gnu, GTK+ Version 3.22.5)
 of 2016-12-26 built on clem
Repository revision: 65b997b95e284e2edc1266663e39791f68d76ad7
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
Recent messages:
 [6 times]
mwheel-scroll: Beginning of buffer [5 times]
Mark saved where search started
Checking new news...
Reading active file from Test via nngnorb...done
Reading active file from archive via nnfolder...done
Reading active file via nndraft...done
Reading active file from MTester via nnmairix...done
Checking new news...done
Processing kill file /home/eric/.emacs.d/News/gmane.emacs.devel.KILL...done

bug#25295: Acknowledgement (26.0.50; Represent eieio objects using object-print in backtraces and edebug)

[Eric Abrahamsen]

I guess this would require going into print.c and adding another branch
under the Lisp_Vectorlike case statement of object_print.

Is this sort of C code allowed to call back up to the lisp object-print
function? This is where I get lost...

bug#25295: Acknowledgement (26.0.50; Represent eieio objects using object-print in backtraces and edebug)

[npostavs]

severity 25295 wishlist
quit

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I guess this would require going into print.c and adding another branch
> under the Lisp_Vectorlike case statement of object_print.

You mean print_object?  I thought the idea would be to change edebug and
backtrace code to (optionally?) use object-print instead of print.

bug#25295: Acknowledgement (26.0.50; Represent eieio objects using object-print in backtraces and edebug)

[Eli Zaretskii]

> From: Eric Abrahamsen <eric@ericabrahamsen.net>
> Date: Thu, 29 Dec 2016 13:50:09 -0800
> 
> I guess this would require going into print.c and adding another branch
> under the Lisp_Vectorlike case statement of object_print.
> 
> Is this sort of C code allowed to call back up to the lisp object-print
> function?

If that Lisp function will then call print.c again, that's not a good
idea, since print.c internally uses a buffer by a certain fixed name.

bug#25295: Acknowledgement (26.0.50; Represent eieio objects using object-print in backtraces and edebug)

[Eric Abrahamsen]

On 12/29/16 22:42 PM, npostavs@users.sourceforge.net wrote:

> You mean print_object?  I thought the idea would be to change edebug and
> backtrace code to (optionally?) use object-print instead of print.

I do mean print_object, sorry. (And sorry I keep replying to you only!)

On 12/30/16 09:51 AM, Eli Zaretskii wrote:
>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>> Date: Thu, 29 Dec 2016 13:50:09 -0800
>> 
>> I guess this would require going into print.c and adding another branch
>> under the Lisp_Vectorlike case statement of object_print.
>> 
>> Is this sort of C code allowed to call back up to the lisp object-print
>> function?
>
> If that Lisp function will then call print.c again, that's not a good
> idea, since print.c internally uses a buffer by a certain fixed name.

`object-print' ends up using `format', which looks like it calls print.c
functions, so I guess that's out.

I don't know the right level at which to intervene. All other lisp
objects get a hard-coded #<obj representation from print_object in
print.c, only eieio objects "fake it" with a user-overrideable lisp
function. I suspect eieio objects won't be considered "fundamental" on
the same level as markers, buffers, etc., so maybe they don't belong in
print_object (plus the above problem of calling lisp-c-lisp-c).

I don't see how we could hijack at the lisp level, though. Functions
like `eval-expression' and `backtrace--print-frame' simply toss whole
lisp structures to prin1, there's no way to know that there's an eieio
object somewhere in that structure.

Personally, I'd be willing to lose the ability to customize object
representations with `object-print', if it meant that print_object could
produce a #<obj notation for eieio objects. That would mean writing a 
C test like INSTANCEP or what have you.

I don't know what the right solution is!

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

bug#25295: Acknowledgement (26.0.50; Represent eieio objects using object-print in backtraces and edebug)

[npostavs]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:
> On 12/30/16 09:51 AM, Eli Zaretskii wrote:
>>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>>> Date: Thu, 29 Dec 2016 13:50:09 -0800
>>> 
>>> I guess this would require going into print.c and adding another branch
>>> under the Lisp_Vectorlike case statement of object_print.
>>> 
>>> Is this sort of C code allowed to call back up to the lisp object-print
>>> function?
>>
>> If that Lisp function will then call print.c again, that's not a good
>> idea, since print.c internally uses a buffer by a certain fixed name.
>
> `object-print' ends up using `format', which looks like it calls print.c
> functions, so I guess that's out.
>
> I don't know the right level at which to intervene. All other lisp
> objects get a hard-coded #<obj representation from print_object in
> print.c, only eieio objects "fake it" with a user-overrideable lisp
> function. I suspect eieio objects won't be considered "fundamental" on
> the same level as markers, buffers, etc., so maybe they don't belong in
> print_object (plus the above problem of calling lisp-c-lisp-c).
>
> I don't see how we could hijack at the lisp level, though. Functions
> like `eval-expression' and `backtrace--print-frame' simply toss whole
> lisp structures to prin1, there's no way to know that there's an eieio
> object somewhere in that structure.
>

I think the only way to integrate `object-print' with the existing
`print' functions, would be to make it follow the same protocol.  That
is, currently `object-print' is really `object-to-string', it should be
changed (or perhaps a new function (e.g., `print-object') would be a
better idea, so as not to break existing code too much) to accept a
PRINTCHARFUN argument, and print to it.

> Personally, I'd be willing to lose the ability to customize object
> representations with `object-print', if it meant that print_object could
> produce a #<obj notation for eieio objects. That would mean writing a 
> C test like INSTANCEP or what have you.
>

That's easier, of course, but a non-customized representation would be
pretty uninformative.

bug#25295: Acknowledgement (26.0.50; Represent eieio objects using object-print in backtraces and edebug)

[Eric Abrahamsen]

On 12/31/16 00:48 AM, npostavs@users.sourceforge.net wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:

[...]

>> I don't see how we could hijack at the lisp level, though. Functions
>> like `eval-expression' and `backtrace--print-frame' simply toss whole
>> lisp structures to prin1, there's no way to know that there's an eieio
>> object somewhere in that structure.
>>
>
> I think the only way to integrate `object-print' with the existing
> `print' functions, would be to make it follow the same protocol.  That
> is, currently `object-print' is really `object-to-string', it should be
> changed (or perhaps a new function (e.g., `print-object') would be a
> better idea, so as not to break existing code too much) to accept a
> PRINTCHARFUN argument, and print to it.

The problem is that pretty much all of the printing happens at the C
level. Whole lisp structures are sent directly to C, and it's the C code
that recurses through them and decides how to print everything it finds
inside. Lisp code never gets a chance (except in a few very specific
situations).

For example: when an error is raised, `backtrace--print-frame' gets all
the contents of the error as a single argument. It simply punts that to
`prin1', and then it's done. There's no chance to pick apart that single
argument and see if there is an object inside. `eval-expression'
essentially does the same thing.

>> Personally, I'd be willing to lose the ability to customize object
>> representations with `object-print', if it meant that print_object could
>> produce a #<obj notation for eieio objects. That would mean writing a 
>> C test like INSTANCEP or what have you.
>>
>
> That's easier, of course, but a non-customized representation would be
> pretty uninformative.

Having looked at the code, I'm not too optimistic about achieving the
ideal solution. Getting eval-expression and backtraces to stop exploding
seems like enough for now.

bug#25295: Acknowledgement (26.0.50; Represent eieio objects using object-print in backtraces and edebug)

[npostavs]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> On 12/31/16 00:48 AM, npostavs@users.sourceforge.net wrote:
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>> I think the only way to integrate `object-print' with the existing
>> `print' functions, would be to make it follow the same protocol.  That
>> is, currently `object-print' is really `object-to-string', it should be
>> changed (or perhaps a new function (e.g., `print-object') would be a
>> better idea, so as not to break existing code too much) to accept a
>> PRINTCHARFUN argument, and print to it.
>
> The problem is that pretty much all of the printing happens at the C
> level. Whole lisp structures are sent directly to C, and it's the C code
> that recurses through them and decides how to print everything it finds
> inside. Lisp code never gets a chance (except in a few very specific
> situations).
>
> For example: when an error is raised, `backtrace--print-frame' gets all
> the contents of the error as a single argument. It simply punts that to
> `prin1', and then it's done. There's no chance to pick apart that single
> argument and see if there is an object inside. `eval-expression'
> essentially does the same thing.
>

You would need to add a branch in print_object to detect eieio objects,
same as for the other alternative.

>>> Personally, I'd be willing to lose the ability to customize object
>>> representations with `object-print', if it meant that print_object could
>>> produce a #<obj notation for eieio objects. That would mean writing a 
>>> C test like INSTANCEP or what have you.
>>>
>>
>> That's easier, of course, but a non-customized representation would be
>> pretty uninformative.
>
> Having looked at the code, I'm not too optimistic about achieving the
> ideal solution. Getting eval-expression and backtraces to stop exploding
> seems like enough for now.

bug#25295: Acknowledgement (26.0.50; Represent eieio objects using object-print in backtraces and edebug)

[Eric Abrahamsen]

On 12/31/16 14:23 PM, npostavs@users.sourceforge.net wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> On 12/31/16 00:48 AM, npostavs@users.sourceforge.net wrote:
>>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>>
>>> I think the only way to integrate `object-print' with the existing
>>> `print' functions, would be to make it follow the same protocol.  That
>>> is, currently `object-print' is really `object-to-string', it should be
>>> changed (or perhaps a new function (e.g., `print-object') would be a
>>> better idea, so as not to break existing code too much) to accept a
>>> PRINTCHARFUN argument, and print to it.
>>
>> The problem is that pretty much all of the printing happens at the C
>> level. Whole lisp structures are sent directly to C, and it's the C code
>> that recurses through them and decides how to print everything it finds
>> inside. Lisp code never gets a chance (except in a few very specific
>> situations).
>>
>> For example: when an error is raised, `backtrace--print-frame' gets all
>> the contents of the error as a single argument. It simply punts that to
>> `prin1', and then it's done. There's no chance to pick apart that single
>> argument and see if there is an object inside. `eval-expression'
>> essentially does the same thing.
>>
>
> You would need to add a branch in print_object to detect eieio objects,
> same as for the other alternative.

Well sure, that's what this whole thread started out with.

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> I guess this would require going into print.c and adding another branch
> under the Lisp_Vectorlike case statement of object_print.

That's right.

> Is this sort of C code allowed to call back up to the lisp object-print
> function? This is where I get lost...

The Elisp-level printer and the interaction between the C code and
the Elisp ill have to be worked out, indeed.

E.g. you probably won't want to have Elisp-level printer to just call
`format` or `prin1` since that does more than just print (it sets up
a printing destination, initializes the cycle-detection table, then
checks for cycles, ...).

This said, making `prin1` reentrant will probably be helpful.

One more thing: there have already been other desires/needs to tweak the
printing from Elisp.  E.g. to special case printing of objects that
can't be printed readably (e.g. buffers, markers, overlays, ...).


        Stefan


PS: One more thing; currently cl-structs and EIEIO objects are
represented as plain normal arrays (and they mostly use the same
representation now, where the first field contains a symbol whose
function cell contains `:quick-object-witness-check` and whose value
cell is the actual class object, which is a cl-struct that inherits
from cl--class).

It'd be good to instead provide a new kind of array just for
objects/records/structs (just like we have the `compiled-function` type
which is basically a normal array just with a different tag).
This would let us get rid of the "symbol with
a :quick-object-witness-check" indirection and would make it easier for
the print.c code to delegate to the Elisp-level printer since it
wouldn't need to know about the hackish representation choices of EIEIO
and cl-structs.

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Eric Abrahamsen]

On 01/03/17 13:21 PM, Stefan Monnier wrote:
>> I guess this would require going into print.c and adding another branch
>> under the Lisp_Vectorlike case statement of object_print.
>
> That's right.
>
>> Is this sort of C code allowed to call back up to the lisp object-print
>> function? This is where I get lost...
>
> The Elisp-level printer and the interaction between the C code and
> the Elisp ill have to be worked out, indeed.
>
> E.g. you probably won't want to have Elisp-level printer to just call
> `format` or `prin1` since that does more than just print (it sets up
> a printing destination, initializes the cycle-detection table, then
> checks for cycles, ...).
>
> This said, making `prin1` reentrant will probably be helpful.

If an Elisp-level printer is available to users and developers, I think
we'd have to assume they will throw any old thing in there, including
`format' and `prin1'. Would making `prin1' reentrant solve all of the
resulting problems?

> One more thing: there have already been other desires/needs to tweak the
> printing from Elisp.  E.g. to special case printing of objects that
> can't be printed readably (e.g. buffers, markers, overlays, ...).

That's good! But the previous issue would still need to be resolved.

>
>         Stefan
>
>
> PS: One more thing; currently cl-structs and EIEIO objects are
> represented as plain normal arrays (and they mostly use the same
> representation now, where the first field contains a symbol whose
> function cell contains `:quick-object-witness-check` and whose value
> cell is the actual class object, which is a cl-struct that inherits
> from cl--class).
>
> It'd be good to instead provide a new kind of array just for
> objects/records/structs (just like we have the `compiled-function` type
> which is basically a normal array just with a different tag).
> This would let us get rid of the "symbol with
> a :quick-object-witness-check" indirection and would make it easier for
> the print.c code to delegate to the Elisp-level printer since it
> wouldn't need to know about the hackish representation choices of EIEIO
> and cl-structs.

I really wish I could help, but I've never written a line of C, and
in twenty minutes of looking through the sources earlier today, I
couldn't even find where the array types were defined.

It sounds like this would be Step One of solving the problem, but I
would not be able to do it without some substantial hand-holding.

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> If an Elisp-level printer is available to users and developers, I think
> we'd have to assume they will throw any old thing in there, including
> `format' and `prin1'. Would making `prin1' reentrant solve all of the
> resulting problems?

Maybe the better solution would be to make `prin1` (and `format`, ..)
signal errors.  Instead, we could provide the printer with a special
prin1-like function which basically "continues" the print job in the
C code.

>> One more thing: there have already been other desires/needs to tweak the
>> printing from Elisp.  E.g. to special case printing of objects that
>> can't be printed readably (e.g. buffers, markers, overlays, ...).

> That's good! But the previous issue would still need to be resolved.

Yes, and they're largely independent: the printer you're talking about
would be used when we don't want to print readably, whereas this case
would apply to the situation where we want to print readably (in which
case we don't want to go through custom pretty printers like the one you
need).


        Stefan

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> Maybe the better solution would be to make `prin1` (and `format`, ..)
> signal errors.  Instead, we could provide the printer with a special
> prin1-like function which basically "continues" the print job in the
> C code.

Actually, print.c is already setup to be (at least partially) reentrant
(for the case `printcharfun` is a function), so maybe making it fully
reentrant won't be too hard.  But we'd still want to provide some ad-hoc
prin1-like function which continues the print job rather than starting
a new nested one; this is needed e.g. if the custom printer wants to
print a value V but V has already been printed so it should be printed
as a reference #NN# to the previous one.


        Stefan

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> Actually, print.c is already setup to be (at least partially) reentrant
> (for the case `printcharfun` is a function), so maybe making it fully
> reentrant won't be too hard.  But we'd still want to provide some ad-hoc
> prin1-like function which continues the print job rather than starting
> a new nested one; this is needed e.g. if the custom printer wants to
> print a value V but V has already been printed so it should be printed
> as a reference #NN# to the previous one.

Looking further at the code, maybe this isn't even needed, since the
print-number-table can survive calls to print.  But that can get ugly
when your printer code also want to call `print` for "unrelated"
purposes (e.g. to print debug messages in a side buffer).
And I'll stop thinking about interaction with concurrency.


        Stefan

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 04 Jan 2017 23:37:26 -0500
> Cc: 25295@debbugs.gnu.org
> 
> And I'll stop thinking about interaction with concurrency.

There should be none, since only one thread running Lisp could be
doing that at any given time.  Right?

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

>> And I'll stop thinking about interaction with concurrency.
> There should be none, since only one thread running Lisp could be
> doing that at any given time.  Right?

I'm thinking of

   prin1
   -> enters print.c
   -> calls back the custom Elisp-side object-print
   -> which decides to accept-process-output


-- Stefan

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: eric@ericabrahamsen.net,  25295@debbugs.gnu.org
> Date: Thu, 05 Jan 2017 19:42:55 -0500
> 
>    prin1
>    -> enters print.c
>    -> calls back the custom Elisp-side object-print
>    -> which decides to accept-process-output

This will cause problems without any threads as well, because the last
line might cause timers to run.

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[npostavs]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Actually, print.c is already setup to be (at least partially) reentrant
>> (for the case `printcharfun` is a function), so maybe making it fully
>> reentrant won't be too hard.  But we'd still want to provide some ad-hoc
>> prin1-like function which continues the print job rather than starting
>> a new nested one; this is needed e.g. if the custom printer wants to
>> print a value V but V has already been printed so it should be printed
>> as a reference #NN# to the previous one.
>
> Looking further at the code, maybe this isn't even needed, since the
> print-number-table can survive calls to print.  But that can get ugly
> when your printer code also want to call `print` for "unrelated"
> purposes (e.g. to print debug messages in a side buffer).
> And I'll stop thinking about interaction with concurrency.

Can we allow overriding printing of primitive types too?  I'm wanting
that for e.g., printing byte code functions in nicer ways.

Looking to Common Lisp for inspiration, there is generic function
print-object[1], and also *print-pprint-dispatch* (an alist of type
specifiers -> printer functions).

Cycle detection state seems to be packed in with STREAM, which is
convenient, although dispatching on STREAM becomes harder.

[1]: http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/stagenfun_print-object.html
[2]: http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/sec_22-2-1-4.html

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> Can we allow overriding printing of primitive types too?
> I'm wanting that for e.g., printing byte code functions in nicer ways.

Maybe we should just switch to an Elisp version of printing, in that case.
We could keep the C code for the "print-readably" case only.
The main question is whether it's fast enough.


        Stefan


;;; cl-print.el --- Generic printer facilies         -*- lexical-binding: t; -*-

;; Copyright (C) 2017  Stefan Monnier

;; Author: Stefan Monnier <monnier@iro.umontreal.ca>
;; Keywords: 

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;;; Code:

;;;###autoload
(cl-defgeneric cl-print-object (object stream)
  "Dispatcher to print OBJECT on STREAM according to its type."
  (prin1 object stream))

(cl-defmethod cl-print-object ((object cons) stream)
  (let ((car (pop object)))
    (if (and (memq car '(\, quote \` \,@ \,.))
             (consp object)
             (null (cdr object)))
        (progn
          (princ (if (eq car 'quote) '\' car) stream)
          (cl-print-object (car object) stream))
      (princ "(" stream)
      (cl-print-object car stream)
      (while (consp object)
        (princ " " stream)
        (cl-print-object (pop object) stream))
      (when object
        (princ " . ") (cl-print-object object stream))
      (princ ")"))))

(cl-defmethod cl-print-object ((object vector) stream)
  (princ "[" stream)
  (dotimes (i (length object))
    (unless (zerop i) (princ " " stream))
    (cl-print-object (aref object i) stream))
  (princ "]" stream))

(cl-defmethod cl-print-object ((object compiled-function) stream)
  ;; FIXME: Give a prettier representation.
  (princ "#<compiled-function>" stream))

(cl-defmethod cl-print-object ((object cl-structure-object) stream)
  (princ "#s(")
  (let* ((class (symbol-value (aref object 0)))
         (slots (cl--struct-class-slots class)))
    (princ (cl--struct-class-name class) stream)
    (dotimes (i (length slots))
      (let ((slot (aref slots i)))
        (princ " :" stream)
        (princ (cl--slot-descriptor-name slot) stream)
        (princ " " stream)
        (cl-print-object (aref object (1+ i)) stream))))
  (princ ")"))

;;; Circularity and sharing.

;; I don't try to support the `print-continuous-numbering', because
;; I think it's ill defined anyway: if an object appears only once in each call
;; its sharing can't be properly preserved!

(defvar cl-print--number-index nil)
(defvar cl-print--number-table nil)

(cl-defmethod cl-print-object :around (object stream)
  ;; FIXME: Only put such an :around method on types where it's relevant.
  (let ((n (if cl-print--number-table (gethash object cl-print--number-table))))
    (if (not (numberp n))
        (cl-call-next-method)
      (if (> n 0)
          ;; Already printed.  Just print a reference.
          (progn (princ "#" stream) (princ n stream) (princ "#" stream))
        (puthash object (- n) cl-print--number-table)
        (princ "#" stream) (princ (- n) stream) (princ "=" stream)
        (cl-call-next-method)))))

(defun cl-print--find-sharing (object table)
  (unless
      ;; Skip objects which don't have identity!
      (or (floatp object) (numberp object))
    (let ((n (gethash object table)))
      (cond
       ((numberp n))                   ;All done.
       (n                              ;Already seen, but only once.
        (let ((n (1+ cl-print--number-index)))
          (setq cl-print--number-index n)
          (puthash object (- n) table)))
       (t
         (puthash object t table)
         (pcase object
           (`(,car . ,cdr)
            (cl-print--find-sharing car table)
            (cl-print--find-sharing cdr table))
           ((pred stringp)
            ;; We presumably won't print its text-properties.
            nil)
           ((pred arrayp)             ;FIXME: Inefficient for char-tables!
            (dotimes (i (length object))
              (cl-print--find-sharing (aref object i) table)))))))))

;;;###autoload
(defun cl-prin1 (object &optional stream)
  (if (not print-circle)
      (cl-print-object object stream)
    (let ((cl-print--number-table (make-hash-table :test 'eq))
          (cl-print--number-index 0))
      (cl-print--find-sharing object cl-print--number-table)
      (cl-print-object object stream))))

(provide 'cl-print)
;;; cl-print.el ends here

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Most of the time, the speed of printing is not important.
But I suspect there are cases where people print a lot of data
and the speed of printing is crucial.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

bug#25295: Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> Most of the time, the speed of printing is not important.
> But I suspect there are cases where people print a lot of data
> and the speed of printing is crucial.

Indeed.  I know `C-h v load-history` is often sluggish, for example,
because of the size of the value.

I've been trying my cl-print code and the speed is clearly inferior to
the C code, but at least in the case of C-h v the main slowdown comes
from the prettifying step that happens after printing (where we insert
newlines and indent the result), so while it gets measurably slower, the
impact might be tolerable.

Clearly, for cases such as when the byte-compiler wants to print the
result into a .elc file, or when we want to save an undo-log in a file,
speed will be very important and my cl-print is unlikely to be
good enough.  That's why I was suggesting that maybe we should have
2 printers: a fast one in C for the `print-readably` case, and
a slower but customizable one in Elisp for "human consumption".


        Stefan

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> Objects have an `object-print' method which produces the #<foo>
> representation, but this appears to only sometimes be used in edebug,
> and never in backtraces.

I've installed a cl-print.el package into Emacs's master branch which is
like EIEIO's object-print but more thorough.  This is now used in
edebug, so it should hopefully fix the "only sometimes" part (and it
additionally knows about cl-defstructs as well, rather than only EIEIO
objects).

Using it in backtraces should be a simple matter of applying the patch
below.  I haven't installed it because I'm afraid of the potential
breakage and slowdown of using cl-prin1, so I'd rather wait and see how
it fares in Edebug first.  My experiments seem to indicate that it might
be "good enough" but it might also suffer from the overhead of
cl-generic (mostly the fact that a call to a generic-function ends up
going through some &rest+apply which in cl-print's case end up
generating a lot of garbage: in one of my tests, the cl-print code was
twice slower than the C code but the added time was almost entirely
spent in the GC and that seemed to come exclusively from the lists
allocated for &rest.  I.e. if we could change cl-generic so it doesn't
use &rest+apply as much there's a chance it could potentially be about
as fast as the C code).


        Stefan


diff --git a/lisp/subr.el b/lisp/subr.el
index ea159e733c..68bfceb8d2 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -4660,10 +4660,10 @@ backtrace--print-frame
   (princ (if (plist-get flags :debug-on-exit) "* " "  "))
   (cond
    ((and evald (not debugger-stack-frame-as-list))
-    (prin1 func)
-    (if args (prin1 args) (princ "()")))
+    (cl-prin1 func)
+    (if args (cl-prin1 args) (princ "()")))
    (t
-    (prin1 (cons func args))))
+    (cl-prin1 (cons func args))))
   (princ "\n"))
 
 (defun backtrace ()

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[npostavs]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> My experiments seem to indicate that it might
> be "good enough" but it might also suffer from the overhead of
> cl-generic (mostly the fact that a call to a generic-function ends up
> going through some &rest+apply which in cl-print's case end up
> generating a lot of garbage: in one of my tests, the cl-print code was
> twice slower than the C code but the added time was almost entirely
> spent in the GC and that seemed to come exclusively from the lists
> allocated for &rest.  I.e. if we could change cl-generic so it doesn't
> use &rest+apply as much there's a chance it could potentially be about
> as fast as the C code).

I have an idea to use this for the pretty printer so it that produces
the correct spacing while printing.  I think this would end up faster
than calling indent-sexp afterwards.  But it would involve dispatching
on STREAM, and according to the comments in cl-generic.el multiple
dispatch adds more apply calls.  Thoughts?

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> I have an idea to use this for the pretty printer so it that produces
> the correct spacing while printing.  I think this would end up faster
> than calling indent-sexp afterwards.  But it would involve dispatching
> on STREAM, and according to the comments in cl-generic.el multiple
> dispatch adds more apply calls.  Thoughts?

No idea, how would that work?


        Stefan

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[Eric Abrahamsen]

On 03/02/17 02:00 AM, Stefan Monnier wrote:
>> Objects have an `object-print' method which produces the #<foo>
>> representation, but this appears to only sometimes be used in edebug,
>> and never in backtraces.
>
> I've installed a cl-print.el package into Emacs's master branch which is
> like EIEIO's object-print but more thorough.  This is now used in
> edebug, so it should hopefully fix the "only sometimes" part (and it
> additionally knows about cl-defstructs as well, rather than only EIEIO
> objects).

Very cool! How does this interact with the existing `object-print'
method? That method seems to be getting called at some point, but I
don't see where. Is the intention that we'd stop using that?

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> Very cool! How does this interact with the existing `object-print'
> method?

The default cl-print-object method for EIEIO objects calls object-print.
So you can add methods to either of object-print or cl-print-object
(modulo issues with ordering between methods if one is applied to
cl-print-object and the other to object-print).

> That method seems to be getting called at some point, but I
> don't see where. Is the intention that we'd stop using that?

Yes, object-print is now obsolete.


        Stefan

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[npostavs]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I have an idea to use this for the pretty printer so it that produces
>> the correct spacing while printing.  I think this would end up faster
>> than calling indent-sexp afterwards.  But it would involve dispatching
>> on STREAM, and according to the comments in cl-generic.el multiple
>> dispatch adds more apply calls.  Thoughts?
>
> No idea, how would that work?

I posted an initial draft at
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25122#21, I'm not seeing
as much speedup as I'd hoped, though I haven't tried optimizing it yet.

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[Eric Abrahamsen]

On 03/02/17 02:00 AM, Stefan Monnier wrote:
>> Objects have an `object-print' method which produces the #<foo>
>> representation, but this appears to only sometimes be used in edebug,
>> and never in backtraces.
>
> I've installed a cl-print.el package into Emacs's master branch which is
> like EIEIO's object-print but more thorough.  This is now used in
> edebug, so it should hopefully fix the "only sometimes" part (and it
> additionally knows about cl-defstructs as well, rather than only EIEIO
> objects).
>
> Using it in backtraces should be a simple matter of applying the patch
> below.  I haven't installed it because I'm afraid of the potential
> breakage and slowdown of using cl-prin1, so I'd rather wait and see how
> it fares in Edebug first.  My experiments seem to indicate that it might
> be "good enough" but it might also suffer from the overhead of
> cl-generic (mostly the fact that a call to a generic-function ends up
> going through some &rest+apply which in cl-print's case end up
> generating a lot of garbage: in one of my tests, the cl-print code was
> twice slower than the C code but the added time was almost entirely
> spent in the GC and that seemed to come exclusively from the lists
> allocated for &rest.  I.e. if we could change cl-generic so it doesn't
> use &rest+apply as much there's a chance it could potentially be about
> as fast as the C code).
>
>
>         Stefan
>
>
> diff --git a/lisp/subr.el b/lisp/subr.el
> index ea159e733c..68bfceb8d2 100644
> --- a/lisp/subr.el
> +++ b/lisp/subr.el
> @@ -4660,10 +4660,10 @@ backtrace--print-frame
>    (princ (if (plist-get flags :debug-on-exit) "* " "  "))
>    (cond
>     ((and evald (not debugger-stack-frame-as-list))
> -    (prin1 func)
> -    (if args (prin1 args) (princ "()")))
> +    (cl-prin1 func)
> +    (if args (cl-prin1 args) (princ "()")))
>     (t
> -    (prin1 (cons func args))))
> +    (cl-prin1 (cons func args))))
>    (princ "\n"))
>  
>  (defun backtrace ()

How about we just apply this and see what happens? Developing with EIEIO
is pretty painful -- oftentimes the *Messages* buffer is such a mess you
can't even tell if there's been an error or not.

I've been running the above patch, as well as an equivalent change in
`eval-expression', and nothing bad has happened. Perhaps there's a
slowdown, but from my own selfish point of view any slowdown is far, far
preferable to the several seconds you need to wait for a class object to
be spat out into *Messages*.

BTW, this works in edebug but only at the top level: if a nested
structure is printed, interior objects are still represented as the full
struct.

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[Stefan Monnier]

> How about we just apply this and see what happens?

AFAIK, that's exactly what's been done in Emacs-26 already.


        Stefan

bug#25295: 26.0.50; Represent eieio objects using object-print in backtraces and edebug

[Eric Abrahamsen]

This is either done, or thoroughly out of date.

On 10/21/17 23:21 PM, Stefan Monnier wrote:
>> How about we just apply this and see what happens?
>
> AFAIK, that's exactly what's been done in Emacs-26 already.
>
>
>         Stefan