Enhancement suggestion: prin1 extension mechanism

Enhancement suggestion: prin1 extension mechanism

[Eric M. Ludlam]

Hi all,

  CEDET uses EIEIO, which represents CLOS like objects as vectors.
These are used extensively in CEDET, and in particular as simple
databases with some potentially circular structure.

  I've used some hacks to override edebug's prin1 to replace these
objects with syntax like #<semanticdb-table foo.c (100 tags)>, or
#<FIND RESULTS (5 tags)> where the actual printed representation from
prin1 might take a minute or so to generate.  (Or, at least, longer
than I've been willing to wait during a debug session tonight.)
Fortunately, M-: prints a short form with large parts replaced with
"..."

  ielm doesn't have such a mechanism, other than me editing my copy of
ielm to use the edebug prin1 symbol.  I would expect there are other
situations I haven't bumped into yet.

  It would be handy to have a way for a tool which knows it will
create large data structures that usually only need to be referred to
by a simple name to do that via a prin1 configuration.  It may be that
ielm and tools like it use some intermediate layer that is extensible
instead.  That would be fine with me too.

  At the end, find a file that has my edebug modifications as a
suggestion for a possible style, plus some sample code using it.  (It
has a couple key bindings not related to this request too.)  This code
has already been assigned via the CEDET assignment so it is safe to
recycle if desired.

  The full usage is a part of cedet.  http://cedet.sf.net

Thanks
Eric
---------------------
;;; cedet-edebug.el --- Special EDEBUG augmentation code

;;;
;; Copyright (C) 2003, 2004, 2007, 2008 Eric M. Ludlam
;;
;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 2, or (at your option)
;; any later version.
;;
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.
;;
;; You should have received a copy of the GNU General Public License
;; along with this program; if not, you can either send email to this
;; program's author (see below) or write to:
;;
;;              The Free Software Foundation, Inc.
;;              675 Mass Ave.
;;              Cambridge, MA 02139, USA.
;;
;; Please send bug reports, etc. to zappo@gnu.org

;;; Commentary:
;;
;; Some aspects of EDEBUG are not extensible.  It is possible to extend
;; edebug through other means, such as alias or advice, but those don't stack
;; very well when there are multiple tools trying to do the same sort of thing.
;;
;; This package provides a way to extend some aspects of edebug, such as value
;; printing.


;;; Code:
(defvar cedet-edebug-prin1-extensions
  nil
  "An alist of of code that can extend PRIN1 for edebug.
Each entry has the value: (CONDITION . PRIN1COMMAND).")

(defun cedet-edebug-prin1-recurse (object)
  "Recurse into OBJECT for prin1 on `cedet-edebug-prin1-to-string'."
  (concat "(" (mapconcat 'cedet-edebug-prin1-to-string object " ") ")"))

(defun cedet-edebug-rebuild-prin1 ()
  "Rebuild the function `cedet-edebug-prin1-to-string'.
Use the values of `cedet-edebug-prin1-extensions' as the means of
constructing the function."
  (interactive)
  (let ((c cedet-edebug-prin1-extensions)
	(code nil))
    (while c
      (setq code (append (list (list (car (car c))
				     (cdr (car c))))
			 code))
      (setq c (cdr c)))
    (fset 'cedet-edebug-prin1-to-string-inner
	  `(lambda (object &optional noescape)
	     "Display eieio OBJECT in fancy format.  Overrides the edebug default.
Optional argument NOESCAPE is passed to `prin1-to-string' when appropriate."
	     (cond
	      ,@(nreverse code)
	      (t (prin1-to-string object noescape)))))
    ))

(defun cedet-edebug-prin1-to-string (object &optional noescape)
  "CEDET version of `edebug-prin1-to-string' that adds specialty
print methods for very large complex objects."
  (if (not (fboundp 'cedet-edebug-prin1-to-string-inner))
      ;; Recreate the official fcn now.
      (cedet-edebug-rebuild-prin1))

  ;; Call the auto-generated version.
  ;; This is not going to be available at compile time.
  (cedet-edebug-prin1-to-string-inner object noescape))


(defun cedet-edebug-add-print-override (testfcn printfcn)
  "Add a new EDEBUG print override.
TESTFCN is a routine that returns nil if the first argument
passed to it is not to use PRINTFCN.
PRINTFCN accepts an object identified by TESTFCN and
returns a string.
New tests are always added to the END of the list of tests.
See `cedet-edebug-prin1-extensions' for the official list."
  (condition-case nil
      (add-to-list 'cedet-edebug-prin1-extensions
		   (cons testfcn printfcn)
		   t)
    (error ;; That failed, it must be an older version of Emacs
     ;; withouth the append argument for `add-to-list'
     ;; Doesn't handle the don't add twice case, but that's a
     ;; development thing and developers probably use new emacsen.
     (setq cedet-edebug-prin1-extensions
	   (append cedet-edebug-prin1-extensions
		   (list (cons testfcn printfcn))))))
  ;; whack the old implementation to force a rebuild.
  (fmakunbound 'cedet-edebug-prin1-to-string-inner))

;;; NOTE TO SELF.  Make this system used as an extension
;;; and then autoload the below.
;;;###autoload
(add-hook 'edebug-setup-hook
	  (lambda ()
	    (require 'cedet-edebug)
	    ;; I suspect this isn't the best way to do this, but when
	    ;; cust-print was used on my system all my objects
	    ;; appeared as "#1 =" which was not useful.  This allows
	    ;; edebug to print my objects in the nice way they were
	    ;; meant to with `object-print' and `class-name'
	    (defalias 'edebug-prin1-to-string 'cedet-edebug-prin1-to-string)
	    ;; Add a fancy binding into EDEBUG's keymap for ADEBUG.
	    (define-key edebug-mode-map "A" 'data-debug-edebug-expr)
	    ))

;;; DEBUG MODE TOO
;; This seems like as good a place as any to stick this hack.
;;;###autoload
(add-hook 'debugger-mode-hook
	  (lambda ()
	    (require 'cedet-edebug)
	    ;; Add a fancy binding into the debug mode map for ADEBUG.
	    (define-key debugger-mode-map "A" 'data-debug-edebug-expr)
	    ))

(provide 'cedet-edebug)

;;; cedet-edebug.el ends here


----------------

Here is a sample of usage in eieio.el

-----------------

(eval-after-load "cedet-edebug"
  '(progn
     (cedet-edebug-add-print-override '(class-p object) '(class-name object) )
     (cedet-edebug-add-print-override '(object-p object) '(object-print object) )
     (cedet-edebug-add-print-override '(and (listp object)
					    (or (class-p (car object)) (object-p (car object))))
				      '(cedet-edebug-prin1-recurse object) )
     ))

Re: Enhancement suggestion: prin1 extension mechanism

[Richard M. Stallman]

I have two questions:

* Why have cedet-edebug-prin1-to-string
construct a function definition rather than
scan the list cedet-edebug-prin1-extensions directly?

* Would it make sense to define the conditions in a more structured
(and thus, less general and more predictable) way?
For instance, could it fit naturally into the way EIEIO defines a class?

Re: Enhancement suggestion: prin1 extension mechanism

[Stefan Monnier]

>   I've used some hacks to override edebug's prin1 to replace these
> objects with syntax like #<semanticdb-table foo.c (100 tags)>, or
> #<FIND RESULTS (5 tags)> where the actual printed representation from
> prin1 might take a minute or so to generate.  (Or, at least, longer
> than I've been willing to wait during a debug session tonight.)
> Fortunately, M-: prints a short form with large parts replaced with
> "..."

There are 2 issues:
- the time to print the object
- the size and shape of the output
The first problem is probably a reflection of the naive way we handle
cycles (i.e. by scanning a singly-linked-list of already seen objects):
if someone were to fix this code to use a hash-table rather than a list,
the minute would probably come down to an acceptable time.  There are
already some places where we disable print-circle to avoid this slowdown
(but in your case, it's not an option since you data is circular).

Of course, you may still want to have an ad-hoc output that hides the
internal representation, even if the internal representation can be
printed in an acceptable amount of time.

Maybe we should introduce a notion of "object" (in the OO sense) in
Emacs's internal representation, so EIEIO (and CL's defstruct) could
then provide its own printer code.

The simplest way to do that might be to simply introduce a new array
type called `object', which would work just like array, except that when
it's printed, it's passed through
(run-hook-with-args-until-success 'object-print-functions obj), so EIEIO
and CL could hook into object-print-functions.  Or we could even stuff
a "class ID" small integer inside the array's size info, so we could
lookup an object-printer-table.


        Stefan

Re: Enhancement suggestion: prin1 extension mechanism

[Helmut Eller]

* Stefan Monnier [2008-09-13 04:22+0200] writes:

> Of course, you may still want to have an ad-hoc output that hides the
> internal representation, even if the internal representation can be
> printed in an acceptable amount of time.
>
> Maybe we should introduce a notion of "object" (in the OO sense) in
> Emacs's internal representation, so EIEIO (and CL's defstruct) could
> then provide its own printer code.
>
> The simplest way to do that might be to simply introduce a new array
> type called `object', which would work just like array, except that when
> it's printed, it's passed through
> (run-hook-with-args-until-success 'object-print-functions obj), so EIEIO
> and CL could hook into object-print-functions.  Or we could even stuff
> a "class ID" small integer inside the array's size info, so we could
> lookup an object-printer-table.

Instead of changing the internal representation, you could allow entries
for vector and cons etc. in the printer table.  That's how Common Lisp
allows you to customize the printer for built-in types.

Helmut.

Re: Enhancement suggestion: prin1 extension mechanism

[Stefan Monnier]

> Instead of changing the internal representation, you could allow entries
> for vector and cons etc. in the printer table.  That's how Common Lisp
> allows you to customize the printer for built-in types.

I fear it'd be difficult to make it work without incurring a significant
performance overhead.


        Stefan

Re: Enhancement suggestion: prin1 extension mechanism

[Helmut Eller]

* Stefan Monnier [2008-09-13 21:22+0200] writes:

>> Instead of changing the internal representation, you could allow entries
>> for vector and cons etc. in the printer table.  That's how Common Lisp
>> allows you to customize the printer for built-in types.
>
> I fear it'd be difficult to make it work without incurring a significant
> performance overhead.

If the table is nil you could use the default hardcoded printer without
lookups. 

If the entries in the table can be accessed from Lisp code, we could
remember the default print function for a type before installing a
custom printer.  E.g. if a vector doesn't have the special symbol in the
first entry, we could call the previously remembered default printer
instead of iterating manually over the vector.  Recursive calls to print
would again use the custom print function.

It seems to me that, compared to testing for circularity, those table
lookups would only introduce a minor slowdown.  And if someone produces
so much output the that it takes to long to print, he probably doesn't
want to read it anyway :-)

Helmut.

Re[2]: Enhancement suggestion: prin1 extension mechanism

[Eric M. Ludlam]

>>> "Richard M. Stallman" <rms@gnu.org> seems to think that:
>I have two questions:
>
>* Why have cedet-edebug-prin1-to-string
>construct a function definition rather than
>scan the list cedet-edebug-prin1-extensions directly?

I wrote that a long time ago, and don't remember why I chose that.  It
was likely to make it faster by embedding everything into a cond
statement, though I didn't do any profiling.

>* Would it make sense to define the conditions in a more structured
>(and thus, less general and more predictable) way?
>For instance, could it fit naturally into the way EIEIO defines a class?

All of EIEIO is supported by a single addition to this feature.  I
also use it for other data structures that are not EIEIO classes.

Eric

Re: Enhancement suggestion: prin1 extension mechanism

[Stefan Monnier]

>> I fear it'd be difficult to make it work without incurring a significant
>> performance overhead.

> If the table is nil you could use the default hardcoded printer without
> lookups.

In many important cases (e.g. CL has been loaded), the table wouldn't be
nil, and performance should still be good.

> If the entries in the table can be accessed from Lisp code, we could
> remember the default print function for a type before installing a
> custom printer.  E.g. if a vector doesn't have the special symbol in the
> first entry, we could call the previously remembered default printer
> instead of iterating manually over the vector.  Recursive calls to print
> would again use the custom print function.

I don't know what you're talking about.  I'd probably understand better
with a piece of code.

> It seems to me that, compared to testing for circularity, those table
> lookups would only introduce a minor slowdown.

Testing for circularity is indeed too slow right now.  That should be
fixed as well.

> And if someone produces so much output the that it takes to long to
> print, he probably doesn't want to read it anyway :-)

But it's difficult for Emacs to find ou before it's too late.


        Stefan

Re: Enhancement suggestion: prin1 extension mechanism

[Helmut Eller]

* Stefan Monnier [2008-09-14 21:35+0200] writes:

>>> I fear it'd be difficult to make it work without incurring a significant
>>> performance overhead.
>
>> If the table is nil you could use the default hardcoded printer without
>> lookups.
>
> In many important cases (e.g. CL has been loaded), the table wouldn't be
> nil, and performance should still be good.

Why?  It could be nil by default, just like print-circle is nil by
default.

>> If the entries in the table can be accessed from Lisp code, we could
>> remember the default print function for a type before installing a
>> custom printer.  E.g. if a vector doesn't have the special symbol in the
>> first entry, we could call the previously remembered default printer
>> instead of iterating manually over the vector.  Recursive calls to print
>> would again use the custom print function.
>
> I don't know what you're talking about.  I'd probably understand better
> with a piece of code.

(defvar my-printer-table 
  (let* ((table (make-pprint-dispatch-table))
         (default (get-pprint-dispatch table 'vector)))
    (set-pprint-dispatch table 'vector 
                         (lambda (obj stream)
                           (if (eq (aref obj 0) 'my-type)
                               (princ "#<my-type>" stream)
                               (funcall default obj stream))))
    table))

(defun my-prin1 (obj stream)
  (let ((pprint-dispatch-table my-printer-table))
    (prin1 obj stream)))

The interface functions would be make-pprint-dispatch-table,
get-pprint-dispatch, and set-pprint-dispatch. pprint-dispatch-table
contains the current table in use.

>> And if someone produces so much output the that it takes to long to
>> print, he probably doesn't want to read it anyway :-)
>
> But it's difficult for Emacs to find ou before it's too late.

The user can press C-g if it takes to long.  Oh wait, print isn't
interruptible.  To bad.
 
Helmut.

Re: Enhancement suggestion: prin1 extension mechanism

[Richard M. Stallman]

    >* Would it make sense to define the conditions in a more structured
    >(and thus, less general and more predictable) way?
    >For instance, could it fit naturally into the way EIEIO defines a class?

    All of EIEIO is supported by a single addition to this feature.  I
    also use it for other data structures that are not EIEIO classes.

IF we are going to have an extension of this kind, it would be much
cleaner to built it into print.c, using data structures convenient for
print.c to implement.

Re: Enhancement suggestion: prin1 extension mechanism

[Ted Zlatanov]

On Mon, 15 Sep 2008 09:23:26 -0400 "Richard M. Stallman" <rms@gnu.org> wrote: 
RMS> Eric Ludlam wrote:
RMS> EL> All of EIEIO is supported by a single addition to this feature.  I
RMS> EL> also use it for other data structures that are not EIEIO classes.

RMS> IF we are going to have an extension of this kind, it would be much
RMS> cleaner to built it into print.c, using data structures convenient for
RMS> print.c to implement.

Sorry to jump in, but I recently asked about a way to cleanly dump and
restore a hashtable.  Can that work coincide with the enhancement Eric
is discussing, which if I understand correctly is a condensed prin1
suitable for debugging?  I mention it because it seems sensible to think
both in the direction of Eric's work and in the direction of deeper
inspection of the built-in Emacs Lisp data structures.  Maybe it can be
simply a "dump level" parameter to prin1, going from "condensed" to
"normal" to "deep," separate from the print-level parameter or joined
with it.

I don't know print.c (yet), but if no one else is interested in doing
this, I can put the hashtable dump functionality on my TODO list.

Thanks
Ted