[PATCH] Add inspection command "source (,src)" which shows Scheme code of loaded module

[PATCH] Add inspection command "source (,src)" which shows Scheme code of loaded module

[Nala Ginrut]

[-- Attachment #1: Type: text/plain, Size: 1196 bytes --]

Attached patch added an inspection command "source" to display the
Scheme code of loaded module, it'll be useful for folks:

-------------Scheme proc------------
scheme@(guile-user)> ,use (srfi srfi-1)
scheme@(guile-user)> ,src any
(define (any pred ls . lists)
  (check-arg procedure? pred any)
  (if (null? lists)
    (any1 pred ls)
    (let lp ((lists (cons ls lists)))
      (cond ((any1 null? lists) #f)
            ((any1 null? (map cdr lists))
             (apply pred (map car lists)))
            (else
             (or (apply pred (map car lists))
                 (lp (map cdr lists))))))))
--------------------end-------------------------


C implemented proc will give a notion.
-------------------cut------------
scheme@(guile-user)> ,src read
"It's inner procedure implemented with C"
-------------------end------------------------


proc defined in REPL shows nothing.
-----------------cut-----------
scheme@(guile-user)> (define (func) 1)
scheme@(guile-user)> ,src func
-----------------end-----------

non-proc shows nothing.
--------------cut------------------
scheme@(guile-user)> (define a 1)
scheme@(guile-user)> ,src a
--------------end------------------


Happy hacking!


[-- Attachment #2: 0001-Add-src-command-in-REPL-to-show-Scheme-code-of-loade.patch --]
[-- Type: text/x-patch, Size: 2223 bytes --]

From 454af1f4326d600d6044de903b12812dfd9310ad Mon Sep 17 00:00:00 2001
From: Nala Ginrut <nalaginrut@gmail.com>
Date: Sat, 30 Mar 2013 21:48:35 +0800
Subject: [PATCH] Add src command in REPL to show Scheme code of loaded module

* system/repl/command.scm: Add inspection command "source (,src)"
  which shows Scheme code of loaded module.
---
 module/system/repl/command.scm |   36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)

diff --git a/module/system/repl/command.scm b/module/system/repl/command.scm
index 8ad00da..bda6dfe 100644
--- a/module/system/repl/command.scm
+++ b/module/system/repl/command.scm
@@ -65,7 +65,7 @@
               (tracepoint tp)
               (traps) (delete del) (disable) (enable)
               (registers regs))
-    (inspect  (inspect i) (pretty-print pp))
+    (inspect  (inspect i) (pretty-print pp) (source src))
     (system   (gc) (statistics stat) (option o)
               (quit q continue cont))))
 
@@ -869,6 +869,40 @@ Pretty-print the result(s) of evaluating EXP."
          (pp x))
        args))))
 
+(define (get-src source)
+  (define any (@ (srfi srfi-1) any))
+  (define (skip-lines port n)
+    (cond
+     ((zero? n) port)
+     (else (read-line port) (skip-lines port (1- n)))))
+  
+  (let* ((file (source:file source))
+         (line (source:line source))
+         (fp (any (lambda (x) 
+                    (let ((f (string-append x "/" file))) 
+                      (if (file-exists? f) (open-input-file f) #f))) %load-path)))
+    (skip-lines fp line)
+    (let ((src (read fp)))
+      (close fp)
+      src)))
+
+(define (print-src p)
+  (define (get-program-src p)
+    (let ((source (program-source p 0)))
+      (cond
+       ((not source) "It's inner procedure implemented with C")
+       ((not (source:file source)) #f)
+       (else (get-src source)))))
+  (let ((src (and (program? p) (get-program-src p))))
+    (and src (pp src))))
+
+(define-meta-command (source repl (form))
+  "source PROC
+Pretty-print the source code of PROC"
+  (call-with-values (repl-prepare-eval-thunk repl (repl-parse repl form))
+    (lambda args
+      (for-each print-src args))))
+
 \f
 ;;;
 ;;; System commands
-- 
1.7.10.4

Re: [PATCH] Add inspection command "source (, src)" which shows Scheme code of loaded module

[Mark H Weaver]

Nala Ginrut <nalaginrut@gmail.com> writes:

> Attached patch added an inspection command "source" to display the
> Scheme code of loaded module, it'll be useful for folks:
>
> -------------Scheme proc------------
> scheme@(guile-user)> ,use (srfi srfi-1)
> scheme@(guile-user)> ,src any
> (define (any pred ls . lists)
>   (check-arg procedure? pred any)
>   (if (null? lists)
>     (any1 pred ls)
>     (let lp ((lists (cons ls lists)))
>       (cond ((any1 null? lists) #f)
>             ((any1 null? (map cdr lists))
>              (apply pred (map car lists)))
>             (else
>              (or (apply pred (map car lists))
>                  (lp (map cdr lists))))))))
> --------------------end-------------------------

Nice hack! :)

I'd be glad to see something like this in Guile core.  This code is a
great demonstration, but it has some problems.

> From 454af1f4326d600d6044de903b12812dfd9310ad Mon Sep 17 00:00:00 2001
> From: Nala Ginrut <nalaginrut@gmail.com>
> Date: Sat, 30 Mar 2013 21:48:35 +0800
> Subject: [PATCH] Add src command in REPL to show Scheme code of loaded module
>
> * system/repl/command.scm: Add inspection command "source (,src)"
>   which shows Scheme code of loaded module.
> ---
>  module/system/repl/command.scm |   36 +++++++++++++++++++++++++++++++++++-
>  1 file changed, 35 insertions(+), 1 deletion(-)
>
> diff --git a/module/system/repl/command.scm b/module/system/repl/command.scm
> index 8ad00da..bda6dfe 100644
> --- a/module/system/repl/command.scm
> +++ b/module/system/repl/command.scm
> @@ -65,7 +65,7 @@
>                (tracepoint tp)
>                (traps) (delete del) (disable) (enable)
>                (registers regs))
> -    (inspect  (inspect i) (pretty-print pp))
> +    (inspect  (inspect i) (pretty-print pp) (source src))
>      (system   (gc) (statistics stat) (option o)
>                (quit q continue cont))))
>  
> @@ -869,6 +869,40 @@ Pretty-print the result(s) of evaluating EXP."
>           (pp x))
>         args))))
>  
> +(define (get-src source)
> +  (define any (@ (srfi srfi-1) any))
> +  (define (skip-lines port n)
> +    (cond
> +     ((zero? n) port)
> +     (else (read-line port) (skip-lines port (1- n)))))
> +  
> +  (let* ((file (source:file source))
> +         (line (source:line source))
> +         (fp (any (lambda (x) 
> +                    (let ((f (string-append x "/" file))) 
> +                      (if (file-exists? f) (open-input-file f) #f))) %load-path)))
> +    (skip-lines fp line)
> +    (let ((src (read fp)))
> +      (close fp)
> +      src)))

This strategy of reading the code is not robust.

* It assumes that the procedure is the first datum on the specified
  line.  This is not generally true.

* It assumes that there are no reader directives in the file (such as
  #!curly-infix) that affect the operation of the reader.  As is, your
  code will not work with any procedure that uses curly-infix.

* It assumes that the procedure is written in Scheme.

Also, as we discussed on IRC, it would be better to show the original
characters in the file instead of the sexp representation.  I want to
see the comments.  I want to see the programmer-chosen indentation.  If
they chose to use curly-infix for some expressions, I want to see that.
More generally, I want to see the presentation that the programmer
thought was most readable, i.e. the *source* code.

> +
> +(define (print-src p)
> +  (define (get-program-src p)
> +    (let ((source (program-source p 0)))
> +      (cond
> +       ((not source) "It's inner procedure implemented with C")

I'm not sure we can conclude that the procedure is implemented in C just
because 'program-source' returns #f.  There might be other reasons why
that might happen.

> +       ((not (source:file source)) #f)
> +       (else (get-src source)))))
> +  (let ((src (and (program? p) (get-program-src p))))
> +    (and src (pp src))))
> +
> +(define-meta-command (source repl (form))
> +  "source PROC
> +Pretty-print the source code of PROC"
> +  (call-with-values (repl-prepare-eval-thunk repl (repl-parse repl form))
> +    (lambda args
> +      (for-each print-src args))))
> +
>  \f
>  ;;;
>  ;;; System commands

     Regards,
       Mark

Re: [PATCH] Add inspection command "source (, src)" which shows Scheme code of loaded module

[Daniel Hartwig]

On 31 March 2013 05:17, Mark H Weaver <mhw@netris.org> wrote:
> Nala Ginrut <nalaginrut@gmail.com> writes:
>> +
>> +(define (print-src p)
>> +  (define (get-program-src p)
>> +    (let ((source (program-source p 0)))
>> +      (cond
>> +       ((not source) "It's inner procedure implemented with C")
>
> I'm not sure we can conclude that the procedure is implemented in C just
> because 'program-source' returns #f.  There might be other reasons why
> that might happen.
>

Yes right, instead of trying to interpret the meaning and display a
message, returning ‘#f’ like e.g. ‘describe’ should be sufficient.
This is enough of a clue to indicate that the material is not
available.

Re: [PATCH] Add inspection command "source (,src)" which shows Scheme code of loaded module

[Nala Ginrut]

On Sat, 2013-03-30 at 17:17 -0400, Mark H Weaver wrote:
> Nala Ginrut <nalaginrut@gmail.com> writes:
> 
> > Attached patch added an inspection command "source" to display the
> > Scheme code of loaded module, it'll be useful for folks:
> >
> > -------------Scheme proc------------
> > scheme@(guile-user)> ,use (srfi srfi-1)
> > scheme@(guile-user)> ,src any
> > (define (any pred ls . lists)
> >   (check-arg procedure? pred any)
> >   (if (null? lists)
> >     (any1 pred ls)
> >     (let lp ((lists (cons ls lists)))
> >       (cond ((any1 null? lists) #f)
> >             ((any1 null? (map cdr lists))
> >              (apply pred (map car lists)))
> >             (else
> >              (or (apply pred (map car lists))
> >                  (lp (map cdr lists))))))))
> > --------------------end-------------------------
> 
> Nice hack! :)
> 
> I'd be glad to see something like this in Guile core.  This code is a
> great demonstration, but it has some problems.
> 
> > From 454af1f4326d600d6044de903b12812dfd9310ad Mon Sep 17 00:00:00 2001
> > From: Nala Ginrut <nalaginrut@gmail.com>
> > Date: Sat, 30 Mar 2013 21:48:35 +0800
> > Subject: [PATCH] Add src command in REPL to show Scheme code of loaded module
> >
> > * system/repl/command.scm: Add inspection command "source (,src)"
> >   which shows Scheme code of loaded module.
> > ---
> >  module/system/repl/command.scm |   36 +++++++++++++++++++++++++++++++++++-
> >  1 file changed, 35 insertions(+), 1 deletion(-)
> >
> > diff --git a/module/system/repl/command.scm b/module/system/repl/command.scm
> > index 8ad00da..bda6dfe 100644
> > --- a/module/system/repl/command.scm
> > +++ b/module/system/repl/command.scm
> > @@ -65,7 +65,7 @@
> >                (tracepoint tp)
> >                (traps) (delete del) (disable) (enable)
> >                (registers regs))
> > -    (inspect  (inspect i) (pretty-print pp))
> > +    (inspect  (inspect i) (pretty-print pp) (source src))
> >      (system   (gc) (statistics stat) (option o)
> >                (quit q continue cont))))
> >  
> > @@ -869,6 +869,40 @@ Pretty-print the result(s) of evaluating EXP."
> >           (pp x))
> >         args))))
> >  
> > +(define (get-src source)
> > +  (define any (@ (srfi srfi-1) any))
> > +  (define (skip-lines port n)
> > +    (cond
> > +     ((zero? n) port)
> > +     (else (read-line port) (skip-lines port (1- n)))))
> > +  
> > +  (let* ((file (source:file source))
> > +         (line (source:line source))
> > +         (fp (any (lambda (x) 
> > +                    (let ((f (string-append x "/" file))) 
> > +                      (if (file-exists? f) (open-input-file f) #f))) %load-path)))
> > +    (skip-lines fp line)
> > +    (let ((src (read fp)))
> > +      (close fp)
> > +      src)))
> 
> This strategy of reading the code is not robust.
> 
> * It assumes that the procedure is the first datum on the specified
>   line.  This is not generally true.

Yes, I saw that. But I don't know how to get the procedure definition
line from program-source, since there's no sufficient docs for that.
Someone in IRC suggested me use the first datum.
Could you point me out how to do that? ;-)

> 
> * It assumes that there are no reader directives in the file (such as
>   #!curly-infix) that affect the operation of the reader.  As is, your
>   code will not work with any procedure that uses curly-infix.
> 
> * It assumes that the procedure is written in Scheme.
> 


> Also, as we discussed on IRC, it would be better to show the original
> characters in the file instead of the sexp representation.  I want to
> see the comments.  I want to see the programmer-chosen indentation.  If
> they chose to use curly-infix for some expressions, I want to see that.
> More generally, I want to see the presentation that the programmer
> thought was most readable, i.e. the *source* code.
> 

For these purposes above, I think there should be a modified 'read' to
read the code from source file with comments.

> > +
> > +(define (print-src p)
> > +  (define (get-program-src p)
> > +    (let ((source (program-source p 0)))
> > +      (cond
> > +       ((not source) "It's inner procedure implemented with C")
> 
> I'm not sure we can conclude that the procedure is implemented in C just
> because 'program-source' returns #f.  There might be other reasons why
> that might happen.
> 

Well, I'll take Daniel's suggestion to return #f if the source code is
unavailable.

> > +       ((not (source:file source)) #f)
> > +       (else (get-src source)))))
> > +  (let ((src (and (program? p) (get-program-src p))))
> > +    (and src (pp src))))
> > +
> > +(define-meta-command (source repl (form))
> > +  "source PROC
> > +Pretty-print the source code of PROC"
> > +  (call-with-values (repl-prepare-eval-thunk repl (repl-parse repl form))
> > +    (lambda args
> > +      (for-each print-src args))))
> > +
> >  \f
> >  ;;;
> >  ;;; System commands
> 
>      Regards,
>        Mark

Re: [PATCH] Add inspection command "source (, src)" which shows Scheme code of loaded module

[Daniel Hartwig]

On 31 March 2013 20:47, Nala Ginrut <nalaginrut@gmail.com> wrote:
> On Sat, 2013-03-30 at 17:17 -0400, Mark H Weaver wrote:
>> This strategy of reading the code is not robust.
>>
>> * It assumes that the procedure is the first datum on the specified
>>   line.  This is not generally true.
>
> Yes, I saw that. But I don't know how to get the procedure definition
> line from program-source, since there's no sufficient docs for that.
> Someone in IRC suggested me use the first datum.
> Could you point me out how to do that? ;-)
>

Use ‘source:column’ to get to the character where the definition
starts.

>>
>> * It assumes that there are no reader directives in the file (such as
>>   #!curly-infix) that affect the operation of the reader.  As is, your
>>   code will not work with any procedure that uses curly-infix.
>>
>> * It assumes that the procedure is written in Scheme.
>>
>
>
>> Also, as we discussed on IRC, it would be better to show the original
>> characters in the file instead of the sexp representation.  I want to
>> see the comments.  I want to see the programmer-chosen indentation.  If
>> they chose to use curly-infix for some expressions, I want to see that.
>> More generally, I want to see the presentation that the programmer
>> thought was most readable, i.e. the *source* code.
>>
>
> For these purposes above, I think there should be a modified 'read' to
> read the code from source file with comments.
>

There are other transformations that occur as Mark points out.  It is
far better to just return the unmodified source from the file.  As
discussed, the source file, column, line information gives you the
start, and ‘read’ can be used to locate the end:

 (let ((start (begin
                (skip-lines fp (source:line src))
                (seek fp (source:column src) SEEK_CUR)
                (ftell fp)))
       (end (begin
              (read fp)
              (ftell fp))))
   …

these two points are enough information to obtain the unmodified
source from the file.

Re: [PATCH] Add inspection command "source (, src)" which shows Scheme code of loaded module

[Mark H Weaver]

Daniel Hartwig <mandyke@gmail.com> writes:

> On 31 March 2013 20:47, Nala Ginrut <nalaginrut@gmail.com> wrote:
>> On Sat, 2013-03-30 at 17:17 -0400, Mark H Weaver wrote:
>>> Also, as we discussed on IRC, it would be better to show the original
>>> characters in the file instead of the sexp representation.  I want to
>>> see the comments.  I want to see the programmer-chosen indentation.  If
>>> they chose to use curly-infix for some expressions, I want to see that.
>>> More generally, I want to see the presentation that the programmer
>>> thought was most readable, i.e. the *source* code.
>>>
>>
>> For these purposes above, I think there should be a modified 'read' to
>> read the code from source file with comments.
>>
>
> There are other transformations that occur as Mark points out.  It is
> far better to just return the unmodified source from the file.  As
> discussed, the source file, column, line information gives you the
> start, and ‘read’ can be used to locate the end:
>
>  (let ((start (begin
>                 (skip-lines fp (source:line src))
>                 (seek fp (source:column src) SEEK_CUR)
>                 (ftell fp)))
>        (end (begin
>               (read fp)
>               (ftell fp))))
>    …
>
> these two points are enough information to obtain the unmodified
> source from the file.

This is enough to get the original characters, but then there's the
other problem I mentioned: reader directives such as #!curly-infix
earlier in the file.

For this reason, I think we need to use 'read' from the beginning of the
file, and look at the source properties of the returned datums to find
the right top-level datum.  In most cases, a top-level datum is what is
desired, but in some cases not.  In general, you will need to traverse
the sublists of a top-level datum to find the right one.

     Good luck!
        Mark

Re: [PATCH] Add inspection command "source (, src)" which shows Scheme code of loaded module

[Daniel Hartwig]

On 1 April 2013 11:54, Mark H Weaver <mhw@netris.org> wrote:
> Daniel Hartwig <mandyke@gmail.com> writes:
>> these two points are enough information to obtain the unmodified
>> source from the file.
>
> This is enough to get the original characters, but then there's the
> other problem I mentioned: reader directives such as #!curly-infix
> earlier in the file.
>
> For this reason, I think we need to use 'read' from the beginning of the
> file, and look at the source properties of the returned datums to find
> the right top-level datum.  In most cases, a top-level datum is what is
> desired, but in some cases not.  In general, you will need to traverse
> the sublists of a top-level datum to find the right one.

Ah yes, right.  Not such a trivial task after all.  I had neglected
that such reader directives will sufficiently alter the meaning of the
file.

I think I will stick to ‘geiser-edit-symbol-at-point’ :-)

Re: [PATCH] Add inspection command "source (, src)" which shows Scheme code of loaded module

[Andy Wingo]

On Sat 30 Mar 2013 22:17, Mark H Weaver <mhw@netris.org> writes:

> I'd be glad to see something like this in Guile core.  This code is a
> great demonstration, but it has some problems.

It would be nice to reify the original source as a compressed string in
the debug section of the ELF file.  One day...

Andy
-- 
http://wingolog.org/

Show source code from .debug_info section (Was Re: [PATCH] Add inspection command "source (, src)" which shows Scheme code of loaded module)

[Nala Ginrut]

On Mon, 2013-04-01 at 21:12 +0200, Andy Wingo wrote:
> On Sat 30 Mar 2013 22:17, Mark H Weaver <mhw@netris.org> writes:
> 
> > I'd be glad to see something like this in Guile core.  This code is a
> > great demonstration, but it has some problems.
> 
> It would be nice to reify the original source as a compressed string in
> the debug section of the ELF file.  One day...
> 
> Andy

I've checked out our latest wip-rtl and found .debug_info is on the TODO
list. So I think the better solution to show source code is to take
advantage of this.
But this feature depends on our future RTL, so my previous patch could
be useful for the 'older' guile2 user, since our REPL is well
extensible, that ',src' could be a plugin for that.

Thanks!