RE: Getting all symbols in a Scheme file as a list

[M <maximedevos@telenet.be>]

>Onderwerp: Getting all symbols in a Scheme file as a list

The following will give you a list of all the symbols (unless I made some syntax errors) (also replace ‘ by a proper quote, e-mail program is corrupting it):

(use-modules (ice-9 match))
(define (all-symbols s-exp)
  (match s-exp
    ((? symbol? a) (list a))
    ((a . b) (append (all-symbols a) (all-symbols b)))
   ;; insert rules for vectors and arrays
    […]
    (_ ‘())))
[ also open file with open-input-file, apply ‘read’ to the port and pass the result to all-symbols, in a loop, and append the results together. ].

Doesn’t seem useful for what you are mentioning later, though …

>I'm trying to write a Guile script to trace symbol definition and reference
between modules in a large Guile repo (GNU/Guix), for the purposes of large
scale refactoring.

There is no such thing as symbol definitions in Scheme – you can make symbols with symbol->string, but you can’t define a symbol to anything, symbols simply are.

You can, however, define variables, which have a symbol as name (and that name may depend on context in case of hygienic macros or renamed imports/exports in modules).

>I'm wondering how I could programmatically get all the values in a 
Scheme file
as an S-expression. From the manual, I know that the REPL has meta keyword
',binding' and ',apropos'
<https://www.gnu.org/software/guile/manual/guile.html#index-apropos> 
which let
you search and list all bindings in accessible to a module. These are 
exactly
what I need only, since they're meta-commands, they don't produce Scheme
expressions.

>Does anyone have any pointers? Should I go down the route of,

>    (open-input-pipe (string-append "guile -l" file "-c ,binding"))

>? Seems a little bit baroque to me, I'd expect a simpler way of doing 
it. Any
libraries anyone knows of?

No. The quoting is incorrect when file has spaces, \, …, guile in PATH might not be the Guile that is being run, … (For the former, consider open-input-pipe*.)

You can probably find a simpler way of doing it by locating where “,binding” is implemented (in the Guile source code). It probably uses the module reflection API. This API is documented in

https://www.gnu.org/software/guile/manual/html_node/Module-System-Reflection.html

However, the documentation is incomplete – IIRC there is a procedure ‘module-bindings’ to find a list of top-level definitions, but it doesn’t seem to be documented.

To find what modules a module uses, there is module-uses, though I doubt it is reliable w.r.t. optimisation, inlining, uses of (@ (module name) variable), …

So, instead, I would propose to instead work  on the Tree-IL level (https://www.gnu.org/software/guile/manual/html_node/Tree_002dIL.html). In particular, see <toplevel-define> for definitions and <module-ref>,<module-set!> for uses of other modules.

This also allows for more fine-grained information – e.g. these <module-ref> etc. objects contain the location in the source code, and if a <module-ref> is inside a <toplevel-define> then you know that it is the procedure (assuming it is a procedure) of the <toplevel-define> that uses the variable of the <module-ref>.

(See https://www.gnu.org/software/guile/manual/html_node/The-Scheme-Compiler.html for how to compile stuff – also, IIRC, the compilation procedure accepts ports instead of only S-expressions, despite what the example suggests.)

Best regards,
Maxime Devos