Re: Using guile as an extension language for GNU make

[Paul Smith <psmith@gnu.org>]

On Tue, 2011-09-20 at 18:17 +0200, Thien-Thi Nguyen wrote:
> () Paul Smith <psmith@gnu.org>
> () Mon, 19 Sep 2011 11:14:34 -0400
> 
>    In make, everything is just words: broken up on whitespace.  So for
>    example, maybe someone writes a Guile function that computes a complex
>    set of prerequisites for a target:
> 
>            target: $(guile (...some Guile program...))
> 
> Even before thinking about the return value (which, scanning ahead,
> you seem to have hit upon a workable plan), i have some questions
> about the "some Guile program":
> 
> - Is that just a procedure call or can it be definition(s) + expression(s)?
>   e.g., single proc call:
>     $(guile (compute-complex-prerequisites "$@"))
>   e.g., complex:
>     $(guile (use-modules (srfi srfi-13))
>             (define (ext filename)
>               (string-append filename ".ext"))
>             (define (ext-all ls)
>               (map ext ls))
>             (define normalize  ;;; nb: alias
>               string-tokenize)
>             (define (compute-complex-prerequisites s)
>               (ext-all (normalize s)))
>             ;; comment: do it!
>             (trace 'compute-complex-prerequisites "$@")
>             (compute-complex-prerequisites "$@"))

I showed the code in my original post, but the way I've implemented it
is that the argument to the make "guile" function (that is, everything
after the "$(guile " to the closing paren--it's handy that Guile parens
will virtually always match, but you can use ${guile ...} instead if you
prefer), will first be make-expanded (so all verbatim instances of "$"
in the Guile script will need to be escaped as "$$"--it doesn't seem
like Guile uses "$" very much so this shouldn't be too painful) and then
the result will be passed to scm_c_eval_string().

As far as I understand it, that means the Guile program can be as
complex as you want.  HOWEVER, you will need to use backslashes to
escape newlines in your example above.

You can also use make's define/endef if you want to write long Guile
programs and avoid the need for backslashes, then you can run $(guile
$(DEFINEDVARIABLE)).

Then make will convert the SCM returned from scm_c_eval_string() into a
string buffer as discussed below etc., and use that as the expansion of
the $(guile ...) function.

> - Would stuff like ‘normalize’ and ‘trace’ be provided (builtin)?

You are now running far beyond my understanding of Guile :-).  If it
works in the environment I describe above then it will work.  If not,
then I'll need more details.

However I just tried your example above and I got an error "unbound
variable: trace" so there's something missing here.  I'll have to read
about that.

> - What happens if the stuff inside the double quotes includes
>   $(call...) or $(eval...)?  What about those constructs elsewhere?

As with all GNU make expansions, quotes are ignored.  The ONLY special
character, to make, in an expansion is "$".  Just as with call and eval,
if you do not escape the "$" then it will be expanded BEFORE the Guile
function is invoked.  So, if you are using (make-expand ...) etc. in
your Guile program you'll need to escape all the "$" there.

There is definitely some complexity here and you have to FIRMLY
understand make's expansion rules, just as you do when you use call and
eval make functions, or even when writing shell script rules.  Some
debugging capability, to show the script we are passing to the guile
function, seems like it would be quite useful.

I suppose it's possible that we could introduce a new, more advanced
parser for the guile function that took into account quotes.  But I'm
nervous about this: it's completely different than other expansions, and
it might be just as annoying as doing it the other way.

> In Scheme, w/ SRFI 13, you could express this as:
> 
> (define (as-string x)
>   (cond ((not x) "")
>         ((unspecified? x) "")
>         ((eq? #t x) "t")
>         ((string? x) x)
>         ((null? x) "")
>         (else (object->string x))))
>         
> (define (space-sep x)
>   (let ((acc '()))
>     (define (walk x)
>       (cond ((pair? x)
>              (walk (car x))
>              (walk (cdr x)))
>             ((null? x))
>             (else
>              (set! acc (cons x acc)))))
>     (walk x)
>     (string-join (map as-string (reverse! acc)))))
>  
> This is not so tricky, i think.

Heh, cute!  I haven't done much Lisp of any kind since I wrote a number
of elisp packages, years ago.  I think I'll have to dust off my Scheme.

>    Well, since Guile is not required and I want GNU make to continue to
>    work as-is on systems where Guile is not available, I won't be rewriting
>    core features in Guile.   Yet?!?!  :-).
> 
> IMHO ‘patsubst’ is not a core feature (sez the spewful ignoramus).  Rather,
> i think rewriting ‘patsubst’ would be a good exercise to help you formulate
> and answer questions about small technical details that will eventually serve
> as a foundation for rewriting core features.  Why not give it a try?

Maybe as an exercise, but patsubst IS a core feature, in that it has
existed forever and there are thousands of makefiles that use it...
basically rewriting existing capabilities so they're not available if
Guile is not present changes things from "GNU make with Guile optional"
to "GNU make with Guile required".  I'm not ready to go there.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@gnu.org>          Find some GNU make tips at:
 http://www.gnu.org                      http://make.mad-scientist.net
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist