* local-eval on syntax-local-binding, bound-identifiers
@ 2012-01-15 22:17 Andy Wingo
2012-01-16 3:30 ` Mark H Weaver
0 siblings, 1 reply; 9+ messages in thread
From: Andy Wingo @ 2012-01-15 22:17 UTC (permalink / raw)
To: Mark H. Weaver; +Cc: guile-devel
[-- Attachment #1: Type: text/plain, Size: 791 bytes --]
Hi Mark,
I had made some noise about preferring an implementation of local-eval
based on primitives from psyntax. But, I didn't clarify my argument by
providing the primitives. Here are some patches that provide
syntax-local-binding, as I noted in my previous mail, and also a
procedure to get all identifiers that are visible within the scope of
another identifier.
I then refactored your patch to implement local-eval entirely in terms
of these primitives and other normal macrology. What do you think? In
the interests of time and debugging, I removed the support for pattern
variables; it should be easy to add back.
These are preliminary patches, but if this approach proves to be viable,
I would prefer it to one that bakes the-environment into psyntax.
WDYT?
Regards,
Andy
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-add-syntax-local-binding.patch --]
[-- Type: text/x-diff, Size: 4372 bytes --]
From 48d8e52e316984f2bf9380df85079bb5fa142253 Mon Sep 17 00:00:00 2001
From: Andy Wingo <wingo@pobox.com>
Date: Sun, 15 Jan 2012 17:51:02 +0100
Subject: [PATCH 1/3] add syntax-local-binding
* module/ice-9/boot-9.scm (syntax-local-binding): New binding.
* module/ice-9/psyntax.scm: Locally define a fluid that holds the
"transformer environment". with-transformer-environment calls a
procedure with the transformer environment, or raises an error if
called outside the extent of a transformer. Bind
transformer-environment in expand-macro.
(syntax-local-binding): New procedure to return binding information of
a lexically bound identifier (a lexical, local macro, a pattern
variable, or a displaced lexical).
---
module/ice-9/boot-9.scm | 1 +
module/ice-9/psyntax.scm | 39 +++++++++++++++++++++++++++++++++++++--
2 files changed, 38 insertions(+), 2 deletions(-)
diff --git a/module/ice-9/boot-9.scm b/module/ice-9/boot-9.scm
index f661d08..9cdd8d1 100644
--- a/module/ice-9/boot-9.scm
+++ b/module/ice-9/boot-9.scm
@@ -389,6 +389,7 @@ If there is no handler at all, Guile prints an error and then exits."
(define generate-temporaries #f)
(define bound-identifier=? #f)
(define free-identifier=? #f)
+(define syntax-local-binding #f)
;; $sc-dispatch is an implementation detail of psyntax. It is used by
;; expanded macros, to dispatch an input against a set of patterns.
diff --git a/module/ice-9/psyntax.scm b/module/ice-9/psyntax.scm
index 1bf3c32..30685bc 100644
--- a/module/ice-9/psyntax.scm
+++ b/module/ice-9/psyntax.scm
@@ -786,6 +786,14 @@
id))))))
(else (syntax-violation 'id-var-name "invalid id" id)))))
+ (define transformer-environment
+ (make-fluid
+ (lambda (k)
+ (error "called outside the dynamic extent of a syntax transformer"))))
+
+ (define (with-transformer-environment k)
+ ((fluid-ref transformer-environment) k))
+
;; free-id=? must be passed fully wrapped ids since (free-id=? x y)
;; may be true even if (free-id=? (wrap x w) (wrap y w)) is not.
@@ -1321,8 +1329,10 @@
(syntax-violation #f "encountered raw symbol in macro output"
(source-wrap e w (wrap-subst w) mod) x))
(else (decorate-source x s)))))
- (rebuild-macro-output (p (source-wrap e (anti-mark w) s mod))
- (new-mark))))
+ (with-fluids ((transformer-environment
+ (lambda (k) (k e r w s rib mod))))
+ (rebuild-macro-output (p (source-wrap e (anti-mark w) s mod))
+ (new-mark)))))
(define expand-body
;; In processing the forms of the body, we create a new, empty wrap.
@@ -2435,6 +2445,31 @@
(set! syntax-source
(lambda (x) (source-annotation x)))
+ (set! syntax-local-binding
+ (lambda (id)
+ (arg-check nonsymbol-id? id 'syntax-local-value)
+ (with-transformer-environment
+ (lambda (e r w s rib mod)
+ (define (strip-anti-mark w)
+ (let ((ms (wrap-marks w)) (s (wrap-subst w)))
+ (if (and (pair? ms) (eq? (car ms) the-anti-mark))
+ ;; output is from original text
+ (make-wrap (cdr ms) (if rib (cons rib (cdr s)) (cdr s)))
+ ;; output introduced by macro
+ (make-wrap ms (if rib (cons rib s) s)))))
+ (let ((label (id-var-name (syntax-object-expression id)
+ (strip-anti-mark (syntax-object-wrap id)))))
+ (if (not (string? label))
+ (error "identifier not lexically bound" id))
+ (let ((b (assq-ref r label)))
+ (if b
+ (case (binding-type b)
+ ((lexical) (values 'lexical (binding-value b)))
+ ((macro) (values 'local-macro (binding-value b)))
+ ((syntax) (values 'pattern-variable (binding-value b)))
+ (else (error "unpossible!" b)))
+ (values 'displaced-lexical #f))))))))
+
(set! generate-temporaries
(lambda (ls)
(arg-check list? ls 'generate-temporaries)
--
1.7.8.3
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-add-bound-identifiers.patch --]
[-- Type: text/x-diff, Size: 3836 bytes --]
From 2c3da44320019453115811af386febaa7eb241c3 Mon Sep 17 00:00:00 2001
From: Andy Wingo <wingo@pobox.com>
Date: Sun, 15 Jan 2012 18:39:44 +0100
Subject: [PATCH 2/3] add bound-identifiers
* module/ice-9/boot-9.scm (bound-identifiers): Declare variable.
* module/ice-9/psyntax.scm: Add all-bound-identifiers helper, and define
bound-identifiers. The identifiers are anti-marked so that syntax
transformers can introduce them, as-is.
---
module/ice-9/boot-9.scm | 1 +
module/ice-9/psyntax.scm | 49 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 50 insertions(+), 0 deletions(-)
diff --git a/module/ice-9/boot-9.scm b/module/ice-9/boot-9.scm
index 9cdd8d1..b8aa842 100644
--- a/module/ice-9/boot-9.scm
+++ b/module/ice-9/boot-9.scm
@@ -389,6 +389,7 @@ If there is no handler at all, Guile prints an error and then exits."
(define generate-temporaries #f)
(define bound-identifier=? #f)
(define free-identifier=? #f)
+(define bound-identifiers #f)
(define syntax-local-binding #f)
;; $sc-dispatch is an implementation detail of psyntax. It is used by
diff --git a/module/ice-9/psyntax.scm b/module/ice-9/psyntax.scm
index 30685bc..25543e0 100644
--- a/module/ice-9/psyntax.scm
+++ b/module/ice-9/psyntax.scm
@@ -786,6 +786,48 @@
id))))))
(else (syntax-violation 'id-var-name "invalid id" id)))))
+ ;;
+ ;; all-bound-identifiers returns a list of all lexically bound
+ ;; identifiers, as syntax objects. They are in order from outer to
+ ;; inner.
+ ;;
+ (define all-bound-identifiers
+ (lambda (w mod)
+ (define scan
+ (lambda (subst results)
+ (if (null? subst)
+ results
+ (let ((fst (car subst)))
+ (if (eq? fst 'shift)
+ (scan (cdr subst) results)
+ (let ((symnames (ribcage-symnames fst))
+ (marks (ribcage-marks fst)))
+ (if (vector? symnames)
+ (scan-vector-rib subst symnames marks results)
+ (scan-list-rib subst symnames marks results))))))))
+ (define scan-list-rib
+ (lambda (subst symnames marks results)
+ (let f ((symnames symnames) (marks marks) (results results))
+ (if (null? symnames)
+ (scan (cdr subst) results)
+ (f (cdr symnames) (cdr marks)
+ (cons (wrap (car symnames)
+ (anti-mark (make-wrap (car marks) subst))
+ mod)
+ results))))))
+ (define scan-vector-rib
+ (lambda (subst symnames marks results)
+ (let ((n (vector-length symnames)))
+ (let f ((i 0) (results results))
+ (if (fx= i n)
+ (scan (cdr subst) results)
+ (f (fx+ i 1)
+ (cons (wrap (vector-ref symnames i)
+ (anti-mark (make-wrap (vector-ref marks i) subst))
+ mod)
+ results)))))))
+ (scan (wrap-subst w) '())))
+
(define transformer-environment
(make-fluid
(lambda (k)
@@ -2470,6 +2512,13 @@
(else (error "unpossible!" b)))
(values 'displaced-lexical #f))))))))
+ (set! bound-identifiers
+ (lambda (x)
+ (arg-check nonsymbol-id? x 'bound-identifiers)
+ (reverse
+ (all-bound-identifiers (syntax-object-wrap x)
+ (syntax-object-module x)))))
+
(set! generate-temporaries
(lambda (ls)
(arg-check list? ls 'generate-temporaries)
--
1.7.8.3
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-Implement-local-eval-local-compile-and-the-environme.patch --]
[-- Type: text/x-diff, Size: 17213 bytes --]
From ddea51310227155e3771c3e6acbbecf24dc74c42 Mon Sep 17 00:00:00 2001
From: Mark H Weaver <mhw@netris.org>
Date: Tue, 3 Jan 2012 04:02:08 -0500
Subject: [PATCH 3/3] Implement `local-eval', `local-compile', and
`the-environment'
* module/ice-9/local-eval.scm: New module (ice-9 local-eval) which
exports `the-environment', `local-eval', and `local-compile'.
* libguile/debug.c (scm_local_eval): New C function that calls the
Scheme implementation of `local-eval' in (ice-9 local-eval).
* libguile/debug.h (scm_local_eval): Add prototype.
* doc/ref/api-evaluation.texi (Local Evaluation): Add documentation.
* test-suite/tests/eval.test (local evaluation): Add tests.
* test-suite/standalone/test-loose-ends.c (test_scm_local_eval):
Add test.
* module/Makefile.am: Add ice-9/local-eval.scm.
---
doc/ref/api-evaluation.texi | 38 ++++++++
libguile/debug.c | 13 +++-
libguile/debug.h | 4 +-
module/Makefile.am | 5 +-
module/ice-9/local-eval.scm | 150 +++++++++++++++++++++++++++++++
test-suite/standalone/test-loose-ends.c | 16 +++-
test-suite/tests/eval.test | 79 ++++++++++++++++-
7 files changed, 298 insertions(+), 7 deletions(-)
create mode 100644 module/ice-9/local-eval.scm
diff --git a/doc/ref/api-evaluation.texi b/doc/ref/api-evaluation.texi
index 2e48dcb..72dd4df 100644
--- a/doc/ref/api-evaluation.texi
+++ b/doc/ref/api-evaluation.texi
@@ -19,6 +19,7 @@ loading, evaluating, and compiling Scheme code at run time.
* Loading:: Loading Scheme code from file.
* Character Encoding of Source Files:: Loading non-ASCII Scheme code from file.
* Delayed Evaluation:: Postponing evaluation until it is needed.
+* Local Evaluation:: Evaluation in a local lexical environment.
@end menu
@@ -954,6 +955,43 @@ value.
@end deffn
+@node Local Evaluation
+@subsection Local Evaluation
+
+@deffn syntax the-environment
+Captures and returns a lexical environment for use with
+@code{local-eval} or @code{local-compile}.
+@end deffn
+
+@deffn {Scheme Procedure} local-eval exp env
+@deffnx {C Function} scm_local_eval (exp, env)
+Evaluate the expression @var{exp} in the lexical environment @var{env}.
+This mostly behaves as if @var{exp} had been wrapped in a lambda
+expression @code{`(lambda () ,@var{exp})} and put in place of
+@code{(the-environment)}, with the resulting procedure called by
+@code{local-eval}. In other words, @var{exp} is evaluated within the
+lexical environment of @code{(the-environment)}, but within the dynamic
+environment of the call to @code{local-eval}.
+@end deffn
+
+@deffn {Scheme Procedure} local-compile exp env [opts=()]
+Compile the expression @var{exp} in the lexical environment @var{env}.
+If @var{exp} is a procedure, the result will be a compiled procedure;
+otherwise @code{local-compile} is mostly equivalent to
+@code{local-eval}. @var{opts} specifies the compilation options.
+@end deffn
+
+Note that the current implementation of @code{(the-environment)} has
+some limitations. It does not capture local syntax transformers bound
+by @code{let-syntax}, @code{letrec-syntax} or non-top-level
+@code{define-syntax} forms. Any attempt to reference such captured
+syntactic keywords via @code{local-eval} or @code{local-compile}
+produces an error. Also, @code{(the-environment)} does not capture
+lexical bindings that are shadowed by inner bindings with the same name,
+nor hidden lexical bindings produced by macro expansion, even though
+such bindings might be accessible using syntax objects.
+
+
@c Local Variables:
@c TeX-master: "guile.texi"
@c End:
diff --git a/libguile/debug.c b/libguile/debug.c
index 88a01d6..d41acc4 100644
--- a/libguile/debug.c
+++ b/libguile/debug.c
@@ -1,5 +1,5 @@
/* Debugging extensions for Guile
- * Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2003, 2006, 2008, 2009, 2010, 2011 Free Software Foundation
+ * Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2003, 2006, 2008, 2009, 2010, 2011, 2012 Free Software Foundation
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public License
@@ -208,6 +208,17 @@ SCM_DEFINE (scm_debug_hang, "debug-hang", 0, 1, 0,
#undef FUNC_NAME
#endif
+SCM
+scm_local_eval (SCM exp, SCM env)
+{
+ static SCM local_eval_var = SCM_BOOL_F;
+
+ if (scm_is_false (local_eval_var))
+ local_eval_var = scm_c_module_lookup
+ (scm_c_resolve_module ("ice-9 local-eval"), "local-eval");
+ return scm_call_2 (SCM_VARIABLE_REF (local_eval_var), exp, env);
+}
+
static void
init_stack_limit (void)
{
diff --git a/libguile/debug.h b/libguile/debug.h
index d862aba..4155d19 100644
--- a/libguile/debug.h
+++ b/libguile/debug.h
@@ -3,7 +3,7 @@
#ifndef SCM_DEBUG_H
#define SCM_DEBUG_H
-/* Copyright (C) 1995,1996,1998,1999,2000,2001,2002,2004,2008,2009,2010
+/* Copyright (C) 1995,1996,1998,1999,2000,2001,2002,2004,2008,2009,2010,2012
* Free Software Foundation, Inc.
*
* This library is free software; you can redistribute it and/or
@@ -41,6 +41,8 @@ typedef union scm_t_debug_info
\f
+SCM_API SCM scm_local_eval (SCM exp, SCM env);
+
SCM_API SCM scm_reverse_lookup (SCM env, SCM data);
SCM_API SCM scm_procedure_source (SCM proc);
SCM_API SCM scm_procedure_name (SCM proc);
diff --git a/module/Makefile.am b/module/Makefile.am
index 56fa48d..9c9d8ed 100644
--- a/module/Makefile.am
+++ b/module/Makefile.am
@@ -1,6 +1,6 @@
## Process this file with automake to produce Makefile.in.
##
-## Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+## Copyright (C) 2009, 2010, 2011, 2012 Free Software Foundation, Inc.
##
## This file is part of GUILE.
##
@@ -243,7 +243,8 @@ ICE_9_SOURCES = \
ice-9/weak-vector.scm \
ice-9/list.scm \
ice-9/serialize.scm \
- ice-9/vlist.scm
+ ice-9/vlist.scm \
+ ice-9/local-eval.scm
SRFI_SOURCES = \
srfi/srfi-1.scm \
diff --git a/module/ice-9/local-eval.scm b/module/ice-9/local-eval.scm
new file mode 100644
index 0000000..cb74881
--- /dev/null
+++ b/module/ice-9/local-eval.scm
@@ -0,0 +1,150 @@
+;;; -*- mode: scheme; coding: utf-8; -*-
+;;;
+;;; Copyright (C) 2012 Free Software Foundation, Inc.
+;;;
+;;; This library is free software; you can redistribute it and/or
+;;; modify it under the terms of the GNU Lesser General Public
+;;; License as published by the Free Software Foundation; either
+;;; version 3 of the License, or (at your option) any later version.
+;;;
+;;; This library is distributed in the hope that it will be useful,
+;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+;;; Lesser General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU Lesser General Public
+;;; License along with this library; if not, write to the Free Software
+;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+(define-module (ice-9 local-eval)
+ #:use-module (ice-9 format)
+ #:use-module (srfi srfi-9)
+ #:use-module (srfi srfi-9 gnu)
+ #:use-module (system base compile)
+ #:export (the-environment local-eval local-compile))
+
+(define-record-type lexical-environment-type
+ (make-lexical-environment module wrapper boxes)
+ lexical-environment?
+ (module lexenv-module)
+ (wrapper lexenv-wrapper)
+ (boxes lexenv-boxes))
+
+(set-record-type-printer!
+ lexical-environment-type
+ (lambda (e port)
+ (format port "#<lexical-environment ~S (~S bindings)>"
+ (module-name (lexenv-module e)) (length (lexenv-boxes e)))))
+
+(define-syntax-rule (make-box v)
+ (case-lambda
+ (() v)
+ ((x) (set! v x))))
+
+(define-syntax let*-syntax
+ (syntax-rules ()
+ ((_ () e e* ...)
+ (begin e e* ...))
+ ((_ ((id trans) (id* trans*) ...) e e* ...)
+ (let-syntax ((id trans))
+ (let*-syntax ((id* trans*) ...)
+ e e* ...)))))
+
+(define-syntax-rule (identifier-syntax-from-box box)
+ (make-transformer-from-box
+ (syntax-object-of box)
+ (identifier-syntax (id (box))
+ ((set! id x) (box x)))))
+
+(define-syntax syntax-object-of
+ (lambda (form)
+ (syntax-case form ()
+ ((_ x) #`(quote #,(datum->syntax #'x #'x))))))
+
+(define (make-transformer-from-box id trans)
+ (set-procedure-property! trans 'identifier-syntax-box id)
+ trans)
+
+(define (unsupported-binding name)
+ (make-variable-transformer
+ (lambda (x)
+ (syntax-violation
+ 'local-eval
+ "unsupported binding captured by (the-environment)"
+ x))))
+
+(define (partition-identifiers ids)
+ (let lp ((ids ids) (lexical '()) (other '()))
+ (if (null? ids)
+ (values lexical other)
+ (call-with-values (lambda () (syntax-local-binding (car ids)))
+ (lambda (type val)
+ (cond
+ ((eq? type 'lexical)
+ (lp (cdr ids)
+ (acons (car ids) (datum->syntax #'here (gensym))
+ lexical)
+ other))
+ ((and (eq? type 'local-macro)
+ (procedure-property val 'identifier-syntax-box))
+ => (lambda (id)
+ (lp (cdr ids)
+ (acons (car ids) id lexical)
+ other)))
+ (else
+ (lp (cdr ids) lexical (cons (car ids) other)))))))))
+
+(define-syntax the-environment
+ (lambda (x)
+ (syntax-case x ()
+ ((the-environment)
+ #'(the-environment the-environment))
+ ((the-environment scope)
+ (call-with-values (lambda ()
+ (partition-identifiers (bound-identifiers #'scope)))
+ (lambda (lexical other)
+ (with-syntax ((module (datum->syntax #'here (module-name (current-module))))
+ (((v . t) ...) lexical)
+ ((u ...) other))
+ #'(make-lexical-environment
+ (resolve-module 'module)
+ (lambda (exp)
+ (with-syntax ((exp (datum->syntax #'scope exp)))
+ #'(lambda (t ...)
+ (let*-syntax ((v (identifier-syntax-from-box t))
+ ...
+ (u (unsupported-binding 'u))
+ ...)
+ #f ; force expression context
+ exp))))
+ (list (make-box v) ...)))))))))
+
+(define (local-eval x e)
+ "Evaluate the expression @var{x} within the lexical environment @var{e}."
+ (cond ((lexical-environment? e)
+ (apply (eval ((lexenv-wrapper e) x) (lexenv-module e))
+ (lexenv-boxes e)))
+ ((module? e)
+ ;; Here we evaluate the expression within `lambda', and then
+ ;; call the resulting procedure outside of the dynamic extent
+ ;; of `eval'. We do this because `eval' sets (current-module)
+ ;; within its dynamic extent, and we don't want that. Also,
+ ;; doing it this way makes this a proper tail call.
+ ((eval #`(lambda () #,x) e)))
+ (else (error "local-eval: invalid lexical environment" e))))
+
+(define* (local-compile x e #:key (opts '()))
+ "Compile and evaluate the expression @var{x} within the lexical environment @var{e}."
+ (cond ((lexical-environment? e)
+ (apply (compile ((lexenv-wrapper e) x)
+ #:env (lexenv-module e)
+ #:from 'scheme #:opts opts)
+ (lexenv-boxes e)))
+ ((module? e)
+ ;; Here we compile the expression within `lambda', and then
+ ;; call the resulting procedure outside of the dynamic extent
+ ;; of `compile'. We do this because `compile' sets
+ ;; (current-module) during evaluation, and we don't want that.
+ ((compile #`(lambda () #,x)
+ #:env e #:from 'scheme #:opts opts)))
+ (else (error "local-compile: invalid lexical environment" e))))
diff --git a/test-suite/standalone/test-loose-ends.c b/test-suite/standalone/test-loose-ends.c
index 2fdbe7d..f815ae2 100644
--- a/test-suite/standalone/test-loose-ends.c
+++ b/test-suite/standalone/test-loose-ends.c
@@ -3,7 +3,7 @@
* Test items of the Guile C API that aren't covered by any other tests.
*/
-/* Copyright (C) 2009 Free Software Foundation, Inc.
+/* Copyright (C) 2009, 2012 Free Software Foundation, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public License
@@ -43,9 +43,23 @@ test_scm_from_locale_keywordn ()
}
static void
+test_scm_local_eval ()
+{
+ SCM result = scm_local_eval
+ (scm_list_3 (scm_from_latin1_symbol ("+"),
+ scm_from_latin1_symbol ("x"),
+ scm_from_latin1_symbol ("y")),
+ scm_c_eval_string ("(let ((x 1) (y 2)) (the-environment))"));
+
+ assert (scm_is_true (scm_equal_p (result,
+ scm_from_signed_integer (3))));
+}
+
+static void
tests (void *data, int argc, char **argv)
{
test_scm_from_locale_keywordn ();
+ test_scm_local_eval ();
}
int
diff --git a/test-suite/tests/eval.test b/test-suite/tests/eval.test
index a128cd7..73f9140 100644
--- a/test-suite/tests/eval.test
+++ b/test-suite/tests/eval.test
@@ -1,5 +1,5 @@
;;;; eval.test --- tests guile's evaluator -*- scheme -*-
-;;;; Copyright (C) 2000, 2001, 2006, 2007, 2009, 2010, 2011 Free Software Foundation, Inc.
+;;;; Copyright (C) 2000, 2001, 2006, 2007, 2009, 2010, 2011, 2012 Free Software Foundation, Inc.
;;;;
;;;; This library is free software; you can redistribute it and/or
;;;; modify it under the terms of the GNU Lesser General Public
@@ -19,7 +19,8 @@
:use-module (test-suite lib)
:use-module ((srfi srfi-1) :select (unfold count))
:use-module ((system vm vm) :select (make-vm call-with-vm))
- :use-module (ice-9 documentation))
+ :use-module (ice-9 documentation)
+ :use-module (ice-9 local-eval))
(define exception:bad-expression
@@ -422,4 +423,78 @@
(thunk (let loop () (cons 's (loop)))))
(call-with-vm vm thunk))))
+;;;
+;;; local-eval
+;;;
+
+(with-test-prefix "local evaluation"
+
+ (pass-if "local-eval"
+
+ (let* ((env1 (let ((x 1) (y 2) (z 3))
+ (define-syntax-rule (foo x) (quote x))
+ (the-environment)))
+ (env2 (local-eval '(let ((x 111) (a 'a))
+ (define-syntax-rule (bar x) (quote x))
+ (the-environment))
+ env1)))
+ (local-eval '(set! x 11) env1)
+ (local-eval '(set! y 22) env1)
+ (local-eval '(set! z 33) env2)
+ (and (equal? (local-eval '(list x y z) env1)
+ '(11 22 33))
+ (equal? (local-eval '(list x y z a) env2)
+ '(111 22 33 a)))))
+
+ (pass-if "local-compile"
+
+ (let* ((env1 (let ((x 1) (y 2) (z 3))
+ (define-syntax-rule (foo x) (quote x))
+ (the-environment)))
+ (env2 (local-compile '(let ((x 111) (a 'a))
+ (define-syntax-rule (bar x) (quote x))
+ (the-environment))
+ env1)))
+ (local-compile '(set! x 11) env1)
+ (local-compile '(set! y 22) env1)
+ (local-compile '(set! z 33) env2)
+ (and (equal? (local-compile '(list x y z) env1)
+ '(11 22 33))
+ (equal? (local-compile '(list x y z a) env2)
+ '(111 22 33 a)))))
+
+ (pass-if "the-environment within a macro"
+
+ (let ()
+ (define-syntax-rule (test)
+ (let ((x 1) (y 2))
+ (the-environment)))
+ (let ((env (let ((x 111) (y 222))
+ (test))))
+ (equal? (local-eval '(list x y) env)
+ '(1 2)))))
+
+ (pass-if "mixed primitive-eval, local-eval and local-compile"
+
+ (let* ((env1 (primitive-eval '(let ((x 1) (y 2) (z 3))
+ (define-syntax-rule (foo x) (quote x))
+ (the-environment))))
+ (env2 (local-eval '(let ((x 111) (a 'a))
+ (define-syntax-rule (bar x) (quote x))
+ (the-environment))
+ env1))
+ (env3 (local-compile '(let ((y 222) (b 'b))
+ (the-environment))
+ env2)))
+ (local-eval '(set! x 11) env1)
+ (local-compile '(set! y 22) env2)
+ (local-eval '(set! z 33) env2)
+ (local-compile '(set! a (* y 2)) env3)
+ (and (equal? (local-compile '(list x y z) env1)
+ '(11 22 33))
+ (equal? (local-eval '(list x y z a) env2)
+ '(111 22 33 444))
+ (equal? (local-eval '(list x y z a b) env3)
+ '(111 222 33 444 b))))))
+
;;; eval.test ends here
--
1.7.8.3
[-- Attachment #5: Type: text/plain, Size: 27 bytes --]
--
http://wingolog.org/
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: local-eval on syntax-local-binding, bound-identifiers
2012-01-15 22:17 local-eval on syntax-local-binding, bound-identifiers Andy Wingo
@ 2012-01-16 3:30 ` Mark H Weaver
2012-01-16 11:01 ` Andy Wingo
0 siblings, 1 reply; 9+ messages in thread
From: Mark H Weaver @ 2012-01-16 3:30 UTC (permalink / raw)
To: Andy Wingo; +Cc: guile-devel
Hi Andy,
Thanks very much for heeding my call for `local-eval' in 2.0.4, and
for putting so much time into this.
For the record, I still think it's better for `the-environment' to be
implemented within psyntax as a core form. It's a fundamental syntactic
construct with clean semantics, and it belongs in psyntax with its
brethren. Your desire to remove it from psyntax has caused you to add
far less elegant interfaces that have been hastily designed, and that
may not even be sufficient for a full implementation of
`the-environment' that captures mutually-recursive local macros.
That said, there's a lot to like in your implementation, and it has some
notable improvements over mine. It also has some problems, not all of
which are trivial.
Please see below, where I have inserted specific questions and comments
into your patch.
Thanks again,
Mark
[... skipped the first patch, which looks very well implemented, though
I'm still not sure that we should be exposing this in our API ...]
> From 2c3da44320019453115811af386febaa7eb241c3 Mon Sep 17 00:00:00 2001
> From: Andy Wingo <wingo@pobox.com>
> Date: Sun, 15 Jan 2012 18:39:44 +0100
> Subject: [PATCH 2/3] add bound-identifiers
>
> * module/ice-9/boot-9.scm (bound-identifiers): Declare variable.
> * module/ice-9/psyntax.scm: Add all-bound-identifiers helper, and define
> bound-identifiers. The identifiers are anti-marked so that syntax
> transformers can introduce them, as-is.
> ---
> module/ice-9/boot-9.scm | 1 +
> module/ice-9/psyntax.scm | 49 ++++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 50 insertions(+), 0 deletions(-)
>
> diff --git a/module/ice-9/boot-9.scm b/module/ice-9/boot-9.scm
> index 9cdd8d1..b8aa842 100644
> --- a/module/ice-9/boot-9.scm
> +++ b/module/ice-9/boot-9.scm
> @@ -389,6 +389,7 @@ If there is no handler at all, Guile prints an error and then exits."
> (define generate-temporaries #f)
> (define bound-identifier=? #f)
> (define free-identifier=? #f)
> +(define bound-identifiers #f)
> (define syntax-local-binding #f)
>
> ;; $sc-dispatch is an implementation detail of psyntax. It is used by
> diff --git a/module/ice-9/psyntax.scm b/module/ice-9/psyntax.scm
> index 30685bc..25543e0 100644
> --- a/module/ice-9/psyntax.scm
> +++ b/module/ice-9/psyntax.scm
> @@ -786,6 +786,48 @@
> id))))))
> (else (syntax-violation 'id-var-name "invalid id" id)))))
>
> + ;;
> + ;; all-bound-identifiers returns a list of all lexically bound
> + ;; identifiers, as syntax objects. They are in order from outer to
> + ;; inner.
> + ;;
> + (define all-bound-identifiers
> + (lambda (w mod)
> + (define scan
> + (lambda (subst results)
> + (if (null? subst)
> + results
> + (let ((fst (car subst)))
> + (if (eq? fst 'shift)
> + (scan (cdr subst) results)
> + (let ((symnames (ribcage-symnames fst))
> + (marks (ribcage-marks fst)))
> + (if (vector? symnames)
> + (scan-vector-rib subst symnames marks results)
> + (scan-list-rib subst symnames marks results))))))))
> + (define scan-list-rib
> + (lambda (subst symnames marks results)
> + (let f ((symnames symnames) (marks marks) (results results))
> + (if (null? symnames)
> + (scan (cdr subst) results)
> + (f (cdr symnames) (cdr marks)
> + (cons (wrap (car symnames)
> + (anti-mark (make-wrap (car marks) subst))
***** Why are you adding anti-marks here?
> + mod)
> + results))))))
> + (define scan-vector-rib
> + (lambda (subst symnames marks results)
> + (let ((n (vector-length symnames)))
> + (let f ((i 0) (results results))
> + (if (fx= i n)
> + (scan (cdr subst) results)
> + (f (fx+ i 1)
> + (cons (wrap (vector-ref symnames i)
> + (anti-mark (make-wrap (vector-ref marks i) subst))
***** Ditto.
> + mod)
> + results)))))))
> + (scan (wrap-subst w) '())))
> +
> (define transformer-environment
> (make-fluid
> (lambda (k)
> @@ -2470,6 +2512,13 @@
> (else (error "unpossible!" b)))
> (values 'displaced-lexical #f))))))))
>
> + (set! bound-identifiers
> + (lambda (x)
> + (arg-check nonsymbol-id? x 'bound-identifiers)
> + (reverse
> + (all-bound-identifiers (syntax-object-wrap x)
> + (syntax-object-module x)))))
> +
> (set! generate-temporaries
> (lambda (ls)
> (arg-check list? ls 'generate-temporaries)
> --
> 1.7.8.3
>
>
>
> From ddea51310227155e3771c3e6acbbecf24dc74c42 Mon Sep 17 00:00:00 2001
> From: Mark H Weaver <mhw@netris.org>
> Date: Tue, 3 Jan 2012 04:02:08 -0500
> Subject: [PATCH 3/3] Implement `local-eval', `local-compile', and
> `the-environment'
***** In general, when you make substantial changes to one of my
patches, please remove my name as the principal author. Instead, please
say something like "Based on a patch by Mark H Weaver <mhw@netris.org>"
in the log entry.
>
> * module/ice-9/local-eval.scm: New module (ice-9 local-eval) which
> exports `the-environment', `local-eval', and `local-compile'.
>
> * libguile/debug.c (scm_local_eval): New C function that calls the
> Scheme implementation of `local-eval' in (ice-9 local-eval).
>
> * libguile/debug.h (scm_local_eval): Add prototype.
>
> * doc/ref/api-evaluation.texi (Local Evaluation): Add documentation.
>
> * test-suite/tests/eval.test (local evaluation): Add tests.
>
> * test-suite/standalone/test-loose-ends.c (test_scm_local_eval):
> Add test.
>
> * module/Makefile.am: Add ice-9/local-eval.scm.
> ---
> doc/ref/api-evaluation.texi | 38 ++++++++
> libguile/debug.c | 13 +++-
> libguile/debug.h | 4 +-
> module/Makefile.am | 5 +-
> module/ice-9/local-eval.scm | 150 +++++++++++++++++++++++++++++++
> test-suite/standalone/test-loose-ends.c | 16 +++-
> test-suite/tests/eval.test | 79 ++++++++++++++++-
> 7 files changed, 298 insertions(+), 7 deletions(-)
> create mode 100644 module/ice-9/local-eval.scm
>
> diff --git a/doc/ref/api-evaluation.texi b/doc/ref/api-evaluation.texi
> index 2e48dcb..72dd4df 100644
> --- a/doc/ref/api-evaluation.texi
> +++ b/doc/ref/api-evaluation.texi
> @@ -19,6 +19,7 @@ loading, evaluating, and compiling Scheme code at run time.
> * Loading:: Loading Scheme code from file.
> * Character Encoding of Source Files:: Loading non-ASCII Scheme code from file.
> * Delayed Evaluation:: Postponing evaluation until it is needed.
> +* Local Evaluation:: Evaluation in a local lexical environment.
> @end menu
>
>
> @@ -954,6 +955,43 @@ value.
> @end deffn
>
>
> +@node Local Evaluation
> +@subsection Local Evaluation
> +
> +@deffn syntax the-environment
> +Captures and returns a lexical environment for use with
> +@code{local-eval} or @code{local-compile}.
> +@end deffn
> +
> +@deffn {Scheme Procedure} local-eval exp env
> +@deffnx {C Function} scm_local_eval (exp, env)
> +Evaluate the expression @var{exp} in the lexical environment @var{env}.
> +This mostly behaves as if @var{exp} had been wrapped in a lambda
> +expression @code{`(lambda () ,@var{exp})} and put in place of
> +@code{(the-environment)}, with the resulting procedure called by
> +@code{local-eval}. In other words, @var{exp} is evaluated within the
> +lexical environment of @code{(the-environment)}, but within the dynamic
> +environment of the call to @code{local-eval}.
> +@end deffn
> +
> +@deffn {Scheme Procedure} local-compile exp env [opts=()]
> +Compile the expression @var{exp} in the lexical environment @var{env}.
> +If @var{exp} is a procedure, the result will be a compiled procedure;
> +otherwise @code{local-compile} is mostly equivalent to
> +@code{local-eval}. @var{opts} specifies the compilation options.
> +@end deffn
> +
> +Note that the current implementation of @code{(the-environment)} has
> +some limitations. It does not capture local syntax transformers bound
> +by @code{let-syntax}, @code{letrec-syntax} or non-top-level
> +@code{define-syntax} forms. Any attempt to reference such captured
> +syntactic keywords via @code{local-eval} or @code{local-compile}
> +produces an error. Also, @code{(the-environment)} does not capture
> +lexical bindings that are shadowed by inner bindings with the same name,
> +nor hidden lexical bindings produced by macro expansion, even though
> +such bindings might be accessible using syntax objects.
***** I guess this last sentence should now be removed. Also, remember
that if pattern variables aren't reimplemented, this limitation needs to
be documented.
> +
> +
> @c Local Variables:
> @c TeX-master: "guile.texi"
> @c End:
[... skipped some files here ...]
> diff --git a/module/ice-9/local-eval.scm b/module/ice-9/local-eval.scm
> new file mode 100644
> index 0000000..cb74881
> --- /dev/null
> +++ b/module/ice-9/local-eval.scm
> @@ -0,0 +1,150 @@
> +;;; -*- mode: scheme; coding: utf-8; -*-
> +;;;
> +;;; Copyright (C) 2012 Free Software Foundation, Inc.
> +;;;
> +;;; This library is free software; you can redistribute it and/or
> +;;; modify it under the terms of the GNU Lesser General Public
> +;;; License as published by the Free Software Foundation; either
> +;;; version 3 of the License, or (at your option) any later version.
> +;;;
> +;;; This library is distributed in the hope that it will be useful,
> +;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> +;;; Lesser General Public License for more details.
> +;;;
> +;;; You should have received a copy of the GNU Lesser General Public
> +;;; License along with this library; if not, write to the Free Software
> +;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> +
> +(define-module (ice-9 local-eval)
> + #:use-module (ice-9 format)
> + #:use-module (srfi srfi-9)
> + #:use-module (srfi srfi-9 gnu)
> + #:use-module (system base compile)
> + #:export (the-environment local-eval local-compile))
> +
> +(define-record-type lexical-environment-type
> + (make-lexical-environment module wrapper boxes)
> + lexical-environment?
> + (module lexenv-module)
> + (wrapper lexenv-wrapper)
> + (boxes lexenv-boxes))
> +
> +(set-record-type-printer!
> + lexical-environment-type
> + (lambda (e port)
> + (format port "#<lexical-environment ~S (~S bindings)>"
> + (module-name (lexenv-module e)) (length (lexenv-boxes e)))))
> +
> +(define-syntax-rule (make-box v)
> + (case-lambda
> + (() v)
> + ((x) (set! v x))))
> +
> +(define-syntax let*-syntax
> + (syntax-rules ()
> + ((_ () e e* ...)
> + (begin e e* ...))
> + ((_ ((id trans) (id* trans*) ...) e e* ...)
> + (let-syntax ((id trans))
> + (let*-syntax ((id* trans*) ...)
> + e e* ...)))))
> +
> +(define-syntax-rule (identifier-syntax-from-box box)
> + (make-transformer-from-box
> + (syntax-object-of box)
> + (identifier-syntax (id (box))
> + ((set! id x) (box x)))))
> +
> +(define-syntax syntax-object-of
> + (lambda (form)
> + (syntax-case form ()
> + ((_ x) #`(quote #,(datum->syntax #'x #'x))))))
> +
> +(define (make-transformer-from-box id trans)
> + (set-procedure-property! trans 'identifier-syntax-box id)
> + trans)
> +
> +(define (unsupported-binding name)
> + (make-variable-transformer
> + (lambda (x)
> + (syntax-violation
> + 'local-eval
> + "unsupported binding captured by (the-environment)"
> + x))))
> +
> +(define (partition-identifiers ids)
> + (let lp ((ids ids) (lexical '()) (other '()))
> + (if (null? ids)
> + (values lexical other)
> + (call-with-values (lambda () (syntax-local-binding (car ids)))
> + (lambda (type val)
> + (cond
> + ((eq? type 'lexical)
> + (lp (cdr ids)
> + (acons (car ids) (datum->syntax #'here (gensym))
> + lexical)
> + other))
> + ((and (eq? type 'local-macro)
> + (procedure-property val 'identifier-syntax-box))
> + => (lambda (id)
> + (lp (cdr ids)
> + (acons (car ids) id lexical)
***** You are putting inconsistent things in the cdr of each alist
entry. In the `lexical' case above, you make a gensym, and here you're
putting the id of the box variable. However, in the code below, you are
not using this box, but rather just using these identifiers for
temporaries.
> + other)))
> + (else
> + (lp (cdr ids) lexical (cons (car ids) other)))))))))
> +
> +(define-syntax the-environment
> + (lambda (x)
> + (syntax-case x ()
> + ((the-environment)
> + #'(the-environment the-environment))
> + ((the-environment scope)
***** This is nice.
> + (call-with-values (lambda ()
> + (partition-identifiers (bound-identifiers #'scope)))
> + (lambda (lexical other)
> + (with-syntax ((module (datum->syntax #'here (module-name (current-module))))
***** This is wrong. You want the module from the `scope'
syntax-object, not the (current-module).
> + (((v . t) ...) lexical)
> + ((u ...) other))
> + #'(make-lexical-environment
> + (resolve-module 'module)
> + (lambda (exp)
> + (with-syntax ((exp (datum->syntax #'scope exp)))
> + #'(lambda (t ...)
> + (let*-syntax ((v (identifier-syntax-from-box t))
> + ...
> + (u (unsupported-binding 'u))
> + ...)
> + #f ; force expression context
> + exp))))
> + (list (make-box v) ...)))))))))
***** Here you are always making new boxes, even for previously-captured
variables that have already been boxed. Therefore boxes become nested
and cost O(n) to access, where N is the nesting depth.
More importantly: I notice that you are not stripping the psyntax wrap
from identifiers placed within the wrapper procedure above. There are
certainly benefits to that, but remember that the wrapper procedure will
in general be serialized to disk and evaluated in a different Guile
session, where the gensym counters have been reset.
This raises a couple of issues that I intentionally sidestepped by
stripping the wraps:
First of all, whereas previously gensyms and psyntax labels only needed
to be unique within a single session, now they must be universally
unique. This needs to be fixed in order to make your patch robust.
Secondly, you are now serializing a psyntax internal data structure to
disk (the wraps), and thus this now effectively becomes part of the ABI.
This is not necessarily a bad thing, but we must consider what will
happen when a new version of Guile with an enhanced psyntax calls
`local-eval' on a lexical environment generated with code compiled by an
older version of Guile. We might need to add some kind of versioning
scheme. It's a bit of a thorny issue, which was much of my motivation
for stripping the wraps in this 2.0.4 version of `local-eval' (though I
admit that a future implementation that supports local syntax
transformers won't be able to avoid this issue).
> +
> +(define (local-eval x e)
> + "Evaluate the expression @var{x} within the lexical environment @var{e}."
> + (cond ((lexical-environment? e)
> + (apply (eval ((lexenv-wrapper e) x) (lexenv-module e))
> + (lexenv-boxes e)))
> + ((module? e)
> + ;; Here we evaluate the expression within `lambda', and then
> + ;; call the resulting procedure outside of the dynamic extent
> + ;; of `eval'. We do this because `eval' sets (current-module)
> + ;; within its dynamic extent, and we don't want that. Also,
> + ;; doing it this way makes this a proper tail call.
> + ((eval #`(lambda () #,x) e)))
***** This was my mistake, but since I'm already marking up the code:
the `lambda' wrap above needs a `#f' before `e' to force expression
context.
> + (else (error "local-eval: invalid lexical environment" e))))
> +
> +(define* (local-compile x e #:key (opts '()))
> + "Compile and evaluate the expression @var{x} within the lexical environment @var{e}."
> + (cond ((lexical-environment? e)
> + (apply (compile ((lexenv-wrapper e) x)
> + #:env (lexenv-module e)
> + #:from 'scheme #:opts opts)
> + (lexenv-boxes e)))
> + ((module? e)
> + ;; Here we compile the expression within `lambda', and then
> + ;; call the resulting procedure outside of the dynamic extent
> + ;; of `compile'. We do this because `compile' sets
> + ;; (current-module) during evaluation, and we don't want that.
> + ((compile #`(lambda () #,x)
***** Ditto.
> + #:env e #:from 'scheme #:opts opts)))
> + (else (error "local-compile: invalid lexical environment" e))))
[... skipped the rest ...]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: local-eval on syntax-local-binding, bound-identifiers
2012-01-16 3:30 ` Mark H Weaver
@ 2012-01-16 11:01 ` Andy Wingo
2012-01-16 13:28 ` Mark H Weaver
0 siblings, 1 reply; 9+ messages in thread
From: Andy Wingo @ 2012-01-16 11:01 UTC (permalink / raw)
To: Mark H Weaver; +Cc: guile-devel
Hi Mark,
Thanks for the review! My summary comments are at the bottom.
On Mon 16 Jan 2012 04:30, Mark H Weaver <mhw@netris.org> writes:
> [... skipped the first patch, which looks very well implemented, though
> I'm still not sure that we should be exposing this in our API ...]
Yeah, not sure what to do with it. There is also the question of what
to do with syntax parameters; something that's easier to ask (and
answer) on master, though.
>> + (cons (wrap (car symnames)
>> + (anti-mark (make-wrap (car marks) subst))
>
> ***** Why are you adding anti-marks here?
As the changelog noted (and a comment should have noted ;), the
identifiers are anti-marked so that syntax transformers can introduce
them, as-is.
The purpose of this procedure is to get a list of identifiers, and to
capture some subset of them. It will do so by introducing references to
them in the expansion of some macro. However they are not introduced
identifiers: they come from the code itself. They are input the macro,
and as such need an anti-mark.
The anti-mark will be stripped from the expansion when the transformer
that called `bound-identifiers' returns.
As the idea of marks and anti-marks is part of the definition of
syntax-case, on a semantic rather than an implementation level, this
aspect of `bound-identifiers' probably needs to be documented.
>> From ddea51310227155e3771c3e6acbbecf24dc74c42 Mon Sep 17 00:00:00 2001
>> From: Mark H Weaver <mhw@netris.org>
>> Date: Tue, 3 Jan 2012 04:02:08 -0500
>> Subject: [PATCH 3/3] Implement `local-eval', `local-compile', and
>> `the-environment'
>
> ***** In general, when you make substantial changes to one of my
> patches, please remove my name as the principal author. Instead, please
> say something like "Based on a patch by Mark H Weaver <mhw@netris.org>"
> in the log entry.
The majority of the lines are yours; though it is difficult to assign
ultimate authorship here, to my mind it was quite fair to put you as the
primary author. But, as you like.
>> +Also, @code{(the-environment)} does not capture +lexical bindings
>> that are shadowed by inner bindings with the same name, +nor hidden
>> lexical bindings produced by macro expansion, even though +such
>> bindings might be accessible using syntax objects.
>
> ***** I guess this last sentence should now be removed. Also, remember
> that if pattern variables aren't reimplemented, this limitation needs to
> be documented.
Yes.
>> +(define (partition-identifiers ids)
>> + (let lp ((ids ids) (lexical '()) (other '()))
>> + (if (null? ids)
>> + (values lexical other)
>> + (call-with-values (lambda () (syntax-local-binding (car ids)))
>> + (lambda (type val)
>> + (cond
>> + ((eq? type 'lexical)
>> + (lp (cdr ids)
>> + (acons (car ids) (datum->syntax #'here (gensym))
>> + lexical)
>> + other))
>> + ((and (eq? type 'local-macro)
>> + (procedure-property val 'identifier-syntax-box))
>> + => (lambda (id)
>> + (lp (cdr ids)
>> + (acons (car ids) id lexical)
>
> ***** You are putting inconsistent things in the cdr of each alist
> entry. In the `lexical' case above, you make a gensym, and here you're
> putting the id of the box variable. However, in the code below, you are
> not using this box, but rather just using these identifiers for
> temporaries.
Ah, indeed. I did say it was very preliminary, no? :-)
>> + (lambda (lexical other)
>> + (with-syntax ((module (datum->syntax #'here (module-name (current-module))))
>
> ***** This is wrong. You want the module from the `scope'
> syntax-object, not the (current-module).
Yes, you are right. However, here I think we need another primitive...
> ***** Here you are always making new boxes, even for previously-captured
> variables that have already been boxed. Therefore boxes become nested
> and cost O(n) to access, where N is the nesting depth.
Good catch, that was certainly not the intention.
> More importantly: I notice that you are not stripping the psyntax wrap
> from identifiers placed within the wrapper procedure above. There are
> certainly benefits to that, but remember that the wrapper procedure will
> in general be serialized to disk and evaluated in a different Guile
> session, where the gensym counters have been reset.
Of course, like all macros! The forgeable gensym issue is something we
have in Guile, more generally, that needs a broader solution.
> Secondly, you are now serializing a psyntax internal data structure to
> disk (the wraps), and thus this now effectively becomes part of the ABI.
This is already the case.
>> + ((module? e)
>> + ;; Here we evaluate the expression within `lambda', and then
>> + ;; call the resulting procedure outside of the dynamic extent
>> + ;; of `eval'. We do this because `eval' sets (current-module)
>> + ;; within its dynamic extent, and we don't want that. Also,
>> + ;; doing it this way makes this a proper tail call.
>> + ((eval #`(lambda () #,x) e)))
>
> ***** This was my mistake, but since I'm already marking up the code:
> the `lambda' wrap above needs a `#f' before `e' to force expression
> context.
OK. (Note though that (eval X e) does indeed evaluate X in tail
position.)
> For the record, I still think it's better for `the-environment' to be
> implemented within psyntax as a core form. It's a fundamental syntactic
> construct with clean semantics, and it belongs in psyntax with its
> brethren. Your desire to remove it from psyntax has caused you to add
> far less elegant interfaces that have been hastily designed, and that
> may not even be sufficient for a full implementation of
> `the-environment' that captures mutually-recursive local macros.
In pursuit of the goal of agreeing on a strategy, I would like to
convince you that you are wrong on all of these points :) So, in that
spirit, I argue:
`the-environment' is not fundamental: it can be implemented in terms of
simpler primitives.
`the-environment' does not have clean semantics, inasmuch as it has
nothing worthy of the name, not yet anyway. The lambda calculus, the
scheme language, even the syntax-case system have well-studied semantics
(denotational and/or operational), and lots of experience. The same
cannot be said of `the-environment'. It is great that we are pushing
the boundaries of Scheme here, but I think it's hubris to think that
we'll get it right the first time.
Implementing the-environment in psyntax also poses implementation
layering problems: it uses wrap hacks to expand out to private forms
from ice-9 local-eval. Instead, this implementation goes with the grain
of the expander, allowing the-environment to be implemented as a macro,
with the normal scoping rules for syntax objects.
It also seems to me that stripping identifiers down to their symbolic
values is the wrong thing to do. It has a bad smell.
I readily grant that the interfaces I posted were "hastily designed",
though there is some prior art in Racket (nb, a different implementation
that's not psyntax). However, if they do prove to have maintainable
semantics, I really do prefer to maintain that (in the syntax expander)
rather than `the-environment'. Feedback here is much appreciated on the
form of these interfaces.
I think it's possible that this approach could work for mutually
recursive macros. The key to note here is that the essence of the
mutual recursion is in the expansion of the syntax transformers (the
rhs), and that was already done within a proper recursive environment.
Later when the expansion produces, or would produce:
(let*-syntax ((x proc-recursive-with-y)
(y proc-recursive-with-x))
(x ..))
the transformer procedures are already expanded (and evaluated). So the
thing is that the bindings from x and y to their transformers must be
visible to the expression being local-eval'd, not to the transformer
procedures themselves.
Have I convinced you now? :-)
Regards,
Andy
--
http://wingolog.org/
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: local-eval on syntax-local-binding, bound-identifiers
2012-01-16 11:01 ` Andy Wingo
@ 2012-01-16 13:28 ` Mark H Weaver
2012-01-16 15:17 ` Andy Wingo
0 siblings, 1 reply; 9+ messages in thread
From: Mark H Weaver @ 2012-01-16 13:28 UTC (permalink / raw)
To: Andy Wingo; +Cc: guile-devel
Hi Andy!
Andy Wingo <wingo@pobox.com> writes:
>>> + (cons (wrap (car symnames)
>>> + (anti-mark (make-wrap (car marks) subst))
>>
>> ***** Why are you adding anti-marks here?
>
> As the changelog noted (and a comment should have noted ;), the
> identifiers are anti-marked so that syntax transformers can introduce
> them, as-is.
>
> The purpose of this procedure is to get a list of identifiers, and to
> capture some subset of them. It will do so by introducing references to
> them in the expansion of some macro. However they are not introduced
> identifiers: they come from the code itself. They are input the macro,
> and as such need an anti-mark.
>
> The anti-mark will be stripped from the expansion when the transformer
> that called `bound-identifiers' returns.
Does this mean that `bound-identifiers' will not function properly when
used outside of a macro? What about if it's used within a macro that
was generated by another macro (or things of that nature)? Are there
cases where you might need to strip more than one anti-mark?
To use your phrase, this has a bad smell.
>> More importantly: I notice that you are not stripping the psyntax wrap
>> from identifiers placed within the wrapper procedure above. There are
>> certainly benefits to that, but remember that the wrapper procedure will
>> in general be serialized to disk and evaluated in a different Guile
>> session, where the gensym counters have been reset.
>
> Of course, like all macros! The forgeable gensym issue is something we
> have in Guile, more generally, that needs a broader solution.
Ah, good point! Macros already serialize syntax-objects to disk.
psyntax wraps are already part of our ABI, so nothing new there.
However, I fear that the gensym issue might be a serious problem for
`local-eval', even though it hasn't been a problem for macros.
The reason it has not been a problem with macros is that, within a
top-level macro (which are the only ones used across Guile sessions),
the only syntax-objects that can be meaningfully _introduced_ into the
expansion are top-level/module bindings. But these bindings have no
associated labels or gensyms, because they're not in the wrap.
On the other hand, with `local-eval', it seems to me quite plausible
that gensym collisions might occur. Suppose in one Guile session you
compile a procedure (foo) that uses (the-environment), and then in
another Guile session, you call (foo) and then `local-eval' with the
environment returned by (foo). Now the wrapper procedure splices
together syntax objects from two different Guile sessions into a single
top-level form, where (unlike in the macro case) all of these syntax
objects are lexicals, and thus depend on the gensyms and the labels.
See how this is a problem now where it wasn't before?
Or am I missing something?
>>> + ((module? e)
>>> + ;; Here we evaluate the expression within `lambda', and then
>>> + ;; call the resulting procedure outside of the dynamic extent
>>> + ;; of `eval'. We do this because `eval' sets (current-module)
>>> + ;; within its dynamic extent, and we don't want that. Also,
>>> + ;; doing it this way makes this a proper tail call.
>>> + ((eval #`(lambda () #,x) e)))
>>
>> ***** This was my mistake, but since I'm already marking up the code:
>> the `lambda' wrap above needs a `#f' before `e' to force expression
>> context.
>
> OK. (Note though that (eval X e) does indeed evaluate X in tail
> position.)
Looks to me like `eval' is initially bound to the C function scm_eval.
Is it later rebound to a Scheme procedure? If so, where?
>> For the record, I still think it's better for `the-environment' to be
>> implemented within psyntax as a core form. It's a fundamental syntactic
>> construct with clean semantics, and it belongs in psyntax with its
>> brethren. Your desire to remove it from psyntax has caused you to add
>> far less elegant interfaces that have been hastily designed, and that
>> may not even be sufficient for a full implementation of
>> `the-environment' that captures mutually-recursive local macros.
>
> In pursuit of the goal of agreeing on a strategy, I would like to
> convince you that you are wrong on all of these points :) So, in that
> spirit, I argue:
Very well, I will endeavor to be open-minded.
> `the-environment' is not fundamental: it can be implemented in terms of
> simpler primitives.
The same can be said of `lambda' or `syntax-case', but that's not the
appropriate way to choose primitives in a language. The set of
primitives chosen in Scheme are not the ones that are simplest to
implement. It makes more sense to choose primitives with simple, clean
semantics, which segues nicely into your next paragraph.
> `the-environment' does not have clean semantics, inasmuch as it has
> nothing worthy of the name, not yet anyway. The lambda calculus, the
> scheme language, even the syntax-case system have well-studied semantics
> (denotational and/or operational), and lots of experience. The same
> cannot be said of `the-environment'. It is great that we are pushing
> the boundaries of Scheme here, but I think it's hubris to think that
> we'll get it right the first time.
I think you read more into my words "clean semantics" than I intended.
Of course I never meant to imply that its semantics are as well-studied
as the rest of standard Scheme. I only meant to express my own personal
judgement that the semantics of `the-environment' and `local-eval' seem
quite clean.
BTW, did you see my most recent model for thinking about `local-eval'?
(the-environment) expands to (list (lambda () <expr>) ...), with one
element for every possible expression: a countably infinite list that
could be built lazily. `local-eval' simply chooses the appropriate
procedure from the list and calls it. A poor implementation strategy,
but the semantic meaning is quite clear, no?
Regardless, I hope it's safe to say that `the-environment' and
`local-eval' are far closer to the spirit of primitives one finds in the
Scheme standards than are the ones you're proposing. Do you not agree?
> Implementing the-environment in psyntax also poses implementation
> layering problems: it uses wrap hacks to expand out to private forms
> from ice-9 local-eval.
I don't see how that's a "hack" at all. That's exactly what wraps are
for: to specify the expander's lexical environment in which to look up
the symbol. I'm using them in a completely straightforward way to
create syntax-objects for identifiers in ice-9/local-eval. This within
psyntax, where wraps are created all over the place.
> It also seems to me that stripping identifiers down to their symbolic
> values is the wrong thing to do. It has a bad smell.
I agree 100%. Keeping the entire syntax-objects is "the right thing",
and indeed I think we'll need this in order to capture local macros.
However, making this work properly requires universally unique gensyms.
I decided to avoid this complication for now, because it seems to me
that it's highly unlikely to matter in practice until we can capture
local macros. However, if you'd like to fix the gensym problem for
2.0.4, that's even better.
> I think it's possible that this approach could work for mutually
> recursive macros. The key to note here is that the essence of the
> mutual recursion is in the expansion of the syntax transformers (the
> rhs), and that was already done within a proper recursive environment.
That only works if we can reuse the transformer procedures that were
originally captured by (the-environment). And indeed, this is the only
way to capture _procedural_ macros properly. Therefore, I have no hope
that such macros could be serialized to disk.
`syntax-rules' macros are a different story. A `syntax-rules' macro can
be _perfectly_ represented as a syntax-object of the `syntax-rules' form
itself. Therefore, I would like to represent them this way in the
expansion of (the-environment), so that `syntax-rules' macros _can_ be
serialized to disk.
However, this requires re-expanding the bodies of the `syntax-rules'
forms, so now the environment structures _do_ matter. Ultimately, this
means that a set of mutually recursive local macros must be recreated in
`local-eval' as a `letrec-syntax'.
Does that make sense?
Thanks,
Mark
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: local-eval on syntax-local-binding, bound-identifiers
2012-01-16 13:28 ` Mark H Weaver
@ 2012-01-16 15:17 ` Andy Wingo
2012-01-16 20:36 ` Mark H Weaver
0 siblings, 1 reply; 9+ messages in thread
From: Andy Wingo @ 2012-01-16 15:17 UTC (permalink / raw)
To: Mark H Weaver; +Cc: guile-devel
Hi Mark,
Thanks for the comments :)
On Mon 16 Jan 2012 14:28, Mark H Weaver <mhw@netris.org> writes:
> Andy Wingo <wingo@pobox.com> writes:
>>>> + (cons (wrap (car symnames)
>>>> + (anti-mark (make-wrap (car marks) subst))
>>>
>>> ***** Why are you adding anti-marks here?
>>
>> As the changelog noted (and a comment should have noted ;), the
>> identifiers are anti-marked so that syntax transformers can introduce
>> them, as-is.
>>
>> The purpose of this procedure is to get a list of identifiers, and to
>> capture some subset of them. It will do so by introducing references to
>> them in the expansion of some macro. However they are not introduced
>> identifiers: they come from the code itself. They are input the macro,
>> and as such need an anti-mark.
>>
>> The anti-mark will be stripped from the expansion when the transformer
>> that called `bound-identifiers' returns.
>
> Does this mean that `bound-identifiers' will not function properly when
> used outside of a macro? What about if it's used within a macro that
> was generated by another macro (or things of that nature)? Are there
> cases where you might need to strip more than one anti-mark?
Well, bound-identifiers is a procedure, so if you are using it outside
the dynamic extent of a transformer procedure, that means that you have
a syntax object that you squirreled away from somewhere, so already
we're in somewhat uncharted territory. Once you introduce it somewhere
else (in another macro), the anti-mark would be stripped though. The
only other interface to the expander is `macroexpand', where we should
probably also strip the anti-mark, if we are allowing anti-marks to
escape.
Macro-generating macros should be fine, here. `expand-macro' is
iterative, not recursive, so you don't need to strip anti-marks twice.
I agree that this anti-mark has a bad smell, but the idea of a
`bound-identifiers' procedure or form sounds like a good idea, so if you
have any suggestions for improvement here, they are most welcome.
>> The forgeable gensym issue is something we have in Guile, more
>> generally, that needs a broader solution.
>
> Ah, good point! Macros already serialize syntax-objects to disk.
> psyntax wraps are already part of our ABI, so nothing new there.
The issue is not in the uniqueness of gensyms associated with lexical
variables, by the way: it's more in the uniqueness of the labels and the
marks. See psyntax-pp.scm, for example, or commit
fd5985271fee3bcb6a290b6ad10525980a97ef8d.
> [W]ith `local-eval', it seems to me quite plausible that gensym
> collisions might occur. Suppose in one Guile session you compile a
> procedure (foo) that uses (the-environment), and then in another Guile
> session, you call (foo) and then `local-eval' with the environment
> returned by (foo). Now the wrapper procedure splices together syntax
> objects from two different Guile sessions into a single top-level
> form, where (unlike in the macro case) all of these syntax objects are
> lexicals, and thus depend on the gensyms and the labels.
>
> See how this is a problem now where it wasn't before?
> Or am I missing something?
To be perfectly honest, this stuff is very confusing to me, but I think
I can see how this can happen, yes.
I do think that it's important to fix this bug at some point, but IMO it
is not a blocker for local-eval, much less 2.0.4.
>> (Note though that (eval X e) does indeed evaluate X in tail
>> position.)
>
> Looks to me like `eval' is initially bound to the C function scm_eval.
> Is it later rebound to a Scheme procedure? If so, where?
Ah, you're right, I was thinking of primitive-eval. Our scm_eval is
buggy wrt the spec (r5rs at least, which says):
Certain built-in procedures are also required to perform tail calls.
The first argument passed to `apply' and to
`call-with-current-continuation', and the second argument passed to
`call-with-values', must be called via a tail call. Similarly, `eval'
must evaluate its argument as if it were in tail position within the
`eval' procedure.
So we need to override eval to a Scheme procedure.
>> `the-environment' is not fundamental: it can be implemented in terms of
>> simpler primitives.
>
> The same can be said of `lambda' or `syntax-case', but that's not the
> appropriate way to choose primitives in a language.
John Shutt would argue with you here, but I digress :)
(http://web.cs.wpi.edu/~jshutt/kernel.html)
> BTW, did you see my most recent model for thinking about `local-eval'?
>
> (the-environment) expands to (list (lambda () <expr>) ...), with one
> element for every possible expression: a countably infinite list that
> could be built lazily. `local-eval' simply chooses the appropriate
> procedure from the list and calls it. A poor implementation strategy,
> but the semantic meaning is quite clear, no?
It sounds clear, but does it have any explanatory power? It sounds like
it could apply just as well to any other computation...
> Regardless, I hope it's safe to say that `the-environment' and
> `local-eval' are far closer to the spirit of primitives one finds in the
> Scheme standards than are the ones you're proposing. Do you not
> agree?
Yes, I grant you that the meanings of the-environment and local-eval
seem more clear than those of their constituent parts (boxes, identifier
syntax, etc). I'm certainly not arguing that lilypond should be using
bound-identifiers somehow! But I'm not sure that this says something
about what psyntax should have in it (especially if it is to have
syntax-local-value or something like it anyway).
>> Implementing the-environment in psyntax also poses implementation
>> layering problems: it uses wrap hacks to expand out to private forms
>> from ice-9 local-eval.
>
> I don't see how that's a "hack" at all. That's exactly what wraps are
> for: to specify the expander's lexical environment in which to look up
> the symbol. I'm using them in a completely straightforward way to
> create syntax-objects for identifiers in ice-9/local-eval. This within
> psyntax, where wraps are created all over the place.
Creating wraps is not the hack. It's creating wraps that are scoped in
another specific module. With the-environment in psyntax, psyntax
depends on (ice-9 local-eval). With the-environment in (ice-9
local-eval), psyntax does not depend on (ice-9 local-eval).
I *very* much prefer a psyntax that does not depend on other modules.
> A `syntax-rules' macro can be _perfectly_ represented as a
> syntax-object of the `syntax-rules' form itself. Therefore, I would
> like to represent them this way in the expansion of (the-environment),
> so that `syntax-rules' macros _can_ be serialized to disk.
All this for local-eval? And you don't get procedural macros? And what
would it do to psyntax? But OK. I still think you should be able to do
this in a module!
Ultimately, Guile should be a great platform for language research and
experimentation. Like Racket is, in many ways. In order to support
their multitudinous syntactic rackets, Racket gives the macro writer
lots of power -- lots of accessors, lots of insight into what the
expander is up to. We should do the same, where it makes sense to allow
access to that information. We shouldn't restrict these sorts of
experiments to the kinds of (grumpy, crotchety) people that maintain
psyntax.scm ;-)
> However, this requires re-expanding the bodies of the `syntax-rules'
> forms, so now the environment structures _do_ matter. Ultimately, this
> means that a set of mutually recursive local macros must be recreated in
> `local-eval' as a `letrec-syntax'.
>
> Does that make sense?
Yes I see where you're going here. IMO, we should be providing the
tools to let you do these experiments in parallel, without delaying the
release of other bugfixes.
Regards,
Andy
--
http://wingolog.org/
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: local-eval on syntax-local-binding, bound-identifiers
2012-01-16 15:17 ` Andy Wingo
@ 2012-01-16 20:36 ` Mark H Weaver
2012-01-16 23:27 ` Andy Wingo
0 siblings, 1 reply; 9+ messages in thread
From: Mark H Weaver @ 2012-01-16 20:36 UTC (permalink / raw)
To: Andy Wingo; +Cc: guile-devel
Hi Andy!
Thanks again for working on this.
Andy Wingo <wingo@pobox.com> writes:
>>>> ***** Why are you adding anti-marks here?
>>>
>>> As the changelog noted (and a comment should have noted ;), the
>>> identifiers are anti-marked so that syntax transformers can introduce
>>> them, as-is.
>>>
>>> The purpose of this procedure is to get a list of identifiers, and to
>>> capture some subset of them. It will do so by introducing references to
>>> them in the expansion of some macro. However they are not introduced
>>> identifiers: they come from the code itself. They are input the macro,
>>> and as such need an anti-mark.
>>>
>>> The anti-mark will be stripped from the expansion when the transformer
>>> that called `bound-identifiers' returns.
>>
>> Does this mean that `bound-identifiers' will not function properly when
>> used outside of a macro? What about if it's used within a macro that
>> was generated by another macro (or things of that nature)? Are there
>> cases where you might need to strip more than one anti-mark?
>
> Well, bound-identifiers is a procedure, so if you are using it outside
> the dynamic extent of a transformer procedure, that means that you have
> a syntax object that you squirreled away from somewhere, so already
> we're in somewhat uncharted territory.
How about something like (bound-identifiers #'here)?
or (bound-identifiers #'x) where `x' is some lexical variable?
> Macro-generating macros should be fine, here. `expand-macro' is
> iterative, not recursive, so you don't need to strip anti-marks twice.
Ah, okay. Good point!
> I agree that this anti-mark has a bad smell, but the idea of a
> `bound-identifiers' procedure or form sounds like a good idea, so if you
> have any suggestions for improvement here, they are most welcome.
As I've already said, I don't think `bound-identifiers' will be useful
in a full implementation of `local-eval', so once we move to that
improved implementation, `bound-identifiers' will be left around as an
orphan: a primitive of dubious value, introduced specifically to
implement something that it turned out to be insufficient for.
If you insist on this strategy, I think what we really need is a list of
ribs, where each rib also specifies whether it is recursive. We don't
actually care about `let' vs `letrec' (though there's no harm in
providing that information in the interface, and it probably makes sense
to for consistency), but we _do_ care about the difference between
`let-syntax', `letrec-syntax', and internal bodies with
mutually-recursive `define-syntax' forms.
See how we're exposing increasingly complex internal psyntax structures
in order to achieve your dream of making `local-eval' sleep outside in
the shed?
>> [W]ith `local-eval', it seems to me quite plausible that gensym
>> collisions might occur. Suppose in one Guile session you compile a
>> procedure (foo) that uses (the-environment), and then in another Guile
>> session, you call (foo) and then `local-eval' with the environment
>> returned by (foo). Now the wrapper procedure splices together syntax
>> objects from two different Guile sessions into a single top-level
>> form, where (unlike in the macro case) all of these syntax objects are
>> lexicals, and thus depend on the gensyms and the labels.
>>
>> See how this is a problem now where it wasn't before?
>> Or am I missing something?
>
> To be perfectly honest, this stuff is very confusing to me, but I think
> I can see how this can happen, yes.
>
> I do think that it's important to fix this bug at some point, but IMO it
> is not a blocker for local-eval, much less 2.0.4.
I strongly disagree. Your implementation will clearly be buggy without
a proper solution to the collision of gensyms (labels and marks, at
least). I don't know about you, but personally I prefer rock-solid code
with clearly documented limitations (that almost no one is likely to hit
anyway) to buggy code.
If you don't want to deal with the gensym problem for 2.0.4, there's an
easy solution. Simply strip the wraps for now (as is done by my patch),
and everything will robust as long as we don't capture local syntax.
>> BTW, did you see my most recent model for thinking about `local-eval'?
>>
>> (the-environment) expands to (list (lambda () <expr>) ...), with one
>> element for every possible expression: a countably infinite list that
>> could be built lazily. `local-eval' simply chooses the appropriate
>> procedure from the list and calls it. A poor implementation strategy,
>> but the semantic meaning is quite clear, no?
>
> It sounds clear, but does it have any explanatory power? It sounds like
> it could apply just as well to any other computation...
I don't understand what you mean here. It seems to me that this model
can answer any question you could possibly have about the observable
behaviors of `the-environment' and `local-eval', besides their
efficiency. Can you provide a counter-example to this claim?
> Creating wraps is not the hack. It's creating wraps that are scoped in
> another specific module. With the-environment in psyntax, psyntax
> depends on (ice-9 local-eval).
See `build-primref' in psyntax. As you see, psyntax _already_ creates
wraps scoped in another specific module, namely (guile). This is
nothing new.
If you'd prefer, these helper macros could be moved into (guile) too.
I just thought it would be better to keep these helper macros private,
but I'd rather expose these helper macros than the new hastily-designed
magic primitives that you're proposing.
Also, it's not quite true that psyntax depends on (ice-9 local-eval) in
my patch. If (and only if) `the-environment' is encountered, (ice-9
local-eval) will be autoloaded. If (ice-9 local-eval) is missing or
contains a serious error, you won't even notice unless you use
(the-environment), and even then it will only prevent compilation of
that single top-level form or file.
> Ultimately, Guile should be a great platform for language research and
> experimentation. Like Racket is, in many ways. In order to support
> their multitudinous syntactic rackets, Racket gives the macro writer
> lots of power -- lots of accessors, lots of insight into what the
> expander is up to. We should do the same, where it makes sense to allow
> access to that information. We shouldn't restrict these sorts of
> experiments to the kinds of (grumpy, crotchety) people that maintain
> psyntax.scm ;-)
Okay, fair enough. This is a reasonable perspective.
>> However, this requires re-expanding the bodies of the `syntax-rules'
>> forms, so now the environment structures _do_ matter. Ultimately, this
>> means that a set of mutually recursive local macros must be recreated in
>> `local-eval' as a `letrec-syntax'.
>>
>> Does that make sense?
>
> Yes I see where you're going here. IMO, we should be providing the
> tools to let you do these experiments in parallel, without delaying the
> release of other bugfixes.
Well, if you insist in this foolish quest to banish `the-environment' to
sleep in the shed as a second-class citizen, I cannot stop you :)
However, if you're going to add new primitives simply to enable this
banishment, let's at least get them right. At least replace
`bound-identifiers' with a more powerful primitive that provides the
information we need, along the lines I've suggested above. Let's not
export a new half-baked interface that turns out to be useless.
Thanks,
Mark
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: local-eval on syntax-local-binding, bound-identifiers
2012-01-16 20:36 ` Mark H Weaver
@ 2012-01-16 23:27 ` Andy Wingo
2012-01-17 10:55 ` David Kastrup
2012-01-19 11:55 ` Andy Wingo
0 siblings, 2 replies; 9+ messages in thread
From: Andy Wingo @ 2012-01-16 23:27 UTC (permalink / raw)
To: Mark H Weaver; +Cc: guile-devel
Hi Mark,
On Mon 16 Jan 2012 21:36, Mark H Weaver <mhw@netris.org> writes:
> Thanks again for working on this.
And thank you again for all your work, and patience with my
pigheadedness.
> if you insist in this foolish quest to banish `the-environment' to
> sleep in the shed as a second-class citizen, I cannot stop you :)
TBH I think this is the best thing we can do for local-eval. We
preserve flexibility for local-eval, make other experiments possible,
and the local-eval implementation is a bit more perspicacious, as the
scoping is more lexical (in the same file, even).
I know there's a smilie in your statement, but really, it's not just
local-eval: there's loads more that should be broken out into modules
over time, somehow :) Think of it as building a hippie commune of
functionality, instead of making everyone live in the same house :)
(OK, that's stretching it a bit, but perhaps it is partially apt?)
Now, specific commentary.
> How about something like (bound-identifiers #'here)?
scheme@(guile-user)> (bound-identifiers #'here)
$5 = ()
scheme@(guile-user)> (let ((x 10)) (bound-identifiers #'here))
$6 = (#(syntax-object x ((#f top) shift #(ribcage #(x) #((top)) #("i176"))) (hygiene guile-user)))
What should the answer be in this case? Would you expect `x' in the
list? Certainly for the-environment you would. But here:
scheme@(guile-user)> (define-syntax bound-here
(lambda (x)
(with-syntax (((id ...)
(map (lambda (id) (datum->syntax x id))
(bound-identifiers #'here))))
#'(list 'id ...))))
scheme@(guile-user)> bound-here
$7 = (#(syntax-object x ((#f top) shift #(ribcage #(x) #((top)) #("i192"))) (hygiene guile-user)))
scheme@(guile-user)> (let ((y 10)) bound-here)
$8 = (#(syntax-object x ((#f top) shift #(ribcage #(x) #((top)) #("i192"))) (hygiene guile-user)))
So, it seems to be sensible.
Now, what to do with these identifiers: you if you introduce one into
another macro, the mark will indeed be stripped. I'm not sure what else
you can do with a syntax-object, actually! Pass it directly to eval or
compile, I guess, and in that case we do lose, as the anti-mark isn't
stripped. But that's the case for other syntax objects captured in a
syntax transformer, as well.
Should we anti-mark only within the dynamic extent of a transformer, I
wonder?
> As I've already said, I don't think `bound-identifiers' will be useful
> in a full implementation of `local-eval', so once we move to that
> improved implementation, `bound-identifiers' will be left around as an
> orphan: a primitive of dubious value, introduced specifically to
> implement something that it turned out to be insufficient for.
Hummmmm. Definitely something to think about.
What if instead we implemented closure serialization somehow? Then we
would handle procedural macros too, and bound-identifiers would still be
sufficient.
Maybe that idea is a little too crazy.
If we have to lexical contours associated with bindings, recursive is
only one bit: you probably also need letrec vs letrec*.
>> To be perfectly honest, this stuff is very confusing to me, but I think
>> I can see how this can happen, yes.
>>
>> I do think that it's important to fix this bug at some point, but IMO it
>> is not a blocker for local-eval, much less 2.0.4.
>
> I strongly disagree. Your implementation will clearly be buggy without
> a proper solution to the collision of gensyms (labels and marks, at
> least). I don't know about you, but personally I prefer rock-solid code
> with clearly documented limitations (that almost no one is likely to hit
> anyway) to buggy code.
>
> If you don't want to deal with the gensym problem for 2.0.4, there's an
> easy solution. Simply strip the wraps for now (as is done by my patch),
> and everything will robust as long as we don't capture local syntax.
Thinking about it a little more, labels are a non-issue. All they need
to be is unique in the sense of eq?. Labels are strings. If they are
loaded in separate compilation units, they will be unique, no matter
what their contents.
Labels are more important than marks, also, for the correctness of the
algorithm. A mark collision is only an issue if there is also a
symbolic collision. Label collision could alias completely unrelated
bindings.
Anyway, I would rather serialize bad marks than no marks. That's my
personal opinion ;-) But if you think this is a huge issue, let's fix
the marks to be more unique, no?
Note that there is a well-known optimization that you don't actually
need to generate the characters corresponding to a gensym until they are
needed. It might serve your purposes.
OK, I'm getting very sleepy now :) Let me know your thoughts. It would
be great if all of this could land before Sunday. Though the cricket
folk say "pace is nothing without guile", Guile is nothing without a
good development pace ;-)
Cheers,
Andy
--
http://wingolog.org/
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: local-eval on syntax-local-binding, bound-identifiers
2012-01-16 23:27 ` Andy Wingo
@ 2012-01-17 10:55 ` David Kastrup
2012-01-19 11:55 ` Andy Wingo
1 sibling, 0 replies; 9+ messages in thread
From: David Kastrup @ 2012-01-17 10:55 UTC (permalink / raw)
To: guile-devel
Andy Wingo <wingo@pobox.com> writes:
> What if instead we implemented closure serialization somehow? Then we
> would handle procedural macros too, and bound-identifiers would still be
> sufficient.
>
> Maybe that idea is a little too crazy.
Are we still talking about Scheme? The language with
call-with-current-continuation? "a little too crazy" is not a
criterion. "Too complex to work or or with" would be. Those are
related, but not necessarily the same.
--
David Kastrup
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: local-eval on syntax-local-binding, bound-identifiers
2012-01-16 23:27 ` Andy Wingo
2012-01-17 10:55 ` David Kastrup
@ 2012-01-19 11:55 ` Andy Wingo
1 sibling, 0 replies; 9+ messages in thread
From: Andy Wingo @ 2012-01-19 11:55 UTC (permalink / raw)
To: guile-devel
On Tue 17 Jan 2012 00:27, Andy Wingo <wingo@pobox.com> writes:
> TBH I think this is the best thing we can do for local-eval. We
> preserve flexibility for local-eval, make other experiments possible,
> and the local-eval implementation is a bit more perspicacious, as the
> scoping is more lexical (in the same file, even).
"Perspicuous", rather. How embarrassing!
Andy
--
http://wingolog.org/
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2012-01-19 11:55 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-01-15 22:17 local-eval on syntax-local-binding, bound-identifiers Andy Wingo
2012-01-16 3:30 ` Mark H Weaver
2012-01-16 11:01 ` Andy Wingo
2012-01-16 13:28 ` Mark H Weaver
2012-01-16 15:17 ` Andy Wingo
2012-01-16 20:36 ` Mark H Weaver
2012-01-16 23:27 ` Andy Wingo
2012-01-17 10:55 ` David Kastrup
2012-01-19 11:55 ` Andy Wingo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).