Fixing the slib mess

Fixing the slib mess

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 3010 bytes --]

Dear Guile hackers,

What nice work you are doing!

For those who don't know me, I'm a Guile developer who has been doing
other stuff for some time.

When trying to use guile 2 for logic programming I discovered that the
slib interface is again broken (and has been for quite some time).
This easily happens because it is a very fragile interface.  The way
this is supposed to be used (and as documented in the manual), one
does a

  (use-modules (ice-9 slib))

and can then do

  (require 'modular)

etc.

The module (ice-9 slib) forms a kind of sandbox so that all new
definitions that are imported through "require" are loaded as local
bindings in the (ice-9 slib) module and are exported through the
public interface of (ice-9 slib).

The implementation of the interface has two sides.  One, the file
ice-9/slib.scm, is owned by Guile.  The other, slib/guile.init, is
owned by slib.  slib has such .init files for some common scheme
implementations but I early on noticed that that the guile.init file
is not really maintained.  I decided that it would be more robust if
slib.scm incorporated most of the interface so that it would be easy
to update it as Guile changed.  But of course slib also changed and at
some point others felt that guile.init should contain most of the
interface and the bulk of slib.scm was moved there.  As we have seen,
this didn't make things much better.

I'll let you ponder on how to handle the fundamental problems with
this interface, but, as a Guile user, I think it would be nice if the
interface works as written in the manual.  Attached to this email
you'll find two patches.  The patch to slib.scm copies a snippet of
code from guile.init so that they agree with eachother and with the
Guile reference manual on how to find slib in the filesystem.  This
patch for example makes SCHEME_LIBRARY_PATH work as described.

I've tried to write the patch to guile.init so that it can play well
with older Guile versions, but we should test this.  In order to make
it work with Guile 2, though, I had to introduce a new syntax binding
syntax-toplevel?.  Given a syntax object (as available within a
syntax-case transformer), it decides if the object originates from top
level context.  It is used, as in the old memoizing macro transformer,
to choose whether to define-public or just define.

*But*, the proper implementation of syntax-toplevel? requires
modification of psyntax.scm and adding it to the (system syntax)
module.  I didn't want to do this until I've had your comments, so the
present patch has its own syntax-object accessors (which breaks
abstraction and is therefore not a real solution).  I should also say
that I have not yet fixed the slib interface to the new Guile uniform
arrays, so there's a lot of slib functionality which won't yet work.

Comments?  Can I add syntax-toplevel? to psyntax.scm and (system
syntax)?  Do you think it is reasonable to submit something along the
line of guile.init.diff to slib guile.init?

Best regards,
Mikael Djurfeldt

[-- Attachment #2: slib.scm.diff --]
[-- Type: application/octet-stream, Size: 1151 bytes --]

--- ice-9/slib.scm.dpkg-dist	2012-03-18 20:16:21.000000000 +0100
+++ ice-9/slib.scm	2012-10-15 23:25:01.675171256 +0200
@@ -33,7 +33,24 @@
 
 \f
 ;; Initialize SLIB.
-(load-from-path "slib/guile.init")
+(define library-vicinity
+  (let ((library-path
+	 (or
+	  ;; Use this getenv if your implementation supports it.
+	  (and (defined? 'getenv) (getenv "SCHEME_LIBRARY_PATH"))
+	  ;; Rob Browning sent this; I'm not sure its a good idea.
+          ;; See if we can find slib/guile.init (cf. implementation-vicinity).
+          (let ((path (%search-load-path "slib/guile.init")))
+            (and path (substring path 0 (- (string-length path) 10))))
+	  ;; Use this path if your scheme does not support GETENV
+	  ;; or if SCHEME_LIBRARY_PATH is not set.
+;; Changed from /usr/lib/slib to /usr/share/slib by tb@debian.org for
+;; Debian packaging.
+;;	  "/usr/lib/slib/"
+	  "/usr/share/slib/")))
+    (lambda () library-path)))
+
+(primitive-load (in-vicinity (library-vicinity) "guile.init"))
 
 ;; SLIB redefines a few core symbols based on their default definition.
 ;; Thus, we only replace them at this point so that their previous definition

[-- Attachment #3: guile.init.diff --]
[-- Type: application/octet-stream, Size: 8326 bytes --]

--- slib/guile.init.dpkg-dist	2010-05-06 08:20:58.000000000 +0200
+++ slib/guile.init	2012-10-22 00:06:21.457707670 +0200
@@ -7,11 +7,29 @@
     (define-module (ice-9 slib)))	; :no-backtrace
 (define slib-module (current-module))
 
-(define base:define define)
-(define define
-  (procedure->memoizing-macro
-   (lambda (exp env)
-     (cons (if (= 1 (length env)) 'define-public 'base:define) (cdr exp)))))
+;;(define base:define define)
+;; The following line should work for all version >= 1.6
+(local-define '(base:define) (local-ref '(define)))
+(cond-expand
+ (guile-2
+  (define-syntax define
+    (let* ((syntax-object-wrap (lambda (s) (vector-ref s 2)))
+	   (wrap-subst cdr)
+	   (syntax-toplevel?
+	    (lambda (s)
+	      (null? (cdr (wrap-subst (syntax-object-wrap s)))))))
+      (lambda (x)
+	(syntax-case x ()
+	  ((_ f0 f1 ...)
+	   (syntax-toplevel? #'f0)
+	   (syntax (define-public f0 f1 ...)))
+	  ((_ f0 f1 ...)
+	   (syntax (base:define f0 f1 ...))))))))
+ (guile
+  (define define
+    (procedure->memoizing-macro
+     (lambda (exp env)
+       (cons (if (= 1 (length env)) 'define-public 'base:define) (cdr exp)))))))
 
 ;;; Hack to make syncase macros work in the slib module
 (if (nested-ref the-root-module '(app modules ice-9 syncase))
@@ -274,12 +292,13 @@
 ;; calling waitpid with WUNTRACED, but allow for it anyway, just in
 ;; case.
 (define system
-      (let ((guile-core-system system))
+      (let ((guile-core-system (nested-ref the-root-module '(system))))
 	(lambda (str)
 	  (define st (guile-core-system str))
 	  (or (status:exit-val st)
 	      (+ 128 (or (status:term-sig st)
 			 (status:stop-sig st)))))))
+
 ;; This has to be done after the definition so that the original
 ;; binding will still be visible during the definition.
 (if (string>=? (scheme-implementation-version) "1.8")
@@ -305,7 +324,7 @@
        (re-export write-line)))
 
 (define delete-file
-      (let ((guile-core-delete-file delete-file))
+      (let ((guile-core-delete-file (nested-ref the-root-module '(delete-file))))
 	(lambda (filename)
 	  (catch 'system-error
 		 (lambda () (guile-core-delete-file filename) #t)
@@ -325,7 +344,7 @@
 (define (make-exchanger obj)
   (lambda (rep) (let ((old obj)) (set! obj rep) old)))
 (define open-file
-      (let ((guile-core-open-file open-file))
+      (let ((guile-core-open-file (nested-ref the-root-module '(open-file))))
 	(lambda (filename modes)
 	  (guile-core-open-file filename
 				(if (symbol? modes)
@@ -346,17 +365,16 @@
     (for-each close-port ports)
     ans))
 
-(if (not (defined? 'browse-url))
-    ;; Nothing special to do for this, so straight from
-    ;; Template.scm.  Maybe "sensible-browser" for a debian
-    ;; system would be worth trying too (and would be good on a
-    ;; tty).
-    (define (browse-url url)
-      (define (try cmd end) (zero? (system (string-append cmd url end))))
-      (or (try "netscape-remote -remote 'openURL(" ")'")
-	  (try "netscape -remote 'openURL(" ")'")
-	  (try "netscape '" "'&")
-	  (try "netscape '" "'"))))
+;; Nothing special to do for this, so straight from
+;; Template.scm.  Maybe "sensible-browser" for a debian
+;; system would be worth trying too (and would be good on a
+;; tty).
+(define (browse-url url)
+  (define (try cmd end) (zero? (system (string-append cmd url end))))
+  (or (try "netscape-remote -remote 'openURL(" ")'")
+      (try "netscape -remote 'openURL(" ")'")
+      (try "netscape '" "'&")
+      (try "netscape '" "'")))
 
 ;;; "rationalize" adjunct procedures.
 ;;(define (find-ratio x e)
@@ -369,7 +387,7 @@
 ;;; be returned by CHAR->INTEGER.
 ;; In Guile-1.8.0: (string>? (string #\000) (string #\200)) ==> #t
 (if (string=? (version) "1.8.0")
-    (define char-code-limit 128))
+    (set! char-code-limit 128))
 
 ;;; MOST-POSITIVE-FIXNUM is used in modular.scm
 ;;(define most-positive-fixnum #x0FFFFFFF)
@@ -386,49 +404,55 @@
 ;;; return if exiting not supported.
 (define slib:exit quit)
 
+(define slib:load #f)
+(define slib:load-from-path #f)
+(define read #f)
 (cond ((string>=? (scheme-implementation-version) "1.8")
-       (define (slib:load-helper loader)
-	 (lambda (name)
-	   (save-module-excursion
-	    (lambda ()
-	      (set-current-module slib-module)
-	      (let ((errinfo (catch 'system-error
-				    (lambda () (loader name) #f)
-				    (lambda args args))))
-		(if (and errinfo
-			 (catch 'system-error
-				(lambda () (loader (string-append name ".scm")) #f)
-				(lambda args args)))
-		    (apply throw errinfo)))))))
-       (define slib:load (slib:load-helper load))
-       (define slib:load-from-path (slib:load-helper load-from-path))
+       (let ((slib:load-helper
+	      (lambda (loader)
+		(lambda (name)
+		  (save-module-excursion
+		   (lambda ()
+		     (set-current-module slib-module)
+		     (let ((errinfo (catch 'system-error
+					   (lambda () (loader name) #f)
+					   (lambda args args))))
+		       (if (and errinfo
+				(catch 'system-error
+				       (lambda () (loader (string-append name ".scm")) #f)
+				       (lambda args args)))
+			   (apply throw errinfo)))))))))
+	 (set! slib:load (slib:load-helper load))
+	 (set! slib:load-from-path (slib:load-helper load-from-path)))
+       (set! read (nested-ref the-root-module '(read)))
        )
       (else
        ;;Here for backward compatability
-       (define scheme-file-suffix
-	 (let ((suffix (case (software-type)
-			 ((nosve) "_scm")
-			 (else ".scm"))))
-	   (lambda () suffix)))
-
-       (define (guile:wrap-case-insensitive proc)
-	 (lambda args
-	   (save-module-excursion
-	    (lambda ()
-	      (set-current-module slib-module)
-	      (let ((old (read-options)))
-		(dynamic-wind
-		    (lambda () (read-enable 'case-insensitive))
-		    (lambda () (apply proc args))
-		    (lambda () (read-options old))))))))
-
-       (define read (guile:wrap-case-insensitive read))
-
-       (define slib:load
-	 (let ((load-file (guile:wrap-case-insensitive load)))
-	   (lambda (<pathname>)
-	     (load-file (string-append <pathname> (scheme-file-suffix))))))
-       ))
+       (let ((scheme-file-suffix
+	      (let ((suffix (case (software-type)
+			      ((nosve) "_scm")
+			      (else ".scm"))))
+		(lambda () suffix)))
+	     (guile:wrap-case-insensitive
+	       (lambda (proc)
+		 (lambda args
+		   (save-module-excursion
+		    (lambda ()
+		      (set-current-module slib-module)
+		      (let ((old (read-options)))
+			(dynamic-wind
+			    (lambda () (read-enable 'case-insensitive))
+			    (lambda () (apply proc args))
+			    (lambda () (read-options old))))))))))
+
+	 (set! read
+	       (guile:wrap-case-insensitive
+		(nested-ref the-root-module '(read))))
+
+	 (set! slib:load
+	       (let ((load-file (guile:wrap-case-insensitive load)))
+		 (lambda (<pathname>)
+		   (load-file (string-append <pathname> (scheme-file-suffix)))))))))
 
 ;;;(SLIB:LOAD-SOURCE "foo") should load "foo.scm" or with whatever
 ;;;suffix all the module files in SLIB have.  See feature 'SOURCE.
@@ -645,20 +669,24 @@
 		  (require 'object->string)
 		  (set! seed (object->limited-string seed 50)))))
     (seed->random-state seed)))
-(if (not (defined? 'random:chunk))
-    (define (random:chunk sta) (random 256 sta)))
+;;(if (not (defined? 'random:chunk))
+(define (random:chunk sta) (random 256 sta))
 
 ;;; workaround for Guile 1.6.7 bug
-(cond ((or (array? 'guile) (array? '(1 6 7)))
-       (define array?
-	 (let ((old-array? array?))
-	   (lambda (obj)
-	     (and (old-array? obj)
-		  (not (or (list? obj)
-			   (symbol? obj)
-			   (record? obj)))))))
-       (if (string>=? (scheme-implementation-version) "1.8")
-	   (module-replace! (current-module) '(array?)))))
+(base:define array? #f)
+(let ((guile-core-array? (nested-ref the-root-module '(array?))))
+  (cond ((or (guile-core-array? 'guile) (guile-core-array? '(1 6 7)))
+	 (set! array?
+	       (lambda (obj)
+		 (and (guile-core-array? obj)
+		      (not (or (list? obj)
+			       (symbol? obj)
+			       (record? obj))))))
+	 (export array?)
+	 (if (string>=? (scheme-implementation-version) "1.8")
+	     (module-replace! (current-module) '(array?))))
+	(else
+	 (set! array? guile-core-array?))))
 
 ;;; Support for older versions of Scheme.  Not enough code for its own file.
 ;;(define (last-pair l) (if (pair? (cdr l)) (last-pair (cdr l)) l))

Re: Fixing the slib mess

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1514 bytes --]

Hi Mikael and welcome back!


*But*, the proper implementation of syntax-toplevel? requires
> modification of psyntax.scm and adding it to the (system syntax)
> module.  I didn't want to do this until I've had your comments, so the
> present patch has its own syntax-object accessors (which breaks
> abstraction and is therefore not a real solution).  I should also say
> that I have not yet fixed the slib interface to the new Guile uniform
> arrays, so there's a lot of slib functionality which won't yet work.
>
> Comments?  Can I add syntax-toplevel? to psyntax.scm and (system
> syntax)?  Do you think it is reasonable to submit something along the
> line of guile.init.diff to slib guile.init?
>
> Best regards,
> Mikael Djurfeldt
>

I can answer with some kind of suggestion here.

in (system syntax) there is syntax-local-binding which you can use for
example as


(define-syntax f
  (lambda (x)
    (syntax-case x ()
      ((_ x)
        (call-with-values (lambda () (syntax-local-binding #'x))
                                (lambda (x y) (pk x) (pk y))) #'#t))))

Then,

scheme@(guile-user) [1]> (f +)

;;; (global)

;;; ((+ guile-user))

And,

scheme@(guile-user) [1]> (let ((s 1)) (f s))

;;; (lexical)

;;; (s-490)

(let ((s 1)) (define-syntax g (lambda (x) #'#f)) (f g))

;;; (displaced-lexical)

;;; (#f)

I'm not sure what exactly syntax-toplevel? does, but can you base it on
syntax-local-binding?
And if not is it possible to change syntax-local-binding so that you can
use it?

Regards
Stefan

[-- Attachment #2: Type: text/html, Size: 1888 bytes --]

Re: Fixing the slib mess

[Mikael Djurfeldt]

On Mon, Oct 22, 2012 at 8:31 PM, Stefan Israelsson Tampe
<stefan.itampe@gmail.com> wrote:
>> Comments?  Can I add syntax-toplevel? to psyntax.scm and (system
>> syntax)?
> [...]
> I can answer with some kind of suggestion here.
>
> in (system syntax) there is syntax-local-binding which you can use for
> example as
>
>
> (define-syntax f
>   (lambda (x)
>     (syntax-case x ()
>       ((_ x)
>         (call-with-values (lambda () (syntax-local-binding #'x))
>                                 (lambda (x y) (pk x) (pk y))) #'#t))))
>
> Then,
>
> scheme@(guile-user) [1]> (f +)
>
> ;;; (global)
>
> ;;; ((+ guile-user))
>
> And,
>
> scheme@(guile-user) [1]> (let ((s 1)) (f s))
>
> ;;; (lexical)
>
> ;;; (s-490)
>
> (let ((s 1)) (define-syntax g (lambda (x) #'#f)) (f g))
>
> ;;; (displaced-lexical)
>
> ;;; (#f)
>
> I'm not sure what exactly syntax-toplevel? does, but can you base it on
> syntax-local-binding?
> And if not is it possible to change syntax-local-binding so that you can use
> it?

Thanks, Stefan.

(syntax-toplevel?) expands to #t if occurs in a context (position in
the code if you prefer) where a (define x #f) would create/set! a
global binding for x.  It expands to #f otherwise.

I had a look at syntax-local-binding, but decided that
syntax-toplevel? was needed since the latter is not trying to
determine the nature of an existing binding but rather the nature of
the context.  Of course oncould probe the context by first creating a
new binding (with some random name) and then use syntax-local-binding
to determine the nature of the context by looking at the new binding,
but that seems somewhat invasive. :-)

Re: Fixing the slib mess

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1886 bytes --]

Yes in that case this stands on it's own!
/Stefan

On Mon, Oct 22, 2012 at 9:11 PM, Mikael Djurfeldt <mikael@djurfeldt.com>wrote:

> On Mon, Oct 22, 2012 at 8:31 PM, Stefan Israelsson Tampe
> <stefan.itampe@gmail.com> wrote:
> >> Comments?  Can I add syntax-toplevel? to psyntax.scm and (system
> >> syntax)?
> > [...]
> > I can answer with some kind of suggestion here.
> >
> > in (system syntax) there is syntax-local-binding which you can use for
> > example as
> >
> >
> > (define-syntax f
> >   (lambda (x)
> >     (syntax-case x ()
> >       ((_ x)
> >         (call-with-values (lambda () (syntax-local-binding #'x))
> >                                 (lambda (x y) (pk x) (pk y))) #'#t))))
> >
> > Then,
> >
> > scheme@(guile-user) [1]> (f +)
> >
> > ;;; (global)
> >
> > ;;; ((+ guile-user))
> >
> > And,
> >
> > scheme@(guile-user) [1]> (let ((s 1)) (f s))
> >
> > ;;; (lexical)
> >
> > ;;; (s-490)
> >
> > (let ((s 1)) (define-syntax g (lambda (x) #'#f)) (f g))
> >
> > ;;; (displaced-lexical)
> >
> > ;;; (#f)
> >
> > I'm not sure what exactly syntax-toplevel? does, but can you base it on
> > syntax-local-binding?
> > And if not is it possible to change syntax-local-binding so that you can
> use
> > it?
>
> Thanks, Stefan.
>
> (syntax-toplevel?) expands to #t if occurs in a context (position in
> the code if you prefer) where a (define x #f) would create/set! a
> global binding for x.  It expands to #f otherwise.
>
> I had a look at syntax-local-binding, but decided that
> syntax-toplevel? was needed since the latter is not trying to
> determine the nature of an existing binding but rather the nature of
> the context.  Of course oncould probe the context by first creating a
> new binding (with some random name) and then use syntax-local-binding
> to determine the nature of the context by looking at the new binding,
> but that seems somewhat invasive. :-)
>

[-- Attachment #2: Type: text/html, Size: 2565 bytes --]

Re: Fixing the slib mess

[Mikael Djurfeldt]

On Mon, Oct 22, 2012 at 1:11 AM, Mikael Djurfeldt <mikael@djurfeldt.com> wrote:
> Comments?  Can I add syntax-toplevel? to psyntax.scm and (system
> syntax)?  Do you think it is reasonable to submit something along the
> line of guile.init.diff to slib guile.init?

If I get an OK, then I would of course put some further work into this
so that the full feature set of slib (including uniform arrays) gets
operational.

If people find my prodigious use of "nested-ref" ugly (but note that
the original code already makes use of module system primitives), I
could remove most of them as well.

Re: Fixing the slib mess

[Mark H Weaver]

Hi Mikael!

It's great to see you on guile-devel again, and it would be good to have
a working slib on Guile 2.  Thanks for working on this :)

Mikael Djurfeldt <mikael@djurfeldt.com> writes:
> Comments?  Can I add syntax-toplevel? to psyntax.scm and (system
> syntax)?

FWIW, it sounds reasonable to add something like 'syntax-toplevel?'.
However, if it's going to be added as a public API, we ought to be
careful to get it right.  Consider this top-level form:

  (let-syntax ((foo (syntax-rules () ((foo x) (quote x)))))
    (define bar (foo (a b))))

Here, although the 'define' is expanded within a non-toplevel syntactic
environment, after expansion it ends up within a top-level environment,
and indeed 'bar' becomes a top-level binding.  That's because the body
of a 'let-syntax' or 'letrec-syntax' gets spliced into the outer
context, sort of like a 'begin' but with some syntax bindings added.

It looks to me like your current implementation of 'syntax-toplevel?'
is actually testing for a top-level _syntactic_ environment, but what
you ought to be testing for here is slightly different.  I'm not sure
what is the proper term, but for now I'll call it a top-level
_execution_ environment.

I'm also not quite sure how best to test for a top-level execution
environment.  I'm not sure whether the wrap contains enough information
to determine that.

It might be easier to handle this with 'define-syntax-parameter' and
'syntax-parameterize'.  The idea would be that within slib, 'define'
would be a syntax parameter.  Its default expansion would turn it into
'define-public', and also parameterize 'define' to mean 'base:define'
within the body.  If needed, you could also define 'let' and maybe some
other things to parameterize 'define' within the body.

Another option would be to make 'export' a syntax parameter, and
parameterize it to a no-op within lexical contours such as 'define' and
'let'.

What do you think?

> I should also say that I have not yet fixed the slib interface to the
> new Guile uniform arrays, so there's a lot of slib functionality which
> won't yet work.

I happen to be working on the reader lately.  Would it help to implement
SRFI-58 in our reader?  Note that SRFI-58 array notation conflicts with
Guile's own array notation, e.g. #1a("a" "b") is legal in both syntaxes
and means different things.  So we'd need to use a reader-directive such
as #!srfi-58 to change the reader mode on that port.  Would that help?

Thanks for working on this!

     Regards,
       Mark

Re: Fixing the slib mess

[Mikael Djurfeldt]

On Mon, Oct 22, 2012 at 11:51 PM, Mark H Weaver <mhw@netris.org> wrote:
> It looks to me like your current implementation of 'syntax-toplevel?'
> is actually testing for a top-level _syntactic_ environment, but what
> you ought to be testing for here is slightly different.

You are absolutely right.  Thank you for spotting this.  Unfortunately
my scheming knowledge is a bit rusty.

> I'm not sure whether the wrap contains enough information to determine that.

I don't think it does either.

> It might be easier to handle this with 'define-syntax-parameter' and
> 'syntax-parameterize'.  The idea would be that within slib, 'define'
> would be a syntax parameter.  Its default expansion would turn it into
> 'define-public', and also parameterize 'define' to mean 'base:define'
> within the body.  If needed, you could also define 'let' and maybe some
> other things to parameterize 'define' within the body.
>
> Another option would be to make 'export' a syntax parameter, and
> parameterize it to a no-op within lexical contours such as 'define' and
> 'let'.
>
> What do you think?

Correct me if I'm wrong, but doesn't this involve re-defining the
syntax for all forms with bodies (in order to introduce the
syntax-parameterize form)?

> I happen to be working on the reader lately.  Would it help to implement
> SRFI-58 in our reader?

While I think SRFI-58 support is great, I don't think slib needs it
because it doesn't, to my knowledge, use read syntax.

Re: Fixing the slib mess

[Mark H Weaver]

Hi Mikael,

Mikael Djurfeldt <mikael@djurfeldt.com> writes:
> On Mon, Oct 22, 2012 at 11:51 PM, Mark H Weaver <mhw@netris.org> wrote:
>> It might be easier to handle this with 'define-syntax-parameter' and
>> 'syntax-parameterize'.  The idea would be that within slib, 'define'
>> would be a syntax parameter.  Its default expansion would turn it into
>> 'define-public', and also parameterize 'define' to mean 'base:define'
>> within the body.  If needed, you could also define 'let' and maybe some
>> other things to parameterize 'define' within the body.
>>
>> Another option would be to make 'export' a syntax parameter, and
>> parameterize it to a no-op within lexical contours such as 'define' and
>> 'let'.
>>
>> What do you think?
>
> Correct me if I'm wrong, but doesn't this involve re-defining the
> syntax for all forms with bodies (in order to introduce the
> syntax-parameterize form)?

Yes, I guess it would, although perhaps you could come up with a smaller
set of forms that would be sufficient for slib.

I'm surprised that slib doesn't have a mechanism for making this job
easier for Scheme systems that support modules.  It seems like
generating the list of exports should not be so difficult.  Why doesn't
slib use something like 'define-public' to facilitate this?

Anyway, here's another idea: after requiring a new slib package, iterate
over the entire list of top-level bindings in the slib module and export
everything.

What do you think?

One more thing: ideally, any logic that peeks into Guile internals or is
likely to change between Guile versions should be in slib.scm, and
anything that's likely to change between slib versions should be in
guile.init.

Does that make sense?

    Thanks,
      Mark

Re: Fixing the slib mess

[Mikael Djurfeldt]

On Tue, Oct 23, 2012 at 8:01 PM, Mark H Weaver <mhw@netris.org> wrote:
> Anyway, here's another idea: after requiring a new slib package, iterate
> over the entire list of top-level bindings in the slib module and export
> everything.
>
> What do you think?

I think it sounds like the best idea so far.  I'll try to go with this.

> One more thing: ideally, any logic that peeks into Guile internals or is
> likely to change between Guile versions should be in slib.scm, and
> anything that's likely to change between slib versions should be in
> guile.init.
>
> Does that make sense?

Three problems come to my mind:

1. guile.init is really mostly a kind of interface, meaning that
changes in both Guile and slib can affect the same pieces of code.

2. guile.init is supposed to work with a series of Guile versions.  If
I now try to do a larger reorganization, I will likely break
compatibility with some older Guile versions, especially if I start
moving things back to ice-9/slib.scm.

3. I don't really have time currently to do a full reorganization.

Otherwise I concur with what you say.

Problem 2 could be solved by asking Aubrey Jaffer (or who is currently
maintaining slib) to include a new version of the file, guile2.init,
in addition to guile.init...

I'll think about your suggestions and try to come up with new patches.

Best regards,
Mikael D.

Re: Fixing the slib mess

[Mark H Weaver]

Mikael Djurfeldt <mikael@djurfeldt.com> writes:
> On Tue, Oct 23, 2012 at 8:01 PM, Mark H Weaver <mhw@netris.org> wrote:
>> Anyway, here's another idea: after requiring a new slib package, iterate
>> over the entire list of top-level bindings in the slib module and export
>> everything.
>>
>> What do you think?
>
> I think it sounds like the best idea so far.  I'll try to go with this.

Sounds good.

>> One more thing: ideally, any logic that peeks into Guile internals or is
>> likely to change between Guile versions should be in slib.scm, and
>> anything that's likely to change between slib versions should be in
>> guile.init.
>>
>> Does that make sense?
>
> Three problems come to my mind:
>
> 1. guile.init is really mostly a kind of interface, meaning that
> changes in both Guile and slib can affect the same pieces of code.
>
> 2. guile.init is supposed to work with a series of Guile versions.  If
> I now try to do a larger reorganization, I will likely break
> compatibility with some older Guile versions, especially if I start
> moving things back to ice-9/slib.scm.
>
> 3. I don't really have time currently to do a full reorganization.

Okay, that's fine.  I'd say go ahead and do whatever you have time for.
We can always rethink the interface between slib.scm and guile.init for
Guile 2.2, if someone volunteers to do that work.

    Thanks,
      Mark

Re: Fixing the slib mess

[Andy Wingo]

Hello Mikael,

A pleasure to see you around!

On Mon 22 Oct 2012 01:11, Mikael Djurfeldt <mikael@djurfeldt.com> writes:

> When trying to use guile 2 for logic programming I discovered that the
> slib interface is again broken (and has been for quite some time).

I am very sorry that I did not see this thread before hacking on this
recently.  Somehow over the past three or four months I just managed to
drop everything and the inboxes filled without being filtered or drained
in any way -- and to attack that I decided to just run through
individual lists in order.  A strange strategy, but it is good for
honing the "does something need to be done about this or can I drop it?"
instinct.

Anyway I picked up something in the user list about Slib, looked into
it, and then decided to fix it, without having seen this mail --
resulting in the recent patches to Slib CVS and Guile git.  I'm sorry to
have stepped on your toes here.  In any case I didn't check it
thoroughly, so surely there are issues yet to resolve.

> The implementation of the interface has two sides.  One, the file
> ice-9/slib.scm, is owned by Guile.  The other, slib/guile.init, is
> owned by slib.  slib has such .init files for some common scheme
> implementations but I early on noticed that that the guile.init file
> is not really maintained.  I decided that it would be more robust if
> slib.scm incorporated most of the interface so that it would be easy
> to update it as Guile changed.  But of course slib also changed and at
> some point others felt that guile.init should contain most of the
> interface and the bulk of slib.scm was moved there.  As we have seen,
> this didn't make things much better.

Yes, in many ways I would like to have the interface in Guile.  However
it seems that time has shown that it really wants to live in slib --
probably because that's where people care most about slib.

At least with Guile 2 we have managed to clean up many of the version
dependent hacks, by just delegating to a fresh file for Guile 2.

Anyway.  Perhaps I did the wrong thing in fixing it this way?  I would
be very happy to commit anything you have.  Please take a look at both
Slib and Guile from their version control systems, and the recent patch
about `include'.  Aubrey seems quite responsive in dealing with patches,
so if there is a change to make, I'm sure we can get it in.

> *But*, the proper implementation of syntax-toplevel? requires
> modification of psyntax.scm and adding it to the (system syntax)
> module.

Do you have a new patch for this one?

Regards,

Andy
-- 
http://wingolog.org/

Re: Fixing the slib mess

[Mikael Djurfeldt]

Hi Andy,

No problem at all!  In fact, apologies are entirely on my side: I
thought I would get time to hack on this before and during Christmas,
but this turned out not to be true.

Great that you fixed it!  If I have anything to add, I will of course
bring that up.

Now, I'm looking into porting Gerald Sussman's scmutils to Guile-2.0.
I'm aware of an older port by Daniel Gildea but I don't think that
uses GOOPS:

I'm currently wondering if it could make sense to try to make an
mit-scheme compatibility module providing the needed bindings.  In
that way a port could become easier to maintain and maybe such a
module could also be useful for other mit-scheme software.

Again, the amount of time I can spend on this is highly unpredictable... :(

Best regards,
Mikael

On Mon, Jan 21, 2013 at 6:58 PM, Andy Wingo <wingo@pobox.com> wrote:
> Hello Mikael,
>
> A pleasure to see you around!
>
> On Mon 22 Oct 2012 01:11, Mikael Djurfeldt <mikael@djurfeldt.com> writes:
>
>> When trying to use guile 2 for logic programming I discovered that the
>> slib interface is again broken (and has been for quite some time).
>
> I am very sorry that I did not see this thread before hacking on this
> recently.  Somehow over the past three or four months I just managed to
> drop everything and the inboxes filled without being filtered or drained
> in any way -- and to attack that I decided to just run through
> individual lists in order.  A strange strategy, but it is good for
> honing the "does something need to be done about this or can I drop it?"
> instinct.
>
> Anyway I picked up something in the user list about Slib, looked into
> it, and then decided to fix it, without having seen this mail --
> resulting in the recent patches to Slib CVS and Guile git.  I'm sorry to
> have stepped on your toes here.  In any case I didn't check it
> thoroughly, so surely there are issues yet to resolve.
>
>> The implementation of the interface has two sides.  One, the file
>> ice-9/slib.scm, is owned by Guile.  The other, slib/guile.init, is
>> owned by slib.  slib has such .init files for some common scheme
>> implementations but I early on noticed that that the guile.init file
>> is not really maintained.  I decided that it would be more robust if
>> slib.scm incorporated most of the interface so that it would be easy
>> to update it as Guile changed.  But of course slib also changed and at
>> some point others felt that guile.init should contain most of the
>> interface and the bulk of slib.scm was moved there.  As we have seen,
>> this didn't make things much better.
>
> Yes, in many ways I would like to have the interface in Guile.  However
> it seems that time has shown that it really wants to live in slib --
> probably because that's where people care most about slib.
>
> At least with Guile 2 we have managed to clean up many of the version
> dependent hacks, by just delegating to a fresh file for Guile 2.
>
> Anyway.  Perhaps I did the wrong thing in fixing it this way?  I would
> be very happy to commit anything you have.  Please take a look at both
> Slib and Guile from their version control systems, and the recent patch
> about `include'.  Aubrey seems quite responsive in dealing with patches,
> so if there is a change to make, I'm sure we can get it in.
>
>> *But*, the proper implementation of syntax-toplevel? requires
>> modification of psyntax.scm and adding it to the (system syntax)
>> module.
>
> Do you have a new patch for this one?
>
> Regards,
>
> Andy
> --
> http://wingolog.org/

Re: Fixing the slib mess

[Andy Wingo]

Howdy :)

On Mon 21 Jan 2013 19:09, Mikael Djurfeldt <mikael@djurfeldt.com> writes:

> Now, I'm looking into porting Gerald Sussman's scmutils to Guile-2.0.
> I'm aware of an older port by Daniel Gildea but I don't think that
> uses GOOPS:

Neat; GOOPS is definitely the way to go there ;-)

> I'm currently wondering if it could make sense to try to make an
> mit-scheme compatibility module providing the needed bindings.  In
> that way a port could become easier to maintain and maybe such a
> module could also be useful for other mit-scheme software.

Yes I think that can make a lot of sense.

Another thing to think about is this "guildhall" thing that folks are
building (especially Ian Price) -- seems that "guile update && guild
install foo" is starting to work.  A guildhall MIT scheme compatibility
package would be great; that would let us have a guildhall package for
scmutils as well.

Dunno, just thoughts for your Copious Free Time ;-)

Cheers!

Andy
-- 
http://wingolog.org/