guile 2.0.0 crash related to call-with-prompt/abort-to-prompt, & more

guile 2.0.0 crash related to call-with-prompt/abort-to-prompt, & more

[Wolfgang J Moeller]

Hi there,

while essentially all of my "old" guile works with 2.0.0,
I found a bug when testing CALL-WITH-PROMPT . Also some
more quibbles further below.

The major bug:

With self-compiled GUILE 2.0.0 (as of 2011-02-16) on OpenSuSE 11.1 -
both 32-bit and 64-bit - built/installed and then renamed "guile2",
I observe a segfault when done loading a particular compiled file.
No such error in interpreted mode.

Details:

moeller@linux3-$ uname -a
Linux linux3 2.6.27.56-0.1-pae #1 SMP 2010-12-01 16:57:58 +0100 i686 i686 i386 GNU/Linux
moeller@linux3-$ build-aux/config.guess
i686-pc-linux-gnu

moeller@louix-$ uname -a
Linux louix 2.6.27.56-0.1-default #1 SMP 2010-12-01 16:57:58 +0100 x86_64 x86_64 x86_64 GNU/Linux
moeller@louix-$ ./guile-2.0.0/build-aux/config.guess
x86_64-unknown-linux-gnu

Unfortunately, I did "make distclean", so no "config.status'.
Anyway, I installed on either machine via
	./configure && make; su -c 'make install'
(no options) the following packages:
    gc-7.1
    libffi-3.0.9
    libunistring-0.9.3
    guile-2.0.0

===============================================================================
Transcript of session (same on 64-bit)
===============================================================================
moeller@linux3-$ guile2 -q --no-auto-compile
GNU Guile 2.0.0
Copyright (C) 1995-2011 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load "/home/moeller/scm/g2crash.scm")
#t ; <-- #t
scheme@(guile-user)> ,cc /home/moeller/scm/g2crash.scm
scheme@(guile-user)> (load "/home/moeller/scm/g2crash.scm")
#t ; <-- #t
Segmentation fault
moeller@linux3-$
==============================================================================
File "/home/moeller/scm/g2crash.scm"
==============================================================================
;;; -*- scheme -*-
;;
;;; try reset & shift with GUILE V2 prompts
;;
;;
(define-syntax reset
  (syntax-rules ()
    ((_ . body)
     (call-with-prompt 'Reset
		       (lambda () . body)
		       (lambda (cont f) (f cont))))))
;;
(define-syntax shift
  (syntax-rules ()
    ((_ var . body)
     (abort-to-prompt 'Reset
		      (lambda (cont)
			((lambda (var) (reset . body))
			 (lambda vals (reset (apply cont vals)))))))))
;;
;;
(define (reset* thunk)
  (reset (thunk)))
;;
(define (shift* fc)
  (shift c (fc c)))
;;
;; ------------------------------- Tests
;;
;;(display (+ 10 (reset (+ 2 (shift k (+ 100 (k (k 3))))))))
;;(display " ; <-- 117")
;;(newline)
;;
;;(display (* 10 (reset (* 2 (shift g (* 5 (shift f (+ (f 1) 1))))))))
;;(display " ; <-- 60")
;;(newline)
;;
;;(display (let ((f (lambda (x) (shift k (k (k x))))))
;;	   (+ 1 (reset (+ 10 (f 100))))))
;;(display " ; <-- 121")
;;(newline)
;;
;;;; shift f1 tests that we implement shift rather than shift0
;;(display (reset
;;	  (let ((x (shift f
;;			  (shift f1 (f1 (cons 'a (f '())))))))
;;	    (shift g x))))
;;(display " ; <-- '(a)")
;;(newline)

(reset (display (let ((x 'abcde)) (eq? x ((shift* shift*) x)))))
(display " ; <-- #t")
(newline)

;;(let
;;    ((traverse
;;      (lambda (xs)
;;	(letrec ((visit
;;		  (lambda (xs)
;;		    (if (null? xs)
;;			'()
;;			(visit (shift*
;;				(lambda (k)
;;				  (cons (car xs) (k (cdr xs))))))))))
;;	  (reset*
;;	   (lambda ()
;;	     (visit xs)))))))
;;
;;  (display "Example by Olivier Danvy: ")
;;  (display (traverse '(1 2 3 4 5)))
;;  (display " ; <-- '(1 2 3 4 5)")
;;  (newline)
;;)

;;;*
===============================================================================
===============================================================================

Another *strange* mis-behaviour, discovered while running with "-q":
(load <filename>) accepts relative paths only after (activate-readline), or so ...

The three commands that I'm typing in the following session
traditionally make up my "~/.guile".

===============================================================================
Transcript of session
===============================================================================
moeller@linux3-$ guile2 -q --no-auto-compile
GNU Guile 2.0.0
Copyright (C) 1995-2011 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load "g2crash.scm")
ERROR: In procedure primitive-load-path:
ERROR: In procedure primitive-load-path: Unable to find file "./g2crash.scm" in load path

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ^D
scheme@(guile-user)> (use-modules (ice-9 r5rs))
scheme@(guile-user)> (load "g2crash.scm")
ERROR: In procedure primitive-load-path:
ERROR: In procedure primitive-load-path: Unable to find file "./g2crash.scm" in load path

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ^D
scheme@(guile-user)> (use-modules (ice-9 readline))
scheme@(guile-user)> (load "g2crash.scm")
ERROR: In procedure primitive-load-path:
ERROR: In procedure primitive-load-path: Unable to find file "./g2crash.scm" in load path

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ^D
scheme@(guile-user)> (activate-readline)
scheme@(guile-user)> (load "g2crash.scm")
#t ; <-- #t
scheme@(guile-user)>
moeller@linux3-$
===============================================================================
===============================================================================

Wishlist:

	(1) Please provide a means by which the debugger prompt (recursive REPL)
            can be turned off/on. Both a 'hook' (like COMMON-LISP:DEBUGGER-HOOK
	    plus COMMON-LISP:ABORT) or a REPL command would be OK with me.
	    I mis-type too often!
            [Anyway, I could not find such a means in 2.0.0].

	(2) Please provide some obvious "undefine" command, since UNINTERN is gone.
            I'd need it only interactively, so a REPL command would be fine.
            Primary use, of course, would be to remove syntax definitions -
            alternating between syntax-based and procedure-based code
	    ought not require a re-start of GUILE.
	    [I believe that the effect still can be had via un-obvious use
	    of 'deprecated' APIs].

Best regards,

Wolfgang J. Moeller, Tel. +49 551 47361, wjm<AT>heenes.com
37085 Goettingen, Germany | Disclaimer: No claim intended!
http://www.wjmoeller.de/ -+-------- http://www.heenes.com/

Re: guile 2.0.0 crash related to call-with-prompt/abort-to-prompt, & more

[Andy Wingo]

Hi Wolfgang,

Thank you for your detailed bug report.  It will take some time to mull
over all of it, but for the moment, I believe I have fixed this one:

On Wed 09 Mar 2011 11:53, Wolfgang J Moeller <wjm@heenes.com> writes:

> Another *strange* mis-behaviour, discovered while running with "-q":
> (load <filename>) accepts relative paths only after (activate-readline), or so ...

Yes, very strange, that; and related I think to the bold but tricky
load-is-a-macro trick.

Let me know if you can still reproduce this issue with current git.

Thanks,

Andy
-- 
http://wingolog.org/

Re: guile 2.0.0 crash related to call-with-prompt/abort-to-prompt, & more

[Andy Wingo]

Hi Wolfgang,

On Wed 09 Mar 2011 11:53, Wolfgang J Moeller <wjm@heenes.com> writes:

> I observe a segfault when done loading a particular compiled file.
> No such error in interpreted mode.

I'm working on this one still, and it's quite perplexing.

The example can be quite short:

    (define-syntax reset
      (syntax-rules ()
        ((_ . body)
         (call-with-prompt 'Reset
                           (lambda () . body)
                           (lambda (cont f) (f cont))))))
    (define-syntax shift
      (syntax-rules ()
        ((_ var . body)
         (abort-to-prompt 'Reset
                          (lambda (cont)
                            ((lambda (var) (reset . body))
                             (lambda vals (reset (apply cont vals)))))))))

    (define (shift* fc)
      (shift c (fc c)))

    (reset (display ((shift* shift*) 'foo)))

It seems to be that somehow this corrupts the wind list, unwinding one
dynamic frame too far, and thus causing errors later.

In any case, thanks again for this little chestnut :)

Andy
-- 
http://wingolog.org/

Re: guile 2.0.0 crash related to call-with-prompt/abort-to-prompt, & more

[Andy Wingo]

On Mon 14 Mar 2011 21:51, Andy Wingo <wingo@pobox.com> writes:

> Hi Wolfgang,
>
> On Wed 09 Mar 2011 11:53, Wolfgang J Moeller <wjm@heenes.com> writes:
>
>> I observe a segfault when done loading a particular compiled file.
>> No such error in interpreted mode.
>
> The example can be quite short:

Or shorter (in terms of code):

    (call-with-prompt
     'p1
     (lambda ()
       (display ((abort-to-prompt
                  'p1
                  (lambda (c1)
                    (call-with-prompt
                     'p2
                     (lambda ()
                       (abort-to-prompt
                        'p2
                        (lambda (c2)
                          (c1 (lambda (x) (c2 x))))))
                     (lambda (c2 fc2)
                       (fc2 c2)))))
                 'foo)))
     (lambda (c1 fc1)
       (fc1 c1)))

Andy
-- 
http://wingolog.org/

Re: guile 2.0.0 crash related to call-with-prompt/abort-to-prompt, & more

[Andy Wingo]

Hello Wolfgang,

On Wed 09 Mar 2011 11:53, Wolfgang J Moeller <wjm@heenes.com> writes:

> With self-compiled GUILE 2.0.0 (as of 2011-02-16) on OpenSuSE 11.1 -
> both 32-bit and 64-bit - built/installed and then renamed "guile2",
> I observe a segfault when done loading a particular compiled file.
> No such error in interpreted mode.

I have fixed this in git, I believe.

Thanks again for the test case!  I'll reply regarding your other
suggestions later.

Regards,

Andy
-- 
http://wingolog.org/

shift and reset in ice-9 control

[Andy Wingo]

Hello again Wolfgang,

On Wed 09 Mar 2011 11:53, Wolfgang J Moeller <wjm@heenes.com> writes:

> ;;; -*- scheme -*-
> ;;
> ;;; try reset & shift with GUILE V2 prompts
> ;;
> ;;
> (define-syntax reset
>   (syntax-rules ()
>     ((_ . body)
>      (call-with-prompt 'Reset
> 		       (lambda () . body)
> 		       (lambda (cont f) (f cont))))))
> ;;
> (define-syntax shift
>   (syntax-rules ()
>     ((_ var . body)
>      (abort-to-prompt 'Reset
> 		      (lambda (cont)
> 			((lambda (var) (reset . body))
> 			 (lambda vals (reset (apply cont vals)))))))))
> ;;
> ;;
> (define (reset* thunk)
>   (reset (thunk)))
> ;;
> (define (shift* fc)
>   (shift c (fc c)))

Did you write these yourself?  May we include them in (ice-9 control)?

If you have additional useful tests, we are happy to include them as
well.  Probably the easiest thing would be to mail them to the list with
a permissive copyright header.

Cheers,

Andy
-- 
http://wingolog.org/

Re: shift and reset in ice-9 control

[Wolfgang J Moeller]

Hello again,

On Thu, 17 Mar 2011, Andy Wingo wrote:
>[...]
> On Wed 09 Mar 2011 11:53, Wolfgang J Moeller <wjm@heenes.com> writes:
>[...]
> > (define-syntax reset
> >   (syntax-rules ()
> >     ((_ . body)
> >      (call-with-prompt 'Reset
> > 		       (lambda () . body)
> > 		       (lambda (cont f) (f cont))))))
> > ;;
> > (define-syntax shift
> >   (syntax-rules ()
> >     ((_ var . body)
> >      (abort-to-prompt 'Reset
> > 		      (lambda (cont)
> > 			((lambda (var) (reset . body))
> > 			 (lambda vals (reset (apply cont vals)))))))))
>[...]
> Did you write these yourself?  May we include them in (ice-9 control)?

Yes and yes.

However, I don't like it anymore, since I seem to have learned
that this "direct implementation" doesn't at all play well
with the use of call/cc within the <body>s, while the call/cc-based
implementations of shift/reset appear to do so.

I've yet to find out why the latter work "better", and if there's
a remedy possible using something like with-continuation-barrier ...

> If you have additional useful tests, we are happy to include them as
> well.  Probably the easiest thing would be to mail them to the list with
> a permissive copyright header.

The tests that I had included come from http://okmij.org/ftp/,
as does almost all I know about "delimited continations"
(I first learned about them only a few weeks before you
released GUILE V2). There are a few more tests at that site.

Best regards,

Wolfgang J. Moeller, Tel. +49 551 47361, wjm<AT>heenes.com
37085 Goettingen, Germany | Disclaimer: No claim intended!
http://www.wjmoeller.de/ -+-------- http://www.heenes.com/

Re: shift and reset in ice-9 control

[Andy Wingo]

On Mon 21 Mar 2011 02:31, Wolfgang J Moeller <wjm@heenes.com> writes:

> Hello again,
>
> On Thu, 17 Mar 2011, Andy Wingo wrote:
>>[...]
>> On Wed 09 Mar 2011 11:53, Wolfgang J Moeller <wjm@heenes.com> writes:
>>[...]
>> > (define-syntax reset
>> > (define-syntax shift
>> Did you write these yourself?  May we include them in (ice-9 control)?
>
> Yes and yes.

Sweet, thanks.

> However, I don't like it anymore, since I seem to have learned
> that this "direct implementation" doesn't at all play well
> with the use of call/cc within the <body>s, while the call/cc-based
> implementations of shift/reset appear to do so.
>
> I've yet to find out why the latter work "better", and if there's
> a remedy possible using something like with-continuation-barrier ...

Can you give some examples?  In Guile things should work fine, AIUI.

In the future (i.e., 2.2), I would like to change continuations to
capture just the Scheme stack, and optionally delimit their extent to a
prompt.  I recommend Flatt et al's ICFP 2007 paper for a discussion.

Regards,

Andy
-- 
http://wingolog.org/

Re: shift and reset in ice-9 control

[Wolfgang J Moeller]

On Wed, 30 Mar 2011, Andy Wingo wrote:
> On Mon 21 Mar 2011 02:31, Wolfgang J Moeller <wjm@heenes.com> writes:
>[...]
> > However, I don't like it anymore, since I seem to have learned
> > that this "direct implementation" doesn't at all play well
> > with the use of call/cc within the <body>s, while the call/cc-based
> > implementations of shift/reset appear to do so.
> >
> > I've yet to find out why the latter work "better", and if there's
> > a remedy possible using something like with-continuation-barrier ...
>
> Can you give some examples?  In Guile things should work fine, AIUI.
>
> In the future (i.e., 2.2), I would like to change continuations to
> capture just the Scheme stack, and optionally delimit their extent to a
> prompt.  I recommend Flatt et al's ICFP 2007 paper for a discussion.

Interesting, but only in order to re-inspect "my" DYNAMIC-WIND,
whose interactions with CALL-WITH-PROMPT and ABORT-TO-PROMPT
I've not even tested yet.

No, I'm not concerned with the implementation of the primitives,
but with an attempt of mine to express a co-routine linkage using
reset/shift (would you know of an example in the literature?).

A stripped-down example goes like this:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(define (f1 ret)
  (let ((c #f))
    (do ((n 0 (+ n 1)))
	((> n 2) #f)
      (display n)	; at start of new line
      (case n
	((0) (ret n))
	((1) (ret (call/cc (lambda (k)
			     (set! c k)
			     -1))))
	(else (c n))))))
;;
(let ((dc #f)
      (saymax 5))
  ;;
  (define (return-saving-dc v)
    (shift d (begin
	       (set! dc d)
	       v)))
  ;;
  (define (say . vs)
    (set! saymax (- saymax 1))
    (if (negative? saymax) (error "saymax < 0"))
    (for-each display vs) (newline))
  ;;
  (say "#0: " (reset (f1 return-saving-dc)))
  ;;
  (say "#1: " (dc "ignored.1"))
  (say "#2: " (dc "ignored.2"))
  (say "#3: " (dc "ignored.3"))
)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Think of (f1) as a sort-of co-routine and of (return-saving-dc)
as my best attempt to save co-routine state using reset/shift only.

Using the "direct" implementaion of reset/shift (mine seems to agree
with Flatt et.al.), it goes into a loop since (dc "ignored.2")
makes (f1) restore the full continuation that makes (dc "ignored.1")
return one more time - "dc" isn't so "delimited" after all.

Remarkably, the example works out differently (no loop) with either
call/cc-based implementation of reset/shift that I know of (one using
a "meta-continuation", the other a continuation stack) - as I understand it,
this comes from the delimited continuations always returning via another
full continuation that was captured at invocation time.

Maybe it's a "discovery" that the different implementations of reset/shift
do have observably different outcomes ...

And because I had first played with call/cc-based reset/shift,
I had the impression that it's indeed possible to do co-routine linkage
using these primitives w/o requiring call/cc. Then there came GUILE V2
and made me try out the "direct" way, plus I (incidentally) had call/cc
(only for _local_ transfer of control, just like in the example)
in one of my co-routines ... that's why I (at the moment) dislike
the "direct" implementation.

I haven't yet found a place where adding (with-continuation barrier)
[as in GUILE V2.0.0, at least] could help me; next thing was to think hard
about the potential of (dynamic-wind).

===

Some confusing observations about a "borderline" case
(evaluation of (define) determines "definition"-property
of the next statement) at GUILE V2's toplevel:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
  (begin
    (define dummy1
      (eval
       ;;
       '(defmacro macro-case clauses
	  (let lp ((cs clauses))
	    (if (null? cs)
		'(define dummy2 #f)
		(if (or (and (symbol? (caar cs))
			     (eq? (caar cs) 'else))
			(eval (caar cs) (interaction-environment)))
		    (cons 'begin (cdar cs))
		    (lp (cdr cs))))))
       ;;
       (interaction-environment)))
    )
  ;;
  (macro-case
   (#f
    (define something #t)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(That's just the traditional _structure_ of my prelude,
intended to have nothing but definitions at the toplevel).

Depending upon what I do with this snippet (auto-compile, load vs. include),
it sometimes gets accepted by GUILE V2.0.0:

		auto,include	auto,load	noauto,include	noauto,load
---------------------------------------------------------------------------
@toplevel	Error		Error		Error		OK
surroundes by
BEGIN	   	Error		Error		Error		Error
EVAL-WHEN	OK		OK		OK		Error

To add to the weirdness, removing the (supposedly redundant) BEGIN
around (DEFINE dummy1) reverses most of the outcomes ...

Not a serious problem - just confusing!

===

Best regards,

Wolfgang J. Moeller, Tel. +49 551 47361, wjm<AT>heenes.com
37085 Goettingen, Germany | Disclaimer: No claim intended!
http://www.wjmoeller.de/ -+-------- http://www.heenes.com/

Re: shift and reset in ice-9 control

[Andy Wingo]

Hi Wolfgang,

Thanks for your note.

On Sat 02 Apr 2011 16:40, Wolfgang J Moeller <wjm@heenes.com> writes:

> Using the "direct" implementaion of reset/shift (mine seems to agree
> with Flatt et.al.), it goes into a loop since (dc "ignored.2")
> makes (f1) restore the full continuation that makes (dc "ignored.1")
> return one more time - "dc" isn't so "delimited" after all.

Have you tried this on Racket?  I would expect that it would do the
right thing.  The difference, AFAIK, is that a prompt delimits full
continuations there as well.  That's why they call them "composable"
continuations instead of "delimited" continuations: because either kind
can be delimited; the real question is what properties do they have.

I think we're going to switch to do that in Guile 2.2 as well.

>   (begin
>     (define dummy1
>       (eval
>        ;;
>        '(defmacro macro-case clauses
[...]
>        (interaction-environment)))
>     )
>   ;;
>   (macro-case
>    (#f
>     (define something #t)))

> Depending upon what I do with this snippet (auto-compile, load vs. include),
> it sometimes gets accepted by GUILE V2.0.0:
>
> 		auto,include	auto,load	noauto,include	noauto,load
> ---------------------------------------------------------------------------
> @toplevel	Error		Error		Error		OK
> surroundes by
> BEGIN	   	Error		Error		Error		Error
> EVAL-WHEN	OK		OK		OK		Error

Indeed.  If you need to evaluate previous expressions in order to expand
later ones, you need eval-when.  That goes for forms within your
prelude, and for the "users" of your prelude; Guile doesn't know that
when it sees a `(load "prelude.scm")' that prelude.scm will define
syntax definitions, so it won't pull in those definitions at
compile-time, without an eval-when wrapping the load...

I don't understand the noauto,load case there though.  Were some of your
files compiled and some not?  Hmm...

Regards,

Andy
-- 
http://wingolog.org/

Re: shift and reset in ice-9 control

[Wolfgang J Moeller]

On Tue, 12 Apr 2011, Andy Wingo wrote:

>[...]
> Have you tried this on Racket?  I would expect that it would do the
> right thing.  The difference, AFAIK, is that a prompt delimits full
> continuations there as well.  That's why they call them "composable"
> continuations instead of "delimited" continuations: because either kind
> can be delimited; the real question is what properties do they have.
>
> I think we're going to switch to do that in Guile 2.2 as well.

Careful! One might easily come up with a reason to use call/cc in order
to leave/reenter a "composable" continuation - after all, it's supposed
to behave just like another procedure, and for tasks like "backtracking",
you do want procedures e.g. to return multiple times.

It seemed to me that the original specification of shift/reset
didn't describe the consequences of transfer of control (via call/cc)
"across" reset - maybe because it wasn't targeted specifically at Scheme.
That's why I had doubts that my co-routine problem could be solved.
Now that it is, I've no more problems with the current implementation.

Anyway, inhibiting some uses of call/cc would be quite non-scheme-y ...

>[...]
> > Depending upon what I do with this snippet (auto-compile, load vs. include),
> > it sometimes gets accepted by GUILE V2.0.0:
> >
> > 		auto,include	auto,load	noauto,include	noauto,load
> > ---------------------------------------------------------------------------
> > @toplevel	Error		Error		Error		OK
> > surrounded by
> > BEGIN	Error		Error		Error		Error
> > EVAL-WHEN	OK		OK		OK		Error
>
> Indeed.  If you need to evaluate previous expressions in order to expand
> later ones, you need eval-when.  That goes for forms within your
> prelude, and for the "users" of your prelude; Guile doesn't know that
> when it sees a `(load "prelude.scm")' that prelude.scm will define
> syntax definitions, so it won't pull in those definitions at
> compile-time, without an eval-when wrapping the load...
>
> I don't understand the noauto,load case there though.  Were some of your
> files compiled and some not?  Hmm...

By "noauto,load" I meant (load "prelude.scm") into "guile --no-auto-compile",
with care taken (as in all of the above ",load" cases ;-) that _no_ compiled
[& cached] files were in place. It's exactly this column that's confusing me
(still with Guile V2.0.0).

Btw, meanwhile I've found that wrapping the second (macro-using) part
in another (eval ...) - thus terminating the top-level definition context -
works just fine w/o EVAL-WHEN, and so my "new" prelude now can be LOADed
into GUILE V2 independent of auto-compilation, as well as into GUILE V1, etc.


Best regards,

Wolfgang J. Moeller, Tel. +49 551 47361, wjm<AT>heenes.com
37085 Goettingen, Germany | Disclaimer: No claim intended!
http://www.wjmoeller.de/ -+-------- http://www.heenes.com/