bug#21944: Error on ordering of define-record-type and define-public in a module is unhelpful - possible improvement?

bug#21944: Error on ordering of define-record-type and define-public in a module is unhelpful - possible improvement?

[Koz Ross]

[-- Attachment #1: Type: text/plain, Size: 2189 bytes --]

I have the following file, called foo.scm:

(define-module (koz foo)
 #:use-module (srfi srfi-9))

(define-public (make-empty-bar)
  (make-bar #f))

(define-record-type <bar>
  (make-bar open)
  bar?
  (open bar-open set-bar-open!))

I then also have this script test-foo.scm in the same directory:

#!/usr/bin/guile \
-L .. -s
!#

(use-modules (koz foo))
(define corner-bar (make-empty-bar))
(display corner-bar)
(newline)

After chmodding and trying to run test-foo.scm (with autocompilation enabled), I get a pile of error messages. After some testing, I discovered that if the order of definitions in foo.scm is inverted (i.e. the define-record-type comes first), this problem does not occur and the script works fine. However, the errors received are extremely unhelpful:

Backtrace:
In ice-9/boot-9.scm:
 157: 8 [catch #t #<catch-closure 92abc0> ...]
In unknown file:
   ?: 7 [apply-smob/1 #<catch-closure 92abc0>]
In ice-9/boot-9.scm:
  63: 6 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 5 [eval # #]
In ice-9/boot-9.scm:
2401: 4 [save-module-excursion #<procedure 947880 at ice-9/boot-9.scm:4045:3 ()>]
4052: 3 [#<procedure 947880 at ice-9/boot-9.scm:4045:3 ()>]
In unknown file:
   ?: 2 [load-compiled/vm "/home/koz/.cache/guile/ccache/2.0-LE-8-2.0/home/koz/documents/programming/guile/koz/foo-test.scm.go"]
In /home/koz/documents/programming/guile/koz/./foo-test.scm:
   7: 1 [#<procedure d4a200 ()>]
In unknown file:
   ?: 0 [#<syntax-transformer make-empty-bar> #f 6 #f]

ERROR: In procedure #<syntax-transformer make-empty-bar>:
ERROR: Wrong type to apply: #<syntax-transformer make-empty-bar>

Would it be possible for the error message in this case to be a bit more helpful? Even better, would it be possible to not make this an issue when compiling?
-- 
Koz Ross <koz.ross@retro-freedom.nz>
www.retro-freedom.nz
If you aren't using GPG, you should be! https://emailselfdefense.fsf.org/en.
***
Please don't send me Word or PowerPoint attachments. See http://www.gnu.org/philosophy/no-word-attachments.html for why.
***
Proud member of the Open Wireless Movement. Find out more at https://openwireless.org/

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]

bug#21944: Error on ordering of define-record-type and define-public in a module is unhelpful - possible improvement?

[Andy Wingo]

On Tue 17 Nov 2015 09:27, Koz Ross <koz.ross@retro-freedom.nz> writes:

> I have the following file, called foo.scm:
>
> (define-module (koz foo)
>  #:use-module (srfi srfi-9))
>
> (define-public (make-empty-bar)
>   (make-bar #f))
>
> (define-record-type <bar>
>   (make-bar open)
>   bar?
>   (open bar-open set-bar-open!))

> Would it be possible for the error message in this case to be a bit
> more helpful? Even better, would it be possible to not make this an
> issue when compiling?

It would be possible to make the scope of make-bar be the whole file.
In theory it should work I guess, given this news entry from 2.0.1:

  ** `begin' expands macros in its body before other expressions

  This enables support for programs like the following:

      (begin
        (define even?
          (lambda (x)
            (or (= x 0) (odd? (- x 1)))))
        (define-syntax odd?
          (syntax-rules ()
            ((odd? x) (not (even? x)))))
        (even? 10))

And indeed if I try something at the REPL that uses `begin' I can't
reproduce this sort of error.  Hmmmm.  Maybe this rings a bell with
Mark.

In the mean-time I added a warning:

  wingo@clucks:~/src/guile$ meta/guile --fresh-auto-compile /tmp/foo.scm
  ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
  ;;;       or pass the --no-auto-compile argument to disable.
  ;;; compiling /tmp/foo.scm
  ;;; /tmp/foo.scm:4:2: warning: macro `make-bar' used before definition
  ;;; compiled /home/wingo/src/guile/cache/guile/ccache/2.2-LE-8-3.9/tmp/foo.scm.go
  Backtrace:
             5 (apply-smob/1 #<catch-closure fcac00>)
  In ice-9/boot-9.scm:
      704:2  4 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
  In ice-9/eval.scm:
      608:8  3 (_ #(#(#<directory (guile-user) fd1f30>)))
  In ice-9/boot-9.scm:
     2325:4  2 (save-module-excursion _)
    3829:12  1 (_)
  In unknown file:
             0 (_ #f)

  ERROR: ERROR: Wrong type to apply: #<syntax-transformer make-bar>

OK the error is terrible, but at least the warning tells you why the
later error is terrible.

I haven't been able to backport it to 2.0 yet though.  But maybe the
warning is useless if we can fix the issue.

Andy

bug#21944: Error on ordering of define-record-type and define-public in a module is unhelpful - possible improvement?

[Andy Wingo]

On Sun 26 Jun 2016 23:06, Andy Wingo <wingo@pobox.com> writes:

> On Tue 17 Nov 2015 09:27, Koz Ross <koz.ross@retro-freedom.nz> writes:
>
>> I have the following file, called foo.scm:
>>
>> (define-module (koz foo)
>>  #:use-module (srfi srfi-9))
>>
>> (define-public (make-empty-bar)
>>   (make-bar #f))
>>
>> (define-record-type <bar>
>>   (make-bar open)
>>   bar?
>>   (open bar-open set-bar-open!))
>
>> Would it be possible for the error message in this case to be a bit
>> more helpful? Even better, would it be possible to not make this an
>> issue when compiling?
>
> It would be possible to make the scope of make-bar be the whole file.
> In theory it should work I guess, given this news entry from 2.0.1:
>
>   ** `begin' expands macros in its body before other expressions

Apparently the reason this doesn't work in Guile right now is that the
compiler currently reads and compiles one Scheme expression at a time,
then stitches them together on the Tree-IL level.  Incidentally,
`primitive-load' works in the same way for the interpreter: it reads and
eval's single expressions in a loop.  We could change this to have Guile
read the whole file and pass it all to the expander at once, within a
`begin'.  This has some user-visible changes though:

  * if evaluating an expression throws an error, primitive-load doesn't
    read the following expressions and so doesn't detect syntax errors;
    try a file like this:

    (error "what")
    )

    With the interpreter (primitive-load) you will get the "what" error,
    not a syntax error.  (Yes the unclosed paren hurts my eyeballs but I
    wanted to demonstrate a syntax error.  Here's a matching paren:
    ")".)

  * Procedural macros won't be able to use bindings defined previously
    in the file unless they are eval-whenned.  Of course this already
    breaks in the compiler, but it succeeds in the interpreter.

Maybe now is a good time to do this though, in 2.2.  Ludovic, Mark:
thoughts?

Andy

bug#21944: Error on ordering of define-record-type and define-public in a module is unhelpful - possible improvement?

[Andy Wingo]

On Mon 27 Jun 2016 10:02, Andy Wingo <wingo@pobox.com> writes:

> Apparently the reason this doesn't work in Guile right now is that the
> compiler currently reads and compiles one Scheme expression at a time,
> then stitches them together on the Tree-IL level.  Incidentally,
> `primitive-load' works in the same way for the interpreter: it reads and
> eval's single expressions in a loop.  We could change this to have Guile
> read the whole file and pass it all to the expander at once, within a
> `begin'.  This has some user-visible changes though:
>
>   * if evaluating an expression throws an error, primitive-load doesn't
>     read the following expressions and so doesn't detect syntax errors;
>     try a file like this:
>
>     (error "what")
>     )
>
>     With the interpreter (primitive-load) you will get the "what" error,
>     not a syntax error.  (Yes the unclosed paren hurts my eyeballs but I
>     wanted to demonstrate a syntax error.  Here's a matching paren:
>     ")".)
>
>   * Procedural macros won't be able to use bindings defined previously
>     in the file unless they are eval-whenned.  Of course this already
>     breaks in the compiler, but it succeeds in the interpreter.

Another user-visible change: changes to read-options would not take
effect in the same places.

Andy

bug#21944: Error on ordering of define-record-type and define-public in a module is unhelpful - possible improvement?

[Mark H Weaver]

Andy Wingo <wingo@pobox.com> writes:

> On Sun 26 Jun 2016 23:06, Andy Wingo <wingo@pobox.com> writes:
>
>> On Tue 17 Nov 2015 09:27, Koz Ross <koz.ross@retro-freedom.nz> writes:
>>
>>> I have the following file, called foo.scm:
>>>
>>> (define-module (koz foo)
>>>  #:use-module (srfi srfi-9))
>>>
>>> (define-public (make-empty-bar)
>>>   (make-bar #f))
>>>
>>> (define-record-type <bar>
>>>   (make-bar open)
>>>   bar?
>>>   (open bar-open set-bar-open!))
>>
>>> Would it be possible for the error message in this case to be a bit
>>> more helpful? Even better, would it be possible to not make this an
>>> issue when compiling?
>>
>> It would be possible to make the scope of make-bar be the whole file.
>> In theory it should work I guess, given this news entry from 2.0.1:
>>
>>   ** `begin' expands macros in its body before other expressions
>
> Apparently the reason this doesn't work in Guile right now is that the
> compiler currently reads and compiles one Scheme expression at a time,
> then stitches them together on the Tree-IL level.  Incidentally,
> `primitive-load' works in the same way for the interpreter: it reads and
> eval's single expressions in a loop.  We could change this to have Guile
> read the whole file and pass it all to the expander at once, within a
> `begin'.  This has some user-visible changes though:
>
>   * if evaluating an expression throws an error, primitive-load doesn't
>     read the following expressions and so doesn't detect syntax errors;
>     try a file like this:
>
>     (error "what")
>     )
>
>     With the interpreter (primitive-load) you will get the "what" error,
>     not a syntax error.  (Yes the unclosed paren hurts my eyeballs but I
>     wanted to demonstrate a syntax error.  Here's a matching paren:
>     ")".)
>
>   * Procedural macros won't be able to use bindings defined previously
>     in the file unless they are eval-whenned.  Of course this already
>     breaks in the compiler, but it succeeds in the interpreter.

Another problem is that in several places, we assume that if a top-level
form calls 'set-current-module', the forms that follow in the file will
now be expanded within that new module.  This behavior is needed for
'define-module' to work properly, and it's also assumed in boot-9.scm,
psyntax-pp.scm, and maybe some other places.

      Mark

bug#21944: Error on ordering of define-record-type and define-public in a module is unhelpful - possible improvement?

[Andy Wingo]

On Mon 27 Jun 2016 18:10, Mark H Weaver <mhw@netris.org> writes:

>>   * if evaluating an expression throws an error, primitive-load doesn't
>>     read the following expressions and so doesn't detect syntax errors;
>>     try a file like this:
>>
>>     (error "what")
>>     )
>>
>>     With the interpreter (primitive-load) you will get the "what" error,
>>     not a syntax error.  (Yes the unclosed paren hurts my eyeballs but I
>>     wanted to demonstrate a syntax error.  Here's a matching paren:
>>     ")".)
>>
>>   * Procedural macros won't be able to use bindings defined previously
>>     in the file unless they are eval-whenned.  Of course this already
>>     breaks in the compiler, but it succeeds in the interpreter.
>
> Another problem is that in several places, we assume that if a top-level
> form calls 'set-current-module', the forms that follow in the file will
> now be expanded within that new module.  This behavior is needed for
> 'define-module' to work properly, and it's also assumed in boot-9.scm,
> psyntax-pp.scm, and maybe some other places.

I think I fixed this in a reasonable way in master; or, reasonable given
the historical mess that this all is anyway :) Your thoughts welcome
here.

If I did manage to fix that, then the remaining problems are the ones
that I mention, plus reader options which I mentioned in another mail.
I think probably reader options are the only significant issue.  WDYT?

Andy