unhandled constant?

unhandled constant?

[Han-Wen Nienhuys]

Some of the lilypond Scheme files do the following:


(define decl '())
(define (make-var n v) (list "var" n v))
(defmacro define-session (name value)
  (define (inner n v)
    (set! decl
        (cons
         (make-var n v)
         decl))
    )
  `(,inner ',name ,value))
(define-session foo 1)
(display decl)
(newline)

In GUILE 2.2, this yields

;;; WARNING: compilation of /home/hanwen/vc/lilypond/q.scm failed:
;;; unhandled constant #<procedure inner (a b)>

What does this error message mean, and what should I do to address the problem?
-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[Ricardo Wurmus]

Han-Wen Nienhuys <hanwenn@gmail.com> writes:

> Some of the lilypond Scheme files do the following:
>
>
> (define decl '())
> (define (make-var n v) (list "var" n v))
> (defmacro define-session (name value)
>   (define (inner n v)
>     (set! decl
>         (cons
>          (make-var n v)
>          decl))
>     )
>   `(,inner ',name ,value))
> (define-session foo 1)
> (display decl)
> (newline)
>
> In GUILE 2.2, this yields
>
> ;;; WARNING: compilation of /home/hanwen/vc/lilypond/q.scm failed:
> ;;; unhandled constant #<procedure inner (a b)>
>
> What does this error message mean, and what should I do to address the problem?

Would it be feasible to use define-syntax-rule here?

--8<---------------cut here---------------start------------->8---
(define decl '())
(define (make-var n v) (list "var" n v))
(define-syntax-rule (define-session name value)
  (let ((inner (lambda (n v)
                 (set! decl
                       (cons
                        (make-var n v)
                        decl)))))
    (inner 'name value)))
(define-session foo 1)
(display decl)
(newline)
--8<---------------cut here---------------end--------------->8---


-- 
Ricardo

Re: unhandled constant?

[Han-Wen Nienhuys]

[guile1.8]$ grep -ir define-syntax-rule .

(empty)

I  need something that works in GUILE 1.8 too.

this is what I got from David Kastrup:

>Got any comments about macros being sooo yesterday compared to syntax
forms?  Syntax forms actually don't work in LilyPond: there was an
incompatibility because the 1.8 implementation will balk if some symbol
used in syntax forms already has a definition, and we have that.  I
forgot the exact symbol at fault.  I think it was some music function
name.

Would there be a way that we can use our source code unchanged with GUILE 2.2?

Can you explain why I get this error message?

On Wed, Jan 29, 2020 at 4:06 PM Ricardo Wurmus <rekado@elephly.net> wrote:
>
>
> Han-Wen Nienhuys <hanwenn@gmail.com> writes:
>
> > Some of the lilypond Scheme files do the following:
> >
> >
> > (define decl '())
> > (define (make-var n v) (list "var" n v))
> > (defmacro define-session (name value)
> >   (define (inner n v)
> >     (set! decl
> >         (cons
> >          (make-var n v)
> >          decl))
> >     )
> >   `(,inner ',name ,value))
> > (define-session foo 1)
> > (display decl)
> > (newline)
> >
> > In GUILE 2.2, this yields
> >
> > ;;; WARNING: compilation of /home/hanwen/vc/lilypond/q.scm failed:
> > ;;; unhandled constant #<procedure inner (a b)>
> >
> > What does this error message mean, and what should I do to address the problem?
>
> Would it be feasible to use define-syntax-rule here?
>
> --8<---------------cut here---------------start------------->8---
> (define decl '())
> (define (make-var n v) (list "var" n v))
> (define-syntax-rule (define-session name value)
>   (let ((inner (lambda (n v)
>                  (set! decl
>                        (cons
>                         (make-var n v)
>                         decl)))))
>     (inner 'name value)))
> (define-session foo 1)
> (display decl)
> (newline)
> --8<---------------cut here---------------end--------------->8---
>
>
> --
> Ricardo



-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[Han-Wen Nienhuys]

On Thu, Jan 30, 2020 at 9:05 AM Han-Wen Nienhuys <hanwenn@gmail.com> wrote:
>
> [guile1.8]$ grep -ir define-syntax-rule .
>
> (empty)
>
> I  need something that works in GUILE 1.8 too.
>
> this is what I got from David Kastrup:
>
> >Got any comments about macros being sooo yesterday compared to syntax
> forms?  Syntax forms actually don't work in LilyPond: there was an
> incompatibility because the 1.8 implementation will balk if some symbol
> used in syntax forms already has a definition, and we have that.  I
> forgot the exact symbol at fault.  I think it was some music function
> name.
>
> Would there be a way that we can use our source code unchanged with GUILE 2.2?
>
> Can you explain why I get this error message?

Also, how is it possible that, when disabling auto-compilation, the
whole thing works?

$ GUILE_AUTO_COMPILE=0 guile2.2 q.scm
((var foo 1))

is there an evaluator in GUILE that is separate from the bytecode VM,
and if so, is this evaluator guaranteed to be supported in upcoming
versions of GUILE? Does the evalation without auto-compilation benefit
from JIT treatment?

If I want to explore this myself, how do I hack on GUILE itself?
Compiling GUILE from scratch takes more than an hour for me. I assume
there must be a faster way to experiment, but what is that? It looks
like the HACKING file is out of date.

-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[Ricardo Wurmus]

Han-Wen Nienhuys <hanwenn@gmail.com> writes:

> [guile1.8]$ grep -ir define-syntax-rule .
>
> (empty)
>
> I  need something that works in GUILE 1.8 too.

I suppose you could use (cond-expand (guile-2.2 …) …) to only use this
with Guile 2 and fall back to the current code with Guile 1.8.

> Would there be a way that we can use our source code unchanged with GUILE 2.2?
>
> Can you explain why I get this error message?

I don’t know the answer.

-- 
Ricardo

Re: unhandled constant?

[Jan Nieuwenhuizen]

Han-Wen Nienhuys writes:

> [guile1.8]$ grep -ir define-syntax-rule .
>
> (empty)
>
> I  need something that works in GUILE 1.8 too.

Guile 1.8 has syntax-rules after you include (ice-9 syncase).
(define-syntax-rule foo ...) is just syntactic sugar for (define-syntax
foo (syntax-rules () ...); see the manual.

Combined with cond-expand, maybe that could work?

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: unhandled constant?

[Linus Björnstam]

Guile1.8's macros are run-time macros: they are executed directly and not transformed to output code that is then compiled. That is the reason why your code works: to newer guiles the (inner ...) is only available at expansion time. The macro output is trying to call code that does not exist at runtime!

For this to be working code the (inner ...) function needs to be available in the macro expansion. I didn't read through exactly what you are trying to do, but try outputting a let:

`(let ((inner (lambda (n v) (set ! ...))))
  (inner ,name ,value))

I doubt you can make the old code work in newer guiles, since I doubt any scheme is a s lax about expansion time and macro time separation.
-- 
  Linus Björnstam

On Wed, 29 Jan 2020, at 00:08, Han-Wen Nienhuys wrote:
> Some of the lilypond Scheme files do the following:
> 
> 
> (define decl '())
> (define (make-var n v) (list "var" n v))
> (defmacro define-session (name value)
>   (define (inner n v)
>     (set! decl
>         (cons
>          (make-var n v)
>          decl))
>     )
>   `(,inner ',name ,value))
> (define-session foo 1)
> (display decl)
> (newline)
> 
> In GUILE 2.2, this yields
> 
> ;;; WARNING: compilation of /home/hanwen/vc/lilypond/q.scm failed:
> ;;; unhandled constant #<procedure inner (a b)>
> 
> What does this error message mean, and what should I do to address the problem?
> -- 
> Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen
> 
>

Re: unhandled constant?

[Han-Wen Nienhuys]

On Fri, Jan 31, 2020 at 3:58 PM Linus Björnstam
<linus.bjornstam@veryfast.biz> wrote:
>
> Guile1.8's macros are run-time macros: they are executed directly and not transformed to output code that is then compiled. That is the reason why your code works: to newer guiles the (inner ...) is only available at expansion time. The macro output is trying to call code that does not exist at runtime!
>
> For this to be working code the (inner ...) function needs to be available in the macro expansion. I didn't read through exactly what you are trying to do, but try outputting a let:
>
> `(let ((inner (lambda (n v) (set ! ...))))
>   (inner ,name ,value))
>
> I doubt you can make the old code work in newer guiles, since I doubt any scheme is a s lax about expansion time and macro time separation.

Thanks for the explanation. This makes sense.

Is it possible to attach doc strings to (define-syntax .. )
declarations, and if so, where do they go?

-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[Han-Wen Nienhuys]

On Fri, Jan 31, 2020 at 3:58 PM Linus Björnstam
<linus.bjornstam@veryfast.biz> wrote:
>
> Guile1.8's macros are run-time macros: they are executed directly and not transformed to output code that is then compiled. That is the reason why your code works: to newer guiles the (inner ...) is only available at expansion time. The macro output is trying to call code that does not exist at runtime!

When is the code executed?  If have complex set of macros to define a
special type of functions (so called markup commands).  Some of these
refer to other markup commands through a macro.

What I can observe that some of the functions involved are not called
during the compilation, but others are.

In particular, the function that registers a markup command using something like

  (module-define! (current-module)
      (string->symbol (format #f "~a-markup" name))   defn))

but this function is not called during the compile

There is a convenience macro that is called within some function
bodies, that does get called. Unfortunately, the latter convenience
macro is expanded and then executed; the execution tries to then do

   (module-ref (current-module)
      (string->symbol (format #f "~a-markup" name)

which fails.



> For this to be working code the (inner ...) function needs to be available in the macro expansion. I didn't read through exactly what you are trying to do, but try outputting a let:
>
> `(let ((inner (lambda (n v) (set ! ...))))
>   (inner ,name ,value))
>
> I doubt you can make the old code work in newer guiles, since I doubt any scheme is a s lax about expansion time and macro time separation.
> --
>   Linus Björnstam
>
> On Wed, 29 Jan 2020, at 00:08, Han-Wen Nienhuys wrote:
> > Some of the lilypond Scheme files do the following:
> >
> >
> > (define decl '())
> > (define (make-var n v) (list "var" n v))
> > (defmacro define-session (name value)
> >   (define (inner n v)
> >     (set! decl
> >         (cons
> >          (make-var n v)
> >          decl))
> >     )
> >   `(,inner ',name ,value))
> > (define-session foo 1)
> > (display decl)
> > (newline)
> >
> > In GUILE 2.2, this yields
> >
> > ;;; WARNING: compilation of /home/hanwen/vc/lilypond/q.scm failed:
> > ;;; unhandled constant #<procedure inner (a b)>
> >
> > What does this error message mean, and what should I do to address the problem?
> > --
> > Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen
> >
> >



-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[Linus Björnstam]

I'm not very familiar with how guile 1.8 works since my first guile version was 2.2, but I remember hearing Andy talk about how macro expansion was done at runtime, due to guile not being compiled.

I just put one and one together and figured that the reason the macro you posted works in 1.8 must be because of what basically rounds down to no separation between runtime and expand time.

Defmacro is expanded to a regular syntax case macro that first strips all syntax information and passes it to the defmacro body and the re-introduces the result. Regarding your other question: any literal string in the defmacro body is moved to the correct define-syntax placement.

I don't really understand your question. With defmacro and syntax-case you can run arbitrary code. If you just output code that does module-define! that won't be run until runtime, and thus you cannot depend on the result of that module-define! during expansion. You can however wrap it in an eval-when to solve that issue. That allows you to specify when code gets run. With module-define! I personally find it all a bit icky, but I usually stay as far away from phasing as I can :)



-- 
  Linus Björnstam

On Fri, 31 Jan 2020, at 18:50, Han-Wen Nienhuys wrote:
> On Fri, Jan 31, 2020 at 3:58 PM Linus Björnstam
> <linus.bjornstam@veryfast.biz> wrote:
> >
> > Guile1.8's macros are run-time macros: they are executed directly and not transformed to output code that is then compiled. That is the reason why your code works: to newer guiles the (inner ...) is only available at expansion time. The macro output is trying to call code that does not exist at runtime!
> 
> When is the code executed?  If have complex set of macros to define a
> special type of functions (so called markup commands).  Some of these
> refer to other markup commands through a macro.
> 
> What I can observe that some of the functions involved are not called
> during the compilation, but others are.
> 
> In particular, the function that registers a markup command using something like
> 
>   (module-define! (current-module)
>       (string->symbol (format #f "~a-markup" name))   defn))
> 
> but this function is not called during the compile
> 
> There is a convenience macro that is called within some function
> bodies, that does get called. Unfortunately, the latter convenience
> macro is expanded and then executed; the execution tries to then do
> 
>    (module-ref (current-module)
>       (string->symbol (format #f "~a-markup" name)
> 
> which fails.
> 
> 
> 
> > For this to be working code the (inner ...) function needs to be available in the macro expansion. I didn't read through exactly what you are trying to do, but try outputting a let:
> >
> > `(let ((inner (lambda (n v) (set ! ...))))
> >   (inner ,name ,value))
> >
> > I doubt you can make the old code work in newer guiles, since I doubt any scheme is a s lax about expansion time and macro time separation.
> > --
> >   Linus Björnstam
> >
> > On Wed, 29 Jan 2020, at 00:08, Han-Wen Nienhuys wrote:
> > > Some of the lilypond Scheme files do the following:
> > >
> > >
> > > (define decl '())
> > > (define (make-var n v) (list "var" n v))
> > > (defmacro define-session (name value)
> > >   (define (inner n v)
> > >     (set! decl
> > >         (cons
> > >          (make-var n v)
> > >          decl))
> > >     )
> > >   `(,inner ',name ,value))
> > > (define-session foo 1)
> > > (display decl)
> > > (newline)
> > >
> > > In GUILE 2.2, this yields
> > >
> > > ;;; WARNING: compilation of /home/hanwen/vc/lilypond/q.scm failed:
> > > ;;; unhandled constant #<procedure inner (a b)>
> > >
> > > What does this error message mean, and what should I do to address the problem?
> > > --
> > > Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen
> > >
> > >
> 
> 
> 
> -- 
> Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen
>

Re: unhandled constant?

[Han-Wen Nienhuys]

On Fri, Jan 31, 2020 at 7:20 PM Linus Björnstam
<linus.bjornstam@veryfast.biz> wrote:
> I don't really understand your question. With defmacro and syntax-case you can run arbitrary code. If you just output code that does module-define! that won't be run until runtime, and thus you cannot depend on the result of that module-define! during expansion. You can however wrap it in an eval-when to solve that issue. That allows you to specify when code gets run. With module-define! I personally find it all a bit icky, but I usually stay as far away from phasing as I can :)

 eval-when looks like it might be a solution to the puzzle , but
honestly, the doc at

https://www.gnu.org/software/guile/manual/html_node/Eval-When.html

has me mystified.  When I run the example through guile 2.2 and
display *compilation-date*,

I get a different answer each time. Shouldn't it be a fixed timestamp
(of when the compile happened?)

[hanwen@localhost lilypond]$ guile2.2 e.scm
;;; note: source file /home/hanwen/vc/lilypond/e.scm
;;;       newer than compiled
/home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/e.scm.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/hanwen/vc/lilypond/e.scm
;;; compiled /home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/e.scm.go
Fri Jan 31 20:15:57+0100 2020
[hanwen@localhost lilypond]$ guile2.2 e.scm
Fri Jan 31 20:15:58+0100 2020
[hanwen@localhost lilypond]$ guile2.2 e.scm
Fri Jan 31 20:16:00+0100 2020

-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[Linus Björnstam]

On Fri, 31 Jan 2020, at 20:17, Han-Wen Nienhuys wrote:
> 
>  eval-when looks like it might be a solution to the puzzle , but
> honestly, the doc at
> 
> https://www.gnu.org/software/guile/manual/html_node/Eval-When.html
> 
> has me mystified.  When I run the example through guile 2.2 and
> display *compilation-date*,
> 
> I get a different answer each time. Shouldn't it be a fixed timestamp
> (of when the compile happened?)

Yup. At least from my understanding of the documentation, but whenever I try to do eval-when I become sad so I have just decided that I am too dumb for it. Try asking in the irc channel.

Re: unhandled constant?

[Linus Björnstam]

Read the docs. That seems to be a documentation bug. Try fiddling with the arguments to eval when and see if you can make it work.

-- 
  Linus Björnstam

On Fri, 31 Jan 2020, at 20:17, Han-Wen Nienhuys wrote:
> On Fri, Jan 31, 2020 at 7:20 PM Linus Björnstam
> <linus.bjornstam@veryfast.biz> wrote:
> > I don't really understand your question. With defmacro and syntax-case you can run arbitrary code. If you just output code that does module-define! that won't be run until runtime, and thus you cannot depend on the result of that module-define! during expansion. You can however wrap it in an eval-when to solve that issue. That allows you to specify when code gets run. With module-define! I personally find it all a bit icky, but I usually stay as far away from phasing as I can :)
> 
>  eval-when looks like it might be a solution to the puzzle , but
> honestly, the doc at
> 
> https://www.gnu.org/software/guile/manual/html_node/Eval-When.html
> 
> has me mystified.  When I run the example through guile 2.2 and
> display *compilation-date*,
> 
> I get a different answer each time. Shouldn't it be a fixed timestamp
> (of when the compile happened?)
> 
> [hanwen@localhost lilypond]$ guile2.2 e.scm
> ;;; note: source file /home/hanwen/vc/lilypond/e.scm
> ;;;       newer than compiled
> /home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/e.scm.go
> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;       or pass the --no-auto-compile argument to disable.
> ;;; compiling /home/hanwen/vc/lilypond/e.scm
> ;;; compiled 
> /home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/e.scm.go
> Fri Jan 31 20:15:57+0100 2020
> [hanwen@localhost lilypond]$ guile2.2 e.scm
> Fri Jan 31 20:15:58+0100 2020
> [hanwen@localhost lilypond]$ guile2.2 e.scm
> Fri Jan 31 20:16:00+0100 2020
> 
> -- 
> Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen
>

Re: unhandled constant?

[Han-Wen Nienhuys]

Here is an example that shows better how things work, and what might
be the cause of my problems. Is it right that programmatically set
contents of "current-module" are not serialized into the compiled
file?

   (define (run-at-compile-time cmd)
     (module-define! (current-module) (string->symbol cmd) #t)
     (format (current-error-port) "I-am-called-at-compile-time ~a\n" cmd))

  (define (runtime-call cmd)
    (format (current-error-port) "I-am-called-at-runtime ~a\n" cmd)
    (format (current-error-port) "val ~a\n"
            (module-ref (current-module) (string->symbol cmd))))

  (defmacro foo (cmd . rest)
  (run-at-compile-time cmd)
  `(runtime-call ,cmd))

  (foo "xy")

$ guile1.8 ew.scm
I-am-called-at-compile-time xy
I-am-called-at-runtime xy
val #t

this is compatible with 2.2 without compilation,

$ GUILE_AUTO_COMPILE=0 guile2.2 ew.scm
I-am-called-at-compile-time xy
I-am-called-at-runtime xy
val #t

but compilation fails

$ GUILE_AUTO_COMPILE=1 guile2.2 ew.scm
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/hanwen/vc/lilypond/ew.scm
;;; WARNING: compilation of /home/hanwen/vc/lilypond/ew.scm failed:
;;; Unbound variable: run-at-compile-time

$ guild2.2 compile ew.scm
Backtrace:
In system/base/target.scm:
     57:6 19 (with-target _ _)
In system/base/compile.scm:
..
Unbound variable: run-at-compile-time


If I encapsulate the run-at-compile-time definition with

 (eval-when
  (compile eval)

it works if I remove the module manipulation, but the module-ref
doesn't work. It looks like the settings from module-define! are not
serialized into the byte code, so I can't have code that relies on
correspondence between module-define driven from macros and module-ref
during evaluation.

[hanwen@localhost lilypond]$ guild2.2 compile ew.scm
I-am-called-at-compile-time xy
wrote `/home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/ew.scm.go'
[hanwen@localhost lilypond]$ GUILE_AUTO_COMPILE=0 guile2.2 ew.scm
I-am-called-at-runtime xy
Backtrace:
           6 (apply-smob/1 #<catch-closure 56514b4c37a0>)
In ice-9/boot-9.scm:
    705:2  5 (call-with-prompt ("prompt") #<procedure 56514b4d5c60 …> …)
In ice-9/eval.scm:
    619:8  4 (_ #(#(#<directory (guile-user) 56514b591140>)))
In ice-9/boot-9.scm:
   2312:4  3 (save-module-excursion #<procedure 56514b54b330 at ice-…>)
  3832:12  2 (_)
In ew.scm:
    10:10  1 (runtime-call "xy")
In unknown file:
           0 (scm-error misc-error #f "~A ~S ~S ~S" ("No variabl…" …) …)

ERROR: In procedure scm-error:
No variable named xy in #<directory (guile-user) 56514b591140>

On Fri, Jan 31, 2020 at 9:01 PM Linus Björnstam
<linus.bjornstam@veryfast.biz> wrote:
>
> Read the docs. That seems to be a documentation bug. Try fiddling with the arguments to eval when and see if you can make it work.
>
> --
>   Linus Björnstam
>
> On Fri, 31 Jan 2020, at 20:17, Han-Wen Nienhuys wrote:
> > On Fri, Jan 31, 2020 at 7:20 PM Linus Björnstam
> > <linus.bjornstam@veryfast.biz> wrote:
> > > I don't really understand your question. With defmacro and syntax-case you can run arbitrary code. If you just output code that does module-define! that won't be run until runtime, and thus you cannot depend on the result of that module-define! during expansion. You can however wrap it in an eval-when to solve that issue. That allows you to specify when code gets run. With module-define! I personally find it all a bit icky, but I usually stay as far away from phasing as I can :)
> >
> >  eval-when looks like it might be a solution to the puzzle , but
> > honestly, the doc at
> >
> > https://www.gnu.org/software/guile/manual/html_node/Eval-When.html
> >
> > has me mystified.  When I run the example through guile 2.2 and
> > display *compilation-date*,
> >
> > I get a different answer each time. Shouldn't it be a fixed timestamp
> > (of when the compile happened?)
> >
> > [hanwen@localhost lilypond]$ guile2.2 e.scm
> > ;;; note: source file /home/hanwen/vc/lilypond/e.scm
> > ;;;       newer than compiled
> > /home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/e.scm.go
> > ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> > ;;;       or pass the --no-auto-compile argument to disable.
> > ;;; compiling /home/hanwen/vc/lilypond/e.scm
> > ;;; compiled
> > /home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/e.scm.go
> > Fri Jan 31 20:15:57+0100 2020
> > [hanwen@localhost lilypond]$ guile2.2 e.scm
> > Fri Jan 31 20:15:58+0100 2020
> > [hanwen@localhost lilypond]$ guile2.2 e.scm
> > Fri Jan 31 20:16:00+0100 2020
> >
> > --
> > Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen
> >



-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[Han-Wen Nienhuys]

+lilypond-devel for visibility.

On Sat, Feb 1, 2020 at 10:54 AM Han-Wen Nienhuys <hanwenn@gmail.com> wrote:
>
> Here is an example that shows better how things work, and what might
> be the cause of my problems. Is it right that programmatically set
> contents of "current-module" are not serialized into the compiled
> file?
>
>    (define (run-at-compile-time cmd)
>      (module-define! (current-module) (string->symbol cmd) #t)
>      (format (current-error-port) "I-am-called-at-compile-time ~a\n" cmd))
>
>   (define (runtime-call cmd)
>     (format (current-error-port) "I-am-called-at-runtime ~a\n" cmd)
>     (format (current-error-port) "val ~a\n"
>             (module-ref (current-module) (string->symbol cmd))))
>
>   (defmacro foo (cmd . rest)
>   (run-at-compile-time cmd)
>   `(runtime-call ,cmd))
>
>   (foo "xy")
>
> $ guile1.8 ew.scm
> I-am-called-at-compile-time xy
> I-am-called-at-runtime xy
> val #t
>
> this is compatible with 2.2 without compilation,
>
> $ GUILE_AUTO_COMPILE=0 guile2.2 ew.scm
> I-am-called-at-compile-time xy
> I-am-called-at-runtime xy
> val #t
>
> but compilation fails
>
> $ GUILE_AUTO_COMPILE=1 guile2.2 ew.scm
> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;       or pass the --no-auto-compile argument to disable.
> ;;; compiling /home/hanwen/vc/lilypond/ew.scm
> ;;; WARNING: compilation of /home/hanwen/vc/lilypond/ew.scm failed:
> ;;; Unbound variable: run-at-compile-time
>
> $ guild2.2 compile ew.scm
> Backtrace:
> In system/base/target.scm:
>      57:6 19 (with-target _ _)
> In system/base/compile.scm:
> ..
> Unbound variable: run-at-compile-time
>
>
> If I encapsulate the run-at-compile-time definition with
>
>  (eval-when
>   (compile eval)
>
> it works if I remove the module manipulation, but the module-ref
> doesn't work. It looks like the settings from module-define! are not
> serialized into the byte code, so I can't have code that relies on
> correspondence between module-define driven from macros and module-ref
> during evaluation.
>
> [hanwen@localhost lilypond]$ guild2.2 compile ew.scm
> I-am-called-at-compile-time xy
> wrote `/home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/ew.scm.go'
> [hanwen@localhost lilypond]$ GUILE_AUTO_COMPILE=0 guile2.2 ew.scm
> I-am-called-at-runtime xy
> Backtrace:
>            6 (apply-smob/1 #<catch-closure 56514b4c37a0>)
> In ice-9/boot-9.scm:
>     705:2  5 (call-with-prompt ("prompt") #<procedure 56514b4d5c60 …> …)
> In ice-9/eval.scm:
>     619:8  4 (_ #(#(#<directory (guile-user) 56514b591140>)))
> In ice-9/boot-9.scm:
>    2312:4  3 (save-module-excursion #<procedure 56514b54b330 at ice-…>)
>   3832:12  2 (_)
> In ew.scm:
>     10:10  1 (runtime-call "xy")
> In unknown file:
>            0 (scm-error misc-error #f "~A ~S ~S ~S" ("No variabl…" …) …)
>
> ERROR: In procedure scm-error:
> No variable named xy in #<directory (guile-user) 56514b591140>
>

-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[David Kastrup]

Han-Wen Nienhuys <hanwenn@gmail.com> writes:

> +lilypond-devel for visibility.
>
> On Sat, Feb 1, 2020 at 10:54 AM Han-Wen Nienhuys <hanwenn@gmail.com> wrote:
>>
>> Here is an example that shows better how things work, and what might
>> be the cause of my problems. Is it right that programmatically set
>> contents of "current-module" are not serialized into the compiled
>> file?
>>
>>    (define (run-at-compile-time cmd)
>>      (module-define! (current-module) (string->symbol cmd) #t)
>>      (format (current-error-port) "I-am-called-at-compile-time ~a\n" cmd))
>>
>>   (define (runtime-call cmd)
>>     (format (current-error-port) "I-am-called-at-runtime ~a\n" cmd)
>>     (format (current-error-port) "val ~a\n"
>>             (module-ref (current-module) (string->symbol cmd))))
>>
>>   (defmacro foo (cmd . rest)
>>   (run-at-compile-time cmd)
>>   `(runtime-call ,cmd))
>>
>>   (foo "xy")
>>
>> $ guile1.8 ew.scm
>> I-am-called-at-compile-time xy
>> I-am-called-at-runtime xy
>> val #t
>>
>> this is compatible with 2.2 without compilation,
>>
>> $ GUILE_AUTO_COMPILE=0 guile2.2 ew.scm
>> I-am-called-at-compile-time xy
>> I-am-called-at-runtime xy
>> val #t
>>
>> but compilation fails
>>
>> $ GUILE_AUTO_COMPILE=1 guile2.2 ew.scm
>> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
>> ;;;       or pass the --no-auto-compile argument to disable.
>> ;;; compiling /home/hanwen/vc/lilypond/ew.scm
>> ;;; WARNING: compilation of /home/hanwen/vc/lilypond/ew.scm failed:
>> ;;; Unbound variable: run-at-compile-time
>>
>> $ guild2.2 compile ew.scm
>> Backtrace:
>> In system/base/target.scm:
>>      57:6 19 (with-target _ _)
>> In system/base/compile.scm:
>> ..
>> Unbound variable: run-at-compile-time
>>
>>
>> If I encapsulate the run-at-compile-time definition with
>>
>>  (eval-when
>>   (compile eval)
>>
>> it works if I remove the module manipulation, but the module-ref
>> doesn't work. It looks like the settings from module-define! are not
>> serialized into the byte code, so I can't have code that relies on
>> correspondence between module-define driven from macros and module-ref
>> during evaluation.
>>
>> [hanwen@localhost lilypond]$ guild2.2 compile ew.scm
>> I-am-called-at-compile-time xy
>> wrote `/home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/ew.scm.go'
>> [hanwen@localhost lilypond]$ GUILE_AUTO_COMPILE=0 guile2.2 ew.scm
>> I-am-called-at-runtime xy
>> Backtrace:
>>            6 (apply-smob/1 #<catch-closure 56514b4c37a0>)
>> In ice-9/boot-9.scm:
>>     705:2  5 (call-with-prompt ("prompt") #<procedure 56514b4d5c60 …> …)
>> In ice-9/eval.scm:
>>     619:8  4 (_ #(#(#<directory (guile-user) 56514b591140>)))
>> In ice-9/boot-9.scm:
>>    2312:4  3 (save-module-excursion #<procedure 56514b54b330 at ice-…>)
>>   3832:12  2 (_)
>> In ew.scm:
>>     10:10  1 (runtime-call "xy")
>> In unknown file:
>>            0 (scm-error misc-error #f "~A ~S ~S ~S" ("No variabl…" …) …)
>>
>> ERROR: In procedure scm-error:
>> No variable named xy in #<directory (guile-user) 56514b591140>
>>

But that is not using a local define at all.  Can you point out the
actual code that failed for you?

-- 
David Kastrup

Re: unhandled constant?

[Han-Wen Nienhuys]

On Sat, Feb 1, 2020 at 11:11 AM David Kastrup <dak@gnu.org> wrote:
> >> Here is an example that shows better how things work, and what might
> >> be the cause of my problems. Is it right that programmatically set
> >> contents of "current-module" are not serialized into the compiled
> >> file?
> >>
> >>    (define (run-at-compile-time cmd)
> >>      (module-define! (current-module) (string->symbol cmd) #t)
> >>      (format (current-error-port) "I-am-called-at-compile-time ~a\n" cmd))
> >>
> >>   (define (runtime-call cmd)
> >>     (format (current-error-port) "I-am-called-at-runtime ~a\n" cmd)
> >>     (format (current-error-port) "val ~a\n"
> >>             (module-ref (current-module) (string->symbol cmd))))
> >>
> >>   (defmacro foo (cmd . rest)
> >>   (run-at-compile-time cmd)
> >>   `(runtime-call ,cmd))
> >>
> >>   (foo "xy")
..
> >> ERROR: In procedure scm-error:
> >> No variable named xy in #<directory (guile-user) 56514b591140>
> >>
>
> But that is not using a local define at all.  Can you point out the
> actual code that failed for you?

There are two independent problems. One is a problem with inner
defines, which is addressed by

  https://codereview.appspot.com/553480044/

the symptom is compilation failing with "unhandled constant #<procedure ... > "

The other is a problem you can reproduce if you check out

  https://github.com/hanwen/lilypond/tree/guile22-experiment

with the symptom being:

;;; compiling /home/hanwen/vc/lilypond/out/share/lilypond/current/scm/define-markup-commands.scm
fatal error: Not a markup command: line

This is because the LilyPond macro "markup" doesn't recognize markup
command and aborts in code that is executed at compile-time. The code
that triggers this are  definitions in scm/define-markup-commands.scm
that use (mark #:blah .. ) in the function body.

You can verify this by rewriting
https://github.com/lilypond/lilypond/blob/c5ffa540fdbe52486b9575567ede70be575adb47/scm/define-markup-commands.scm#L305
and seeing how the error message changes.

I still don't understand why some code is executed compile time (the
expansion of the  markup macro) while other is not (defining the
make-x-markup function in (current-module))

Since we recognize markup commands by looking them up in
(current-module), I believe the example I showed here shows that we
can never make this work, and we  will have to revisit the markup
macros completely.

-- 
Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen

Re: unhandled constant?

[David Kastrup]

Han-Wen Nienhuys <hanwenn@gmail.com> writes:

> On Sat, Feb 1, 2020 at 11:11 AM David Kastrup <dak@gnu.org> wrote:
>> >> Here is an example that shows better how things work, and what might
>> >> be the cause of my problems. Is it right that programmatically set
>> >> contents of "current-module" are not serialized into the compiled
>> >> file?
>> >>
>> >>    (define (run-at-compile-time cmd)
>> >>      (module-define! (current-module) (string->symbol cmd) #t)
>> >>      (format (current-error-port) "I-am-called-at-compile-time ~a\n" cmd))
>> >>
>> >>   (define (runtime-call cmd)
>> >>     (format (current-error-port) "I-am-called-at-runtime ~a\n" cmd)
>> >>     (format (current-error-port) "val ~a\n"
>> >>             (module-ref (current-module) (string->symbol cmd))))
>> >>
>> >>   (defmacro foo (cmd . rest)
>> >>   (run-at-compile-time cmd)
>> >>   `(runtime-call ,cmd))
>> >>
>> >>   (foo "xy")
> ..
>> >> ERROR: In procedure scm-error:
>> >> No variable named xy in #<directory (guile-user) 56514b591140>
>> >>
>>
>> But that is not using a local define at all.  Can you point out the
>> actual code that failed for you?
>
> There are two independent problems. One is a problem with inner
> defines, which is addressed by
>
>   https://codereview.appspot.com/553480044/
>
> the symptom is compilation failing with "unhandled constant
> #<procedure ... > "

Ah, ok.  This appears not to signify that anything is wrong with the
code in principle but rather that the byte compiler is not sufficiently
capable for processing it.

> The other is a problem you can reproduce if you check out
>
>   https://github.com/hanwen/lilypond/tree/guile22-experiment
>
> with the symptom being:
>
> ;;; compiling
> /home/hanwen/vc/lilypond/out/share/lilypond/current/scm/define-markup-commands.scm
> fatal error: Not a markup command: line
>
> This is because the LilyPond macro "markup" doesn't recognize markup
> command and aborts in code that is executed at compile-time. The code
> that triggers this are  definitions in scm/define-markup-commands.scm
> that use (mark #:blah .. ) in the function body.

The LilyPond macro "markup" is something I would not mind getting rid of
sooner rather than later.  It's sick.  It is not completely clear to me
why it needs the actual definitions to work with, but it may have
something to do with its partly workable support of markup list commands.

The make-...-markup convenience functions are quite more sane.  Stacking
a lot of them together may incur a bit of runtime overhead since they do
argument checking that the results from markup macro expansion don't.
But markup command processing times are likely quite negligible in the
total scheme of things even if we end up rechecking large markup lists
as they are passed from one function to another.

> You can verify this by rewriting
> https://github.com/lilypond/lilypond/blob/c5ffa540fdbe52486b9575567ede70be575adb47/scm/define-markup-commands.scm#L305
> and seeing how the error message changes.
>
> I still don't understand why some code is executed compile time (the
> expansion of the  markup macro) while other is not (defining the
> make-x-markup function in (current-module))
>
> Since we recognize markup commands by looking them up in
> (current-module), I believe the example I showed here shows that we
> can never make this work, and we  will have to revisit the markup
> macros completely.

My vote is on throwing them out.  But that would affect a whole bunch of
user-level programs.  We could add a macro _function_ that does the hard
work at execution time rather than at expansion time as a backwards
compatibility measure, and discourage using it for new code.

-- 
David Kastrup

Re: unhandled constant?

[Linus Björnstam]

Did you try just expanding it to a define? Or just output a module-define! at runtime and run a module-define! at compile time.



-- 
  Linus Björnstam

On Sat, 1 Feb 2020, at 10:54, Han-Wen Nienhuys wrote:
> Here is an example that shows better how things work, and what might
> be the cause of my problems. Is it right that programmatically set
> contents of "current-module" are not serialized into the compiled
> file?
> 
>    (define (run-at-compile-time cmd)
>      (module-define! (current-module) (string->symbol cmd) #t)
>      (format (current-error-port) "I-am-called-at-compile-time ~a\n" cmd))
> 
>   (define (runtime-call cmd)
>     (format (current-error-port) "I-am-called-at-runtime ~a\n" cmd)
>     (format (current-error-port) "val ~a\n"
>             (module-ref (current-module) (string->symbol cmd))))
> 
>   (defmacro foo (cmd . rest)
>   (run-at-compile-time cmd)
>   `(runtime-call ,cmd))
> 
>   (foo "xy")
> 
> $ guile1.8 ew.scm
> I-am-called-at-compile-time xy
> I-am-called-at-runtime xy
> val #t
> 
> this is compatible with 2.2 without compilation,
> 
> $ GUILE_AUTO_COMPILE=0 guile2.2 ew.scm
> I-am-called-at-compile-time xy
> I-am-called-at-runtime xy
> val #t
> 
> but compilation fails
> 
> $ GUILE_AUTO_COMPILE=1 guile2.2 ew.scm
> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;       or pass the --no-auto-compile argument to disable.
> ;;; compiling /home/hanwen/vc/lilypond/ew.scm
> ;;; WARNING: compilation of /home/hanwen/vc/lilypond/ew.scm failed:
> ;;; Unbound variable: run-at-compile-time
> 
> $ guild2.2 compile ew.scm
> Backtrace:
> In system/base/target.scm:
>      57:6 19 (with-target _ _)
> In system/base/compile.scm:
> ..
> Unbound variable: run-at-compile-time
> 
> 
> If I encapsulate the run-at-compile-time definition with
> 
>  (eval-when
>   (compile eval)
> 
> it works if I remove the module manipulation, but the module-ref
> doesn't work. It looks like the settings from module-define! are not
> serialized into the byte code, so I can't have code that relies on
> correspondence between module-define driven from macros and module-ref
> during evaluation.
> 
> [hanwen@localhost lilypond]$ guild2.2 compile ew.scm
> I-am-called-at-compile-time xy
> wrote 
> `/home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/ew.scm.go'
> [hanwen@localhost lilypond]$ GUILE_AUTO_COMPILE=0 guile2.2 ew.scm
> I-am-called-at-runtime xy
> Backtrace:
>            6 (apply-smob/1 #<catch-closure 56514b4c37a0>)
> In ice-9/boot-9.scm:
>     705:2  5 (call-with-prompt ("prompt") #<procedure 56514b4d5c60 …> …)
> In ice-9/eval.scm:
>     619:8  4 (_ #(#(#<directory (guile-user) 56514b591140>)))
> In ice-9/boot-9.scm:
>    2312:4  3 (save-module-excursion #<procedure 56514b54b330 at ice-…>)
>   3832:12  2 (_)
> In ew.scm:
>     10:10  1 (runtime-call "xy")
> In unknown file:
>            0 (scm-error misc-error #f "~A ~S ~S ~S" ("No variabl…" …) …)
> 
> ERROR: In procedure scm-error:
> No variable named xy in #<directory (guile-user) 56514b591140>
> 
> On Fri, Jan 31, 2020 at 9:01 PM Linus Björnstam
> <linus.bjornstam@veryfast.biz> wrote:
> >
> > Read the docs. That seems to be a documentation bug. Try fiddling with the arguments to eval when and see if you can make it work.
> >
> > --
> >   Linus Björnstam
> >
> > On Fri, 31 Jan 2020, at 20:17, Han-Wen Nienhuys wrote:
> > > On Fri, Jan 31, 2020 at 7:20 PM Linus Björnstam
> > > <linus.bjornstam@veryfast.biz> wrote:
> > > > I don't really understand your question. With defmacro and syntax-case you can run arbitrary code. If you just output code that does module-define! that won't be run until runtime, and thus you cannot depend on the result of that module-define! during expansion. You can however wrap it in an eval-when to solve that issue. That allows you to specify when code gets run. With module-define! I personally find it all a bit icky, but I usually stay as far away from phasing as I can :)
> > >
> > >  eval-when looks like it might be a solution to the puzzle , but
> > > honestly, the doc at
> > >
> > > https://www.gnu.org/software/guile/manual/html_node/Eval-When.html
> > >
> > > has me mystified.  When I run the example through guile 2.2 and
> > > display *compilation-date*,
> > >
> > > I get a different answer each time. Shouldn't it be a fixed timestamp
> > > (of when the compile happened?)
> > >
> > > [hanwen@localhost lilypond]$ guile2.2 e.scm
> > > ;;; note: source file /home/hanwen/vc/lilypond/e.scm
> > > ;;;       newer than compiled
> > > /home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/e.scm.go
> > > ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> > > ;;;       or pass the --no-auto-compile argument to disable.
> > > ;;; compiling /home/hanwen/vc/lilypond/e.scm
> > > ;;; compiled
> > > /home/hanwen/.cache/guile/ccache/2.2-LE-8-3.A/home/hanwen/vc/lilypond/e.scm.go
> > > Fri Jan 31 20:15:57+0100 2020
> > > [hanwen@localhost lilypond]$ guile2.2 e.scm
> > > Fri Jan 31 20:15:58+0100 2020
> > > [hanwen@localhost lilypond]$ guile2.2 e.scm
> > > Fri Jan 31 20:16:00+0100 2020
> > >
> > > --
> > > Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen
> > >
> 
> 
> 
> -- 
> Han-Wen Nienhuys - hanwenn@gmail.com - http://www.xs4all.nl/~hanwen
>

Re: unhandled constant?

[David Kastrup]

Linus Björnstam <linus.bjornstam@veryfast.biz> writes:

> Guile1.8's macros are run-time macros: they are executed directly and
> not transformed to output code that is then compiled. That is the
> reason why your code works: to newer guiles the (inner ...) is only
> available at expansion time. The macro output is trying to call code
> that does not exist at runtime!
>
> For this to be working code the (inner ...) function needs to be
> available in the macro expansion. I didn't read through exactly what
> you are trying to do, but try outputting a let:
>
> `(let ((inner (lambda (n v) (set ! ...))))
>   (inner ,name ,value))
>
> I doubt you can make the old code work in newer guiles, since I doubt
> any scheme is a s lax about expansion time and macro time separation.

Can you expand about the "expansion time and macro time separation"?

If we have

(define decl '())
(define (make-var n v) (list "var" n v))
(defmacro define-session (name value)
  (define (inner n v)
    (set! decl
        (cons
         (make-var n v)
         decl))
    )
  `(,inner ',name ,value))
(define-session foo 1)
(display decl)
(newline)

as stated, the local function "inner" is defined at macro time, but the
form
`(,inner ',name ,value)
does not export the _name_ inner but rather the defined function.  That
part naively appears to me like it should work; an "expansion time and
macro time" issue appears rather to be that inner calls make-var (and
accesses decl) which is only being defined at expansion time.

The error message, however, rather appears to complain about inner being
undefined rather than the definition of inner referring to undefined
entities.

Can you clarify?

-- 
David Kastrup

Re: unhandled constant?

[Linus Björnstam]

On Sat, 1 Feb 2020, at 12:09, David Kastrup wrote:
> 
> Can you expand about the "expansion time and macro time separation"?
> 
> If we have
> 
> (define decl '())
> (define (make-var n v) (list "var" n v))
> (defmacro define-session (name value)
>   (define (inner n v)
>     (set! decl
>         (cons
>          (make-var n v)
>          decl))
>     )
>   `(,inner ',name ,value))
> (define-session foo 1)
> (display decl)
> (newline)
> 
> as stated, the local function "inner" is defined at macro time, but the
> form
> `(,inner ',name ,value)
> does not export the _name_ inner but rather the defined function.  That
> part naively appears to me like it should work; an "expansion time and
> macro time" issue appears rather to be that inner calls make-var (and
> accesses decl) which is only being defined at expansion time.
> 
> The error message, however, rather appears to complain about inner being
> undefined rather than the definition of inner referring to undefined
> entities.


I am not sure what is really the problem. Either the inner function is not present at runtime due to separation of compile time and runtime, or it is a result of how defmacro re-introduces the result of the macro into the syntactic context of the macro usage (where inner is not visible). Either way, having the inner definition in the macro output will solve the problem of it not being visible.

Re: unhandled constant?

[David Kastrup]

Linus Björnstam <linus.bjornstam@veryfast.biz> writes:

> On Sat, 1 Feb 2020, at 12:09, David Kastrup wrote:
>> 
>> Can you expand about the "expansion time and macro time separation"?
>> 
>> If we have
>> 
>> (define decl '())
>> (define (make-var n v) (list "var" n v))
>> (defmacro define-session (name value)
>>   (define (inner n v)
>>     (set! decl
>>         (cons
>>          (make-var n v)
>>          decl))
>>     )
>>   `(,inner ',name ,value))
>> (define-session foo 1)
>> (display decl)
>> (newline)
>> 
>> as stated, the local function "inner" is defined at macro time, but the
>> form
>> `(,inner ',name ,value)
>> does not export the _name_ inner but rather the defined function.  That
>> part naively appears to me like it should work; an "expansion time and
>> macro time" issue appears rather to be that inner calls make-var (and
>> accesses decl) which is only being defined at expansion time.
>> 
>> The error message, however, rather appears to complain about inner being
>> undefined rather than the definition of inner referring to undefined
>> entities.
>
>
> I am not sure what is really the problem. Either the inner function is
> not present at runtime due to separation of compile time and runtime,
> or it is a result of how defmacro re-introduces the result of the
> macro into the syntactic context of the macro usage (where inner is
> not visible). Either way, having the inner definition in the macro
> output will solve the problem of it not being visible.

This fails when byte-compiling, so it would appear that the equivalent
of

(defmacro ...
  (define local-fun ...
  `(,local-fun ...)))

is not able to represent the local (and thus ultimately anonymous)
function here.  Whether this is a general shortcoming or is conditioned
on local-fun calling functions not available at byte-compile time (in
which case those would need to get wrapped in eval-and-expand) I haven't
checked yet.

-- 
David Kastrup

Re: unhandled constant?

[Linus Björnstam]

tried it in a couple of syntax-case-enabled schemes (defmacro is implemented using syntax-case in guile) and it doesn't work anywhere. The procedure ends up undefined everywhere.

The #<procedure> object ,inner points to doesn't exist at runtime because it is not defined anywhere in any meaningful way to the compiler. One work-around would be to output the literal lambda in place of ,inner but by then you should just output a (let ...) With the lambda in it and let the inliner take care of it.

-- 
  Linus Björnstam

On Sat, 1 Feb 2020, at 15:23, David Kastrup wrote:
> Linus Björnstam <linus.bjornstam@veryfast.biz> writes:
> 
> > On Sat, 1 Feb 2020, at 12:09, David Kastrup wrote:
> >> 
> >> Can you expand about the "expansion time and macro time separation"?
> >> 
> >> If we have
> >> 
> >> (define decl '())
> >> (define (make-var n v) (list "var" n v))
> >> (defmacro define-session (name value)
> >>   (define (inner n v)
> >>     (set! decl
> >>         (cons
> >>          (make-var n v)
> >>          decl))
> >>     )
> >>   `(,inner ',name ,value))
> >> (define-session foo 1)
> >> (display decl)
> >> (newline)
> >> 
> >> as stated, the local function "inner" is defined at macro time, but the
> >> form
> >> `(,inner ',name ,value)
> >> does not export the _name_ inner but rather the defined function.  That
> >> part naively appears to me like it should work; an "expansion time and
> >> macro time" issue appears rather to be that inner calls make-var (and
> >> accesses decl) which is only being defined at expansion time.
> >> 
> >> The error message, however, rather appears to complain about inner being
> >> undefined rather than the definition of inner referring to undefined
> >> entities.
> >
> >
> > I am not sure what is really the problem. Either the inner function is
> > not present at runtime due to separation of compile time and runtime,
> > or it is a result of how defmacro re-introduces the result of the
> > macro into the syntactic context of the macro usage (where inner is
> > not visible). Either way, having the inner definition in the macro
> > output will solve the problem of it not being visible.
> 
> This fails when byte-compiling, so it would appear that the equivalent
> of
> 
> (defmacro ...
>   (define local-fun ...
>   `(,local-fun ...)))
> 
> is not able to represent the local (and thus ultimately anonymous)
> function here.  Whether this is a general shortcoming or is conditioned
> on local-fun calling functions not available at byte-compile time (in
> which case those would need to get wrapped in eval-and-expand) I haven't
> checked yet.
> 
> -- 
> David Kastrup
>

Re: unhandled constant?

[Taylan Kammer]

> (defmacro define-session (name value)
>   (define (inner n v)
>     (set! decl
>         (cons
>          (make-var n v)
>          decl))
>     )
>   `(,inner ',name ,value))

The problem here is that the macro output

  `(,inner ',name ,value)

would include a #<procedure> object, because it evaluates 'inner', which
has been defined as a procedure.  The raw output would look something
like this:

  (#<procedure inner (a b)> 'foo "bar)

And the compiler tries to put that into a file.  But procedure objects
cannot be put into a file.

(They cannot be "serialized" i.e. turned back into text.  Hence the
clumsy "#<procedure ...>" thing which tells us a bit about the procedure
like its name and the number of arguments it takes but doesn't actually
include its code.)

That's why the compiler chokes.  It could put a constant like the number
982346 or the string "foo" or the vector #(x 4 "blah") into the output
it generates, because all those objects can be serialized, but a
procedure object cannot be truly serialized.

Note that something like (lambda () ...) is NOT a procedure; it's a
list.  Such a list can be evaluated as Scheme code, within a larger
context (an "environment"), which then yields a procedure object, but
that resulting procedure object cannot be turned back into text, because
it might be relying on the environment in which it was defined.

Like others pointed out, there's a couple of ways to rewrite that code
to avoid putting a procedure object straight into the macro output.  For
instance, you could put the definition of the procedure into the output,
so it will be evaluated (turned into a procedure) at run-time:

  (defmacro define-session (name value)
    `(begin
       (define (inner n v)
         (set! decl
             (cons
              (make-var n v)
              decl))
         )
       (inner ',name ,value)))

I'm not sure if this code will work verbatim in 1.8 and 2.2, but you get
the idea: don't put procedures into macro output.

- Taylan


On 29.01.2020 00:08, Han-Wen Nienhuys wrote:
> Some of the lilypond Scheme files do the following:
> 
> 
> (define decl '())
> (define (make-var n v) (list "var" n v))
> (defmacro define-session (name value)
>   (define (inner n v)
>     (set! decl
>         (cons
>          (make-var n v)
>          decl))
>     )
>   `(,inner ',name ,value))
> (define-session foo 1)
> (display decl)
> (newline)
> 
> In GUILE 2.2, this yields
> 
> ;;; WARNING: compilation of /home/hanwen/vc/lilypond/q.scm failed:
> ;;; unhandled constant #<procedure inner (a b)>
> 
> What does this error message mean, and what should I do to address the problem?
> 

Fwd: Re: unhandled constant?

[Taylan Kammer]

(Duplicate of the mail I sent to guile-devel before realizing you're not
on the list.)

> (defmacro define-session (name value)
>   (define (inner n v)
>     (set! decl
>         (cons
>          (make-var n v)
>          decl))
>     )
>   `(,inner ',name ,value))

The problem here is that the macro output

  `(,inner ',name ,value)

would include a #<procedure> object, because it evaluates 'inner', which
has been defined as a procedure.  The raw output would look something
like this:

  (#<procedure inner (a b)> 'foo "bar)

And the compiler tries to put that into a file.  But procedure objects
cannot be put into a file.

(They cannot be "serialized" i.e. turned back into text.  Hence the
clumsy "#<procedure ...>" thing which tells us a bit about the procedure
like its name and the number of arguments it takes but doesn't actually
include its code.)

That's why the compiler chokes.  It could put a constant like the number
982346 or the string "foo" or the vector #(x 4 "blah") into the output
it generates, because all those objects can be serialized, but a
procedure object cannot be truly serialized.

Note that something like (lambda () ...) is NOT a procedure; it's a
list.  Such a list can be evaluated as Scheme code, within a larger
context (an "environment"), which then yields a procedure object, but
that resulting procedure object cannot be turned back into text, because
it might be relying on the environment in which it was defined.

Like others pointed out, there's a couple of ways to rewrite that code
to avoid putting a procedure object straight into the macro output.  For
instance, you could put the definition of the procedure into the output,
so it will be evaluated (turned into a procedure) at run-time:

  (defmacro define-session (name value)
    `(begin
       (define (inner n v)
         (set! decl
             (cons
              (make-var n v)
              decl))
         )
       (inner ',name ,value)))

I'm not sure if this code will work verbatim in 1.8 and 2.2, but you get
the idea: don't put procedures into macro output.

- Taylan


On 29.01.2020 00:08, Han-Wen Nienhuys wrote:
> Some of the lilypond Scheme files do the following:
> 
> 
> (define decl '())
> (define (make-var n v) (list "var" n v))
> (defmacro define-session (name value)
>   (define (inner n v)
>     (set! decl
>         (cons
>          (make-var n v)
>          decl))
>     )
>   `(,inner ',name ,value))
> (define-session foo 1)
> (display decl)
> (newline)
> 
> In GUILE 2.2, this yields
> 
> ;;; WARNING: compilation of /home/hanwen/vc/lilypond/q.scm failed:
> ;;; unhandled constant #<procedure inner (a b)>
> 
> What does this error message mean, and what should I do to address the problem?
>