can't export conditional variable set with 'cond'

can't export conditional variable set with 'cond'

[Mortimer Cladwell]

Hi,

Suppose I have the following module "testexport" with the auxiliary module
"env.scm", used to set global environment variables:

-------------testexport.scm begin----------------------------------
(define-module (testexport)
  #:use-module (ice-9 pretty-print)
  #:use-module (env)
  )

(define (main)
  (pretty-print (string-append  "varA in main: " varA))
  (pretty-print (string-append  "varB in main: " varB))
  (pretty-print (string-append  "varC in main: " varC))
  )

(main)
-------------testexport.scm end----------------------------------

-------------env.scm begin----------------------------------
(define-module (env)
  #:use-module (ice-9 pretty-print)
  #:export(varA varB varC))

(define varA "A")
(define varB "")
(define varC "")
(define testval "x")

(cond
 ((string= testval "x") (set! varB "B"))
 ((string= testval "y") (set! varB "error"))
 ((string= testval "z") (set! varB "null"))
 )
(pretty-print (string-append "varB post cond: " varB))

(if (string= testval "x")  (set! varC "C"))
(pretty-print (string-append "varC post if: " varC))
-------------env.scm end----------------------------------

When I run the above I get as output:

"varB post cond: B"
"varC post if: C"
"varA in main: A"
"varB in main: "
"varC in main: C"

Why can I reset the variables with both 'cond' and 'if' in env.scm, but
only the variable reset with 'if' is exported with the updated value?
Thanks
Mortimer

Re: can't export conditional variable set with 'cond'

[Damien Mattei]

Hello Mortimer,
i get in a lot of trouble doing those sort of things with Scheme.
i mean trying like in C , expecting to have exported variable , in a object
file , and thinking that they are fixed at an address in memory, even this
idea is just an assumption of how i expected things to work. But this does
not seem to be the case with scheme module, and that is not special to
Guile.
i test your code and get a different result, so i suppose, it is normal
(not specified to work) or a bug:
scheme@(guile-user)> (load "testexport.scm")
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /Users/mattei/Scheme-PLUS-for-Guile/testexport.scm
;;; compiling /usr/local/share/guile/site/3.0/env.scm
;;; compiled
/Users/mattei/.cache/guile/ccache/3.0-LE-8-4.6/usr/local/share/guile/site/3.0/env.scm.go
"varB post cond: B"
"varC post if: C"
;;; compiled
/Users/mattei/.cache/guile/ccache/3.0-LE-8-4.6/Users/mattei/Scheme-PLUS-for-Guile/testexport.scm.go
"varA in main: A"
"varB in main: "
"varC in main: "

if you want to do that, it is better to do not export the variable
themselves but some accessors, or perheaps use the GOOPS ,object system of
Guile.

with accessors this works for me, and i hope you too:

(define-module (env)
  #:use-module (ice-9 pretty-print)
  #:export(varA getB getC ))

(define varA "A")
(define varB "")
(define varC "")
(define testval "x")

(define (getB)
  varB)

(define (getC)
  varC)


(cond
 ((string= testval "x") (set! varB "B"))
 ((string= testval "y") (set! varB "error"))
 ((string= testval "z") (set! varB "null"))
 )
(pretty-print (string-append "varB post cond: " varB))

(if (string= testval "x")  (set! varC "C"))
(pretty-print (string-append "varC post if: " varC))

(define-module (testexport)
  #:use-module (ice-9 pretty-print)
  #:use-module (env)
  )

(define (main)
  (pretty-print (string-append  "varA in main: " varA))
  ;;(pretty-print (string-append  "varB in main: " varB))
  ;;(pretty-print (string-append  "varC in main: " varC))

  (pretty-print (string-append  "varA in main: " varA))
  (pretty-print (string-append  "varB in main: " (getB)))
  (pretty-print (string-append  "varC in main: " (getC)))
  )

(main)

and the result is ok:

scheme@(guile-user)> (load "testexport.scm")
;;; note: source file /Users/mattei/Scheme-PLUS-for-Guile/testexport.scm
;;;       newer than compiled
/Users/mattei/.cache/guile/ccache/3.0-LE-8-4.6/Users/mattei/Scheme-PLUS-for-Guile/testexport.scm.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /Users/mattei/Scheme-PLUS-for-Guile/testexport.scm
;;; note: source file /usr/local/share/guile/site/3.0/env.scm
;;;       newer than compiled
/Users/mattei/.cache/guile/ccache/3.0-LE-8-4.6/usr/local/share/guile/site/3.0/env.scm.go
;;; compiling /usr/local/share/guile/site/3.0/env.scm
;;; compiled
/Users/mattei/.cache/guile/ccache/3.0-LE-8-4.6/usr/local/share/guile/site/3.0/env.scm.go
"varB post cond: B"
"varC post if: C"
;;; compiled
/Users/mattei/.cache/guile/ccache/3.0-LE-8-4.6/Users/mattei/Scheme-PLUS-for-Guile/testexport.scm.go
"varA in main: A"
"varA in main: A"
"varB in main: B"
"varC in main: C"

Best Regards,

Damien

On Tue, Jan 23, 2024 at 2:38 PM Mortimer Cladwell <mbcladwell@gmail.com>
wrote:

> Hi,
>
> Suppose I have the following module "testexport" with the auxiliary module
> "env.scm", used to set global environment variables:
>
> -------------testexport.scm begin----------------------------------
> (define-module (testexport)
>   #:use-module (ice-9 pretty-print)
>   #:use-module (env)
>   )
>
> (define (main)
>   (pretty-print (string-append  "varA in main: " varA))
>   (pretty-print (string-append  "varB in main: " varB))
>   (pretty-print (string-append  "varC in main: " varC))
>   )
>
> (main)
> -------------testexport.scm end----------------------------------
>
> -------------env.scm begin----------------------------------
> (define-module (env)
>   #:use-module (ice-9 pretty-print)
>   #:export(varA varB varC))
>
> (define varA "A")
> (define varB "")
> (define varC "")
> (define testval "x")
>
> (cond
>  ((string= testval "x") (set! varB "B"))
>  ((string= testval "y") (set! varB "error"))
>  ((string= testval "z") (set! varB "null"))
>  )
> (pretty-print (string-append "varB post cond: " varB))
>
> (if (string= testval "x")  (set! varC "C"))
> (pretty-print (string-append "varC post if: " varC))
> -------------env.scm end----------------------------------
>
> When I run the above I get as output:
>
> "varB post cond: B"
> "varC post if: C"
> "varA in main: A"
> "varB in main: "
> "varC in main: C"
>
> Why can I reset the variables with both 'cond' and 'if' in env.scm, but
> only the variable reset with 'if' is exported with the updated value?
> Thanks
> Mortimer
>

Re: can't export conditional variable set with 'cond'

[Skyler Ferris]

On 1/23/24 05:37, Mortimer Cladwell wrote:
> When I run the above I get as output:
>
> "varB post cond: B"
> "varC post if: C"
> "varA in main: A"
> "varB in main:"
> "varC in main: C"
>
> Why can I reset the variables with both 'cond' and 'if' in env.scm, but
> only the variable reset with 'if' is exported with the updated value?

I don't have a complete answer, but it looks like this has to do with 
compilation. If I run it like this:

$ export GUILE_LOAD_PATH=$pwd
$ guile main.scm

Then I see similar output, except that for me varC has also not been 
updated. I am on guile version 3.0.9.

However, if I first set GUILE_AUTO_COMPILE=0 (and delete the cached 
compiled modules) then the main module sees the updated values.

I'm not sure how imperative code in the top level of a module works when 
compiling. It seems like a grey area.

Re: can't export conditional variable set with 'cond'

[Daniel Meißner]

Hi all,

Skyler Ferris writes:

> On 1/23/24 05:37, Mortimer Cladwell wrote:
> > When I run the above I get as output:
> >
> > "varB post cond: B"
> > "varC post if: C"
> > "varA in main: A"
> > "varB in main:"
> > "varC in main: C"
> >
> > Why can I reset the variables with both 'cond' and 'if' in env.scm, but
> > only the variable reset with 'if' is exported with the updated value?
> 
> I don't have a complete answer, but it looks like this has to do with 
> compilation. If I run it like this:
> 
> $ export GUILE_LOAD_PATH=$pwd
> $ guile main.scm
> 
> Then I see similar output, except that for me varC has also not been 
> updated. I am on guile version 3.0.9.
> 
> However, if I first set GUILE_AUTO_COMPILE=0 (and delete the cached 
> compiled modules) then the main module sees the updated values.
> 
> I'm not sure how imperative code in the top level of a module works when 
> compiling. It seems like a grey area.

I think this has something to do with compilation indeed. More specifically, it is caused by cross module inlining [1]. You probably need to declare your env module as not declarative by setting #:declarative? to #f inside the define-module form of env. I think the compiler inlines varB somehow. Not super sure, I am no Guile expert but you could inspect the assembly for main to see what the compiler did.

1: https://www.wingolog.org/archives/2021/05/13/cross-module-inlining-in-guile
2: https://www.gnu.org/software/guile/manual/html_node/Declarative-Modules.html

-- 
Best,
Daniel

RE: can't export conditional variable set with 'cond'

[M]

>I think this has something to do with compilation indeed. More specifically, it is caused by cross module inlining [1]. You probably need to declare your env module as not declarative by setting #:declarative? to #f inside the define-module form of env. I think the compiler inlines varB somehow. Not super sure, I am no Guile expert but you could inspect the assembly for main to see what the compiler did.

You shouldn’t need to do that – definitions / variables that are set! within the same module are (modulo bugs, which apparently exist) are (supposedly) recognised as non-declarative:

>To allow Guile to reason about the values of top-levels from a module, a module can be marked as declarative. This flag applies only to the subset of top-level definitions that are themselves declarative: those that are defined within the compilation unit, and not assigned (set!) or redefined within the compilation unit. 
> By default, modules are compied declaratively if […].

(Technically it doesn’t say that the compiler will actually act according to this flag, but I think it’s clear what’s meant here.)

Re: can't export conditional variable set with 'cond'

[Skyler Ferris]

It's also worth mentioning that a "lispier" way of doing this is to use 
parameters, described in "6.11.2 Parameters". I get the impression that 
you were asking this more for curiosity about how the language 
implementation works than to solve a problem, but I might be wrong in 
that assumption and people who are looking to solve this problem might 
come upon this thread in the future. The example looks like this after 
re-writing to use parameters (and it displays the correct output on my 
machine):

-------------testexport.scm begin----------------------------------
(define-module (testexport)
   #:use-module (ice-9 pretty-print)
   #:use-module (env)
   )

(define (main)
   (pretty-print (string-append  "varA in main: " (varA)))
   (pretty-print (string-append  "varB in main: " (varB)))
   (pretty-print (string-append  "varC in main: " (varC)))
   )

(main)
-------------testexport.scm end----------------------------------

-------------env.scm begin----------------------------------
(define-module (env)
   #:use-module (ice-9 pretty-print)
   #:export(varA varB varC))

(define varA (make-parameter "A"))
(define varB (make-parameter ""))
(define varC (make-parameter ""))
(define testval "x")

(cond
  ((string= testval "x") (varB "B"))
  ((string= testval "y") (varB "error"))
  ((string= testval "z") (varB "null"))
  )
(pretty-print (string-append "varB post cond: " (varB)))

(when (string= testval "x")  (varC "C"))
(pretty-print (string-append "varC post if: " (varC)))
-------------env.scm end----------------------------------

Re: can't export conditional variable set with 'cond'

[Skyler Ferris]

Hi Daniel,

> I think this has something to do with compilation indeed. More specifically, it is caused by cross module inlining [1]. You probably need to declare your env module as not declarative by setting #:declarative? to #f inside the define-module form of env. I think the compiler inlines varB somehow. Not super sure, I am no Guile expert but you could inspect the assembly for main to see what the compiler did.
>
> 1: https://www.wingolog.org/archives/2021/05/13/cross-module-inlining-in-guile
> 2: https://www.gnu.org/software/guile/manual/html_node/Declarative-Modules.html

Interestingly, it's actually the testexport module that needs to be 
`#:declarative? #f`, not the env module. Which makes some sense after I 
think about it - it is the variables in the testexport module that need 
to get the updates not the env module. The mental model I use when 
thinking about guile imports (which may or may not correspond to the 
implementation) is that importing a variable is just defining a variable 
with that name, and it happens to refer to a memory location allocated 
by a different module. Then, assuming that the cause of the problem is 
inlining as you describe (which seems quite plausible), the memory 
locations are optimized away and the symbols lose their "declarativeness".

Part of me wants to say it would make sense for non-declarative modules 
to propagate their non-declarativeness to dependent modules. But this 
seems like it could cause a significant loss of optimization if there is 
one random module that needs to be non-declarative for some reason and 
happens to be near the root of a large tree. It might be easier to 
manage if declarativeness was associated with memory locations, instead 
of modules or (as the manual implies) symbolic definitions. But I know 
nothing about how declarativeness is implemented/tracked, so that might 
be a big ask.

RE: can't export conditional variable set with 'cond'

[M]

IMO this looks like a minimalised reproducer for a problem encountered in practice.

If you are going to reify variables, I would propose boxes instead, which simply hold a single value and hence are have very straightforward semantics and are very close to the semantics of variables.

Parameters have more involved semantics (reading and setting depends on the dynamic environment, see e.g. ‘parameterize’, also see the bit about conversion functions or whatever they are called), which might or might not be desired, depending on what you are doing. (With the provided code, it is hard to tell).

Best regards,
Maxime Devos

Re: can't export conditional variable set with 'cond'

[Skyler Ferris]

On 2/1/24 16:16, M wrote:

> If you are going to reify variables, I would propose boxes instead, which simply
> hold a single value and hence are have very straightforward semantics and are
> very close to the semantics of variables.
Thanks for bringing that up, I was previously unaware of boxes. For 
others who are also unaware, they are implemented in SRFI-111.

Re: can't export conditional variable set with 'cond'

[Mortimer Cladwell]

Thanks all.
Both parameters and boxes work for me with guile-3.0.9, which is what I run
on my local machine.

>> I get the impression that you were asking this more for curiosity about
how the language implementation works than to solve a problem

I am trying to solve a problem. I have old code running on a remote server
using guile-3.0.5. When I pull to local (guile-3.0.9) that code no longer
works using 3.0.9. I find the problem is that globals are not properly
exported. If I run my original test code on the server - works fine
(without boxes, parameters).  To clarify I am using:

-------------testexport.scm begin----------------------------------
(define-module (testexport)
  #:use-module (ice-9 pretty-print)
  #:use-module (env)
  )

(define (main)
  (pretty-print (string-append  "varA in main: " varA))
  (pretty-print (string-append  "varB in main: " varB))
  (pretty-print (string-append  "varC in main: " varC))
  )

(main)
-------------testexport.scm end----------------------------------

------------env.scm begin-----------------------------------
(define-module (env)
  #:use-module (ice-9 pretty-print)
  #:export(varA varB varC))

(define varA "A")
(define varB "")
(define varC "")
(define testval "x")

(cond
 ((string= testval "x") (set! varB "B"))
 ((string= testval "y") (set! varB "error"))
 ((string= testval "z") (set! varB "null"))
 )
(pretty-print (string-append "varB post cond: " varB))

(if (string= testval "x")  (set! varC "C"))
(pretty-print (string-append "varC post if: " varC))
-------------env.scm end----------------------------------


remote server running guile 2.2.7:

admin@ip-172-31-16-153:~/testexport$  guile -L . ./testexport.scm
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/admin/testexport/./testexport.scm
;;; compiling ./env.scm
;;; compiled
/home/admin/.cache/guile/ccache/2.2-LE-8-3.A/home/admin/testexport/env.scm.go
"varB post cond: B"
"varC post if: C"
;;; compiled
/home/admin/.cache/guile/ccache/2.2-LE-8-3.A/home/admin/testexport/testexport.scm.go
"varA in main: A"
"varB in main: B"
"varC in main: C"
admin@ip-172-31-16-153:~/testexport$ guile -v
guile (GNU Guile) 2.2.7
Copyright (C) 2020 Free Software Foundation, Inc.

License LGPLv3+: GNU LGPL 3 or later <http://gnu.org/licenses/lgpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

-----------------------------------------------------------------
remote server running guile-3.0.5:

admin@ip-172-31-16-153:~/testexport$
/gnu/store/q8brh7j5mwy0hbrly6hjb1m3wwndxqc8-guile-3.0.5/bin/guile -L .
./testexport.scm
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/admin/testexport/./testexport.scm
;;; compiling ./env.scm
;;; compiled
/home/admin/.cache/guile/ccache/3.0-LE-8-4.4/home/admin/testexport/env.scm.go
"varB post cond: B"
"varC post if: C"
;;; compiled
/home/admin/.cache/guile/ccache/3.0-LE-8-4.4/home/admin/testexport/testexport.scm.go
"varA in main: A"
"varB in main: B"
"varC in main: C"