Auto compile from many different languages that interoperates with guile

Auto compile from many different languages that interoperates with guile

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 2004 bytes --]

Hi,

I am maintaining a prolog->scheme compiler and a python->scheme compiler.
The nice thing with those implementation is that they work well with the
guile module system and are proper scheme functions and variables etc. So
python objects can be treated as goops objects and prolog predicates can be
used in kanren etc.

There is a headake though. When loading a module from one language to
another language the autocompilation fails. It would be nice to load a
python module and work with it from scheme land and vice versa.

One problem is the following funciton in system base compile

(define* (compile-file file #:key
                       (output-file #f)
                       (from (current-language))
                       (to 'bytecode)
                       (env (default-environment from))
                       (opts '())
                       (canonicalization 'relative))
...)

form is either specified or current-language and what I propose is to add a
knob that enables another version of the default for from e.g. something
like the following.

(define %extension-map '((("py" "python") python) ((("pl" "prolog") prolog)
(("scm") scheme)))
(define %use-extension-map #f)
(define (default-from-file file)
  (define default (current-language))
  (if %use-extension-map
       (let* ((ext   (get-extension file))
               (lang  (find-language ext %extension-map)))
           (if lang lang default))))

(define* (compile-file file #:key
                       (output-file #f)
                       (from (default-from file))
                       (to 'bytecode)
                       (env (default-environment from))
                       (opts '())
                       (canonicalization 'relative))

...)

I think that we already have variables that discovers the source files that
guile can compile and I don't think that we should get name clashes as long
as we use the prefix (languge prolog module) as a prefix for modules in
other languages than scheme.

WDYT

[-- Attachment #2: Type: text/html, Size: 2764 bytes --]

Re: Auto compile from many different languages that interoperates with guile

[Matt Wette]

> On Sep 1, 2017, at 1:45 PM, Stefan Israelsson Tampe <stefan.itampe@gmail.com> wrote:
> 
> Hi,
> 
> I am maintaining a prolog->scheme compiler and a python->scheme compiler. The nice thing with those implementation is that they work well with the guile module system and are proper scheme functions and variables etc. So python objects can be treated as goops objects and prolog predicates can be used in kanren etc.
> 
> There is a headake though. When loading a module from one language to another language the autocompilation fails. It would be nice to load a python module and work with it from scheme land and vice versa.
> 
> One problem is the following funciton in system base compile
> 
> (define* (compile-file file #:key
>                        (output-file #f)
>                        (from (current-language))
>                        (to 'bytecode)
>                        (env (default-environment from))
>                        (opts '())
>                        (canonicalization 'relative))
> ...)
> 
> form is either specified or current-language and what I propose is to add a knob that enables another version of the default for from e.g. something like the following.
> 
> (define %extension-map '((("py" "python") python) ((("pl" "prolog") prolog) (("scm") scheme)))
> (define %use-extension-map #f)
> (define (default-from-file file)
>   (define default (current-language))
>   (if %use-extension-map       
>        (let* ((ext   (get-extension file))
>                (lang  (find-language ext %extension-map)))
>            (if lang lang default))))
> 
> (define* (compile-file file #:key
>                        (output-file #f)
>                        (from (default-from file))
>                        (to 'bytecode)
>                        (env (default-environment from))
>                        (opts '())
>                        (canonicalization 'relative))
> 
> ...)
> 
> I think that we already have variables that discovers the source files that guile can compile and I don't think that we should get name clashes as long as we use the prefix (languge prolog module) as a prefix for modules in other languages than scheme.
> 
> WDYT

Will this handle all possible option processing?   

Another option is to change compile.scm to hand-off unknown extensions to another script using some convension.  
For example, for py files, compile.scm would call out compile-py.scm, and for prolog, compile-pl.scm.

Re: Auto compile from many different languages that interoperates with guile

[Matt Wette]

> On Sep 2, 2017, at 6:00 AM, Matt Wette <matt.wette@gmail.com> wrote:
> 
> 
>> On Sep 1, 2017, at 1:45 PM, Stefan Israelsson Tampe <stefan.itampe@gmail.com> wrote:
>> 
>> Hi,
>> 
>> I am maintaining a prolog->scheme compiler and a python->scheme compiler. The nice thing with those implementation is that they work well with the guile module system and are proper scheme functions and variables etc. So python objects can be treated as goops objects and prolog predicates can be used in kanren etc.
>> 
>> There is a headake though. When loading a module from one language to another language the autocompilation fails. It would be nice to load a python module and work with it from scheme land and vice versa.
>> 
>> One problem is the following funciton in system base compile
>> 
>> (define* (compile-file file #:key
>>                       (output-file #f)
>>                       (from (current-language))
>>                       (to 'bytecode)
>>                       (env (default-environment from))
>>                       (opts '())
>>                       (canonicalization 'relative))
>> ...)
>> 
>> form is either specified or current-language and what I propose is to add a knob that enables another version of the default for from e.g. something like the following.
>> 
>> (define %extension-map '((("py" "python") python) ((("pl" "prolog") prolog) (("scm") scheme)))
>> (define %use-extension-map #f)
>> (define (default-from-file file)
>>  (define default (current-language))
>>  (if %use-extension-map       
>>       (let* ((ext   (get-extension file))
>>               (lang  (find-language ext %extension-map)))
>>           (if lang lang default))))
>> 
>> (define* (compile-file file #:key
>>                       (output-file #f)
>>                       (from (default-from file))
>>                       (to 'bytecode)
>>                       (env (default-environment from))
>>                       (opts '())
>>                       (canonicalization 'relative))
>> 
>> ...)
>> 
>> I think that we already have variables that discovers the source files that guile can compile and I don't think that we should get name clashes as long as we use the prefix (languge prolog module) as a prefix for modules in other languages than scheme.
>> 
>> WDYT
> 
> Will this handle all possible option processing?   
> 
> Another option is to change compile.scm to hand-off unknown extensions to another script using some convension.  
> For example, for py files, compile.scm would call out compile-py.scm, and for prolog, compile-pl.scm.
> 

Oops -- never mind.   You are talking about the compile procedure; my mind was off in a differnent context: guild.

But there is an issue here, because I would like "guile -s foo.py" to interpret the file foo.py as Python and not Scheme.

Re: Auto compile from many different languages that interoperates with guile

[William ML Leslie]

On 2 September 2017 at 06:45, Stefan Israelsson Tampe
<stefan.itampe@gmail.com> wrote:
> form is either specified or current-language and what I propose is to add a
> knob that enables another version of the default for from e.g. something
> like the following.
>
> (define %extension-map '((("py" "python") python) ((("pl" "prolog") prolog)
> (("scm") scheme)))
> (define %use-extension-map #f)
> (define (default-from-file file)
>   (define default (current-language))
>   (if %use-extension-map
>        (let* ((ext   (get-extension file))
>                (lang  (find-language ext %extension-map)))
>            (if lang lang default))))
>

...

> I think that we already have variables that discovers the source files that
> guile can compile and I don't think that we should get name clashes as long
> as we use the prefix (languge prolog module) as a prefix for modules in
> other languages than scheme.
>
> WDYT

It's really nice that you are thinking about this.  Thank you for
suggesting one direction to move in.

I think python's own mechanism for having different relationships
between the module requested and where it is drawn from is quite
elegant.  In your solution you use the current language to favor a
list of file extensions over others, but I think you will also may
want to have different search paths per extension as well as different
module search algorithms and compile-time module generation.  Some big
python modules find out the ideal implementation for your system and
then replace themselves in the module list, for example, depending on
whether certain FFI modules are available or what operating system we
are running on.

The mechanism for doing this in python is a list of ``importers``.
Importers are tried in order and asked if they can provide the package
with a certain name.  If they can, they return a loader which can load
that package.  In a polyglot environment, it would be reasonable for
each language to have its own importer in the shared importer list,
and also have each language order its own importers first when that
language is making the initial request.

https://docs.python.org/3/library/importlib.html#module-importlib

It is an open question in my mind as to how dynamic the module search
system should be.  At the extreme end you have languages like TCL
which scare me a bit.  Similarly python had some great potential with
setuptools' ``require`` system to have multiple versions of a package
installed and have the application specify which version to run but it
was lacking just that last bit of tooling.  Still, I think importers
are a nice sweet spot at providing a cross-language module system.

-- 
William Leslie

Notice:
Likely much of this email is, by the nature of copyright, covered
under copyright law.  You absolutely MAY reproduce any part of it in
accordance with the copyright law of the nation you are reading this
in.  Any attempt to DENY YOU THOSE RIGHTS would be illegal without
prior contractual agreement.

Re: Auto compile from many different languages that interoperates with guile

[William ML Leslie]

On 8 September 2017 at 11:39, William ML Leslie
<william.leslie.ttg@gmail.com> wrote:
> https://docs.python.org/3/library/importlib.html#module-importlib
>

Hmm I could have said something similar about Java's classloaders or
OSGi or E's eMakers (eMakers are even better, though require more work
to feel guiley).

-- 
William Leslie

Notice:
Likely much of this email is, by the nature of copyright, covered
under copyright law.  You absolutely MAY reproduce any part of it in
accordance with the copyright law of the nation you are reading this
in.  Any attempt to DENY YOU THOSE RIGHTS would be illegal without
prior contractual agreement.

Re: Auto compile from many different languages that interoperates with guile

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 2469 bytes --]

FYI I just added generators with a small extension, to the python compiler,
now this works:

module(f)

def foreach(f,l):
   for x in l:
       f(x)

def gen(l):
    def f(x):
        yield(gen) x

    foreach(f,l)

for x in gen(list(1,2,3)):
    pk(x)

>> 1,2,3

Have fun


On Fri, Sep 1, 2017 at 10:45 PM, Stefan Israelsson Tampe <
stefan.itampe@gmail.com> wrote:

> Hi,
>
> I am maintaining a prolog->scheme compiler and a python->scheme compiler.
> The nice thing with those implementation is that they work well with the
> guile module system and are proper scheme functions and variables etc. So
> python objects can be treated as goops objects and prolog predicates can be
> used in kanren etc.
>
> There is a headake though. When loading a module from one language to
> another language the autocompilation fails. It would be nice to load a
> python module and work with it from scheme land and vice versa.
>
> One problem is the following funciton in system base compile
>
> (define* (compile-file file #:key
>                        (output-file #f)
>                        (from (current-language))
>                        (to 'bytecode)
>                        (env (default-environment from))
>                        (opts '())
>                        (canonicalization 'relative))
> ...)
>
> form is either specified or current-language and what I propose is to add
> a knob that enables another version of the default for from e.g. something
> like the following.
>
> (define %extension-map '((("py" "python") python) ((("pl" "prolog")
> prolog) (("scm") scheme)))
> (define %use-extension-map #f)
> (define (default-from-file file)
>   (define default (current-language))
>   (if %use-extension-map
>        (let* ((ext   (get-extension file))
>                (lang  (find-language ext %extension-map)))
>            (if lang lang default))))
>
> (define* (compile-file file #:key
>                        (output-file #f)
>                        (from (default-from file))
>                        (to 'bytecode)
>                        (env (default-environment from))
>                        (opts '())
>                        (canonicalization 'relative))
>
> ...)
>
> I think that we already have variables that discovers the source files
> that guile can compile and I don't think that we should get name clashes as
> long as we use the prefix (languge prolog module) as a prefix for modules
> in other languages than scheme.
>
> WDYT
>

[-- Attachment #2: Type: text/html, Size: 3693 bytes --]

Re: Auto compile from many different languages that interoperates with guile

[William ML Leslie]

On 9 September 2017 at 08:25, Stefan Israelsson Tampe
<stefan.itampe@gmail.com> wrote:
> def gen(l):
>     def f(x):
>         yield(gen) x
>

Whoa.  You're yielding from a different frame?!

-- 
William Leslie

Notice:
Likely much of this email is, by the nature of copyright, covered
under copyright law.  You absolutely MAY reproduce any part of it in
accordance with the copyright law of the nation you are reading this
in.  Any attempt to DENY YOU THOSE RIGHTS would be illegal without
prior contractual agreement.

Re: Auto compile from many different languages that interoperates with guile

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 3196 bytes --]

Hi, I've also playing with functional objects e.g. objects that updates
functionally meaning that the objects are non mutating. I added a little
suger to handle this in my python to prolog compiler and now we can do this:

module(f)

# Note that the functional parent means that the class is managed using
# functional approach this also constrain that all parent classes is also
# functional else bugs will appear.

class A(functional):
    x = 1
    def __init__(self,y):
        pk('A',self.x)
        self.x = y

class B(functional):
    y = A(1)
    def __init__(self,y):
        self.y=y

a = A(2)
b1 = B(a)

# note that when setting things we update create a new object and this is
the
# syntactic suger
b2 : b1.y.x = 3  # translates to (set! b2 (fset-x b1 '(y x) 3))
b3 : b2.y.y = 4

pk("b1",b1.y.x)
pk("b2",b2.y.x)
pk("b3",b3.y.y)

# there is a nice scheme interface to use this in pf-objects library in the
# code base

>>
;;; ("A" 1)

;;; ("A" 1)

;;; ("b1" 2)

;;; ("b2" 3)

;;; ("b3" 4)


On Fri, Sep 1, 2017 at 10:45 PM, Stefan Israelsson Tampe <
stefan.itampe@gmail.com> wrote:

> Hi,
>
> I am maintaining a prolog->scheme compiler and a python->scheme compiler.
> The nice thing with those implementation is that they work well with the
> guile module system and are proper scheme functions and variables etc. So
> python objects can be treated as goops objects and prolog predicates can be
> used in kanren etc.
>
> There is a headake though. When loading a module from one language to
> another language the autocompilation fails. It would be nice to load a
> python module and work with it from scheme land and vice versa.
>
> One problem is the following funciton in system base compile
>
> (define* (compile-file file #:key
>                        (output-file #f)
>                        (from (current-language))
>                        (to 'bytecode)
>                        (env (default-environment from))
>                        (opts '())
>                        (canonicalization 'relative))
> ...)
>
> form is either specified or current-language and what I propose is to add
> a knob that enables another version of the default for from e.g. something
> like the following.
>
> (define %extension-map '((("py" "python") python) ((("pl" "prolog")
> prolog) (("scm") scheme)))
> (define %use-extension-map #f)
> (define (default-from-file file)
>   (define default (current-language))
>   (if %use-extension-map
>        (let* ((ext   (get-extension file))
>                (lang  (find-language ext %extension-map)))
>            (if lang lang default))))
>
> (define* (compile-file file #:key
>                        (output-file #f)
>                        (from (default-from file))
>                        (to 'bytecode)
>                        (env (default-environment from))
>                        (opts '())
>                        (canonicalization 'relative))
>
> ...)
>
> I think that we already have variables that discovers the source files
> that guile can compile and I don't think that we should get name clashes as
> long as we use the prefix (languge prolog module) as a prefix for modules
> in other languages than scheme.
>
> WDYT
>

[-- Attachment #2: Type: text/html, Size: 4814 bytes --]