unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
* Summer of Code Recap
@ 2017-08-28 18:56 Ian Price
  2017-08-28 19:25 ` Christopher Allan Webber
                   ` (3 more replies)
  0 siblings, 4 replies; 16+ messages in thread
From: Ian Price @ 2017-08-28 18:56 UTC (permalink / raw)
  To: guile-user

1 Introduction
==============

  As many of you are aware, I have been working on compiling Guile
  Scheme to JavaScript this summer, as part of the Google Summer of
  Code. This post serves to bookend my work for the year.

  Before I go any further, I have to give my thanks to my mentor [Chris
  Webber], without whom this project would have fizzled out weeks ago;
  Google and the Gnu Project, naturally, for providing the Summer of
  Code and allowing me to work on this project; and our fearless leader,
  [Andy Wingo], for answering a wide variety of stupid questions.


[Chris Webber] https://dustycloud.org/

[Andy Wingo] https://wingolog.org/


2 Project Aims
==============

  For a full introduction to the project, you can of course refer back
  to my [project proposal], but very briefly my hopes for this summer
  were:

  1. To rewrite the previous version of my compiler from the [previous
     CPS representation] to use the new representation ["CPS Soup"]
     representation.
  2. To completely port ice-9/boot-9.scm (our basic "prelude") to
     JavaScript, and in particular, to support the [Guile Module
     system].
  3. To handle Proper Tail Calls by use of the [Cheney on the MTA]
     strategy.
  4. To include a new `guild' script for bundling compiled JS files with
     their dependencies.


[project proposal] https://shift-reset.com/static/docs/gsoc-2017.pdf

[previous CPS representation]
https://wingolog.org/archives/2014/01/12/a-continuation-passing-style-intermediate-language-for-guile

["CPS Soup"] https://wingolog.org/archives/2015/07/27/cps-soup

[Guile Module system]
https://www.gnu.org/software/guile/manual/html_node/Modules.html#Modules

[Cheney on the MTA] http://www.pipeline.com/~hbaker1/CheneyMTA.html


3 What was Achieved
===================

  You can find all of my work on the [compile-to-js-2017] branch of my
  Gitlab. A full list of the commits can be found [here], but I will
  summarise the changes now:


[compile-to-js-2017]
https://gitlab.com/ijp/guile/tree/compile-to-js-2017

[here] https://gitlab.com/ijp/guile/compare/1b36a76e...gsoc-2017-end

3.1 Compile Guile CPS Soup to JavaScript
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  When I was working on my initial attempt at compiling Guile to
  JavaScript, two years ago, Guile used a different CPS representation
  as its intermediate language. The initial experiments with the CPS
  Soup representation occurred while that work was ongoing, but as it
  was not considered "stable", the plan was not to move to this
  representation until after I had completed my other objectives.

  Now, however, CPS Soup is the IL of Guile, and so the first task that
  was accomplished was to move to this representation. Since I had
  already created my own JS-IL as a target, I did not need to make any
  changes to the code generation side from JS-IL to JavaScript proper.
  The main change was to reconstruct the nested scope structure that was
  implicit in the dominator structure that Guile made available.

  The full code for the compiler is split into several sections,
  corresponding to different stages in the compiler pipeline.


3.1.1 CPS to JS-IL Compiler
---------------------------

  - module/language/cps/compile-js.scm
  - module/language/cps/spec.scm

  These modules constitute the compiler from CPS to my JS-IL
  intermediate language.


3.1.2 JS-IL to JavaScript Compiler
----------------------------------

  - module/language/js-il.scm
  - module/language/js-il/compile-javascript.scm
  - module/language/js-il/inlining.scm
  - module/language/js-il/spec.scm

  These modules constitute a somewhat ad-hoc intermediate representation
  as a target for the CPS compiler. It differs from JavaScript, e.g., by
  continuing to separate continuations and functions, and a slightly
  specialised function representation to handle Guile's complicated
  notion of procedure arity.


3.1.3 JavaScript Representation
-------------------------------

  - module/language/javascript.scm
  - module/language/javascript/simplify.scm
  - module/language/javascript/spec.scm

  This is primarily the representation of JavaScript as Scheme Records.
  This is separate from the representation of JavaScript Guile already
  has in the form of `(language ecmascript)' primarily to avoid a
  circularity when Guile determines which compilers to run in the
  pipeline, as recommended by Andy Wingo.


3.2 A pre-amble capable of running through boot-9
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  In order to run Guile, it is not enough to be able to compile Scheme
  (or indeed any other language supported by Guile) forms to JavaScript,
  we also need to incorporate as much of Guile's runtime as possible.
  This involves implementing VM primitives (such as you might see in
  vm-engine.c); basic Guile types like Symbols, Pairs, and Structs; as
  well as many of the functions that Guile implements in C rather than
  Scheme.

  Although I certainly did not implement all of the functionality Guile
  achieves, I was able to implement sufficiently many (including what
  amounts to a port of much of module.c) that one can successfully run
  though ice-9/boot-9.scm from start to finish.

  This took up the bulk of the time I spent on this project, due to the
  size of the compiled output of boot-9.scm, and my own difficulties
  debugging the bootstrap process. More on this below.

  The code can be found at
  - module/language/js-il/runtime.js


3.3 A linking script for JavaScript
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  Since we are using the `(language ...)' infrastructure, we can take
  advantage of the existing `guild compile' script for compiling to
  JavaScript, we simply need to use the `--to' switch. However, this
  does not produce a file which you can just load up without any
  additional work, especially if you are working with multiple modules.

  In order to make it easier to deal with this, I have included a `guild
  jslink' script, which can be used to package up a "main" script along
  with the `runtime.js' and its dependencies. See below for an example.

  The code can be found at
  - module/scripts/jslink.scm


4 What was not Achieved
=======================

4.1 Cheney on the MTA
~~~~~~~~~~~~~~~~~~~~~

  One of my regrets is that I did not implement Baker's "Cheney on the
  MTA" (as seen in [Chicken Scheme]) for handling Proper Tail Calls in
  JavaScript. Historically, JavaScript has not guaranteed that tail
  position function calls do not grow the stack, and this is obviously
  of fundamental importance for languages like Scheme. Fortunately, ES6
  has added support for [proper tail calls] and we can expect to see
  increased support for it in future JavaScript versions. (Indeed,
  during testing on node v.6.10.3, I did not have to increase the stack
  size until very late).


[Chicken Scheme] https://www.call-cc.org/

[proper tail calls]
https://www.ecma-international.org/ecma-262/6.0/#sec-tail-position-calls


5 How to use it
===============

  I've talked a lot about what I've did and didn't do, but what about
  actually using this thing?


5.1 Obtaining the Code
~~~~~~~~~~~~~~~~~~~~~~

  The code is not currently available from the main Guile repository,
  but only the `compile-to-js-2017' branch on my [GitLab].

  If you already have a checkout of guile, you can add my repo as a
  remote with
  ,----
  | $ git remote add ijp https://gitlab.com/ijp/guile.git
  `----
  and fetch the branch with
  ,----
  | $ git fetch ijp
  `----

  You can then check out the `compile-to-js-2017' branch and build as
  normal.


[GitLab] https://gitlab.com/ijp/guile/tree/compile-to-js-2017


5.2 A Non-Trivial Example
~~~~~~~~~~~~~~~~~~~~~~~~~

  As an example of how to use the JS Backend that is short, but
  non-trivial, I am using John McCarthy's `amb' operator (see [A Basis
  for a Mathematical Theory of Computation]) to search for Pythagorean
  Triples.

  First we have a module for the `amb' operator in amb.scm
  ,----
  | (define-module (amb)
  |   #:export (amb fail))
  |
  | (define original-fail
  |   (lambda _
  |     (error 'amb "No more paths to search")))
  |
  | (define *amb-fail* original-fail)
  |
  | (define (fail)
  |   (*amb-fail* #f))
  |
  | (define (amb-thunks . values)
  |   (let ((failure *amb-fail*))
  |     (call/cc (lambda (escape)
  |                (for-each (lambda (value)
  |                            (call/cc (lambda (continue)
  |                                       (set! *amb-fail* continue)
  |                                       (escape (value)))))
  |                          values)
  |                (failure #f)))))
  |
  | (define-syntax amb
  |   (syntax-rules ()
  |     ((amb exprs ...)
  |      (amb-thunks (lambda () exprs) ...))))
  `----

  Next we have the code performs the search in triple.scm
  ,----
  | (use-modules (amb))
  |
  | (let ((a (amb 4 5 6 7 8 9 10))
  |       (b (amb 4 5 6 7 8 9 10))
  |       (c (amb 4 5 6 7 8 9 10)))
  |   (if (= (* c c) (+ (* a a) (* b b)))
  |       (list a b c)
  |       (fail)))
  `----

  We compile the files in the usual manner, only now we specify the
  `javascript' language (We make sure to add the current directory to
  the load-path for triple.scm).

  ,----
  | $ guild compile amb.scm --to=javascript --output=amb.js
  | $ guild compile -L . triple.scm --to=javascript --output=triple.js
  `----

  Next we link the two together into a file main.js, making sure to
  specify amb.js as a dependency of triple.js. (This step will take a
  little while, since it also compiles a bunch of dependencies)

  ,----
  | $ guild jslink triple.js -o main.js --depends="(\"amb\" . \"amb.scm\")"
  `----

  Finally, you can run it with `node', although as mentioned above you
  may have to increase the stack size.

  ,----
  | $ node  --stack-size=2000 main.js
  `----

  Which should, fingers crossed, print out the triple 6,8,10.


[A Basis for a Mathematical Theory of Computation]
http://www-formal.stanford.edu/jmc/basis1.pdf


6 What is next?
===============

  Having recapped what was and what was not achieved, the next question
  is: where does the project go from here? I have been asked about my
  plans for all sorts of features, e.g. support for [Web Assembly], but
  I think the following things are the most important to think about.


[Web Assembly] http://webassembly.org/

6.1 Inclusion into Guile
~~~~~~~~~~~~~~~~~~~~~~~~

  The entire point of the project is to have something that can be
  included in Guile proper. I have not spoken with Guile's maintainers
  about incorporation into the main distribution, but I expect there
  would be not be too many problems with moving the "official branch" to
  the main repository.


6.2 All Guile built-ins in runtime.js
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  Although I have included enough to get though boot-9.scm, this does
  not include all of the built-ins we would want in our programs. Two
  things I use very often which do not appear in runtime.js are ports
  and bytevectors.

  We would like most, if not all, Guile built-ins to be available for
  those who need them, so these will need to be implemented. However,
  this is a lot of extra code for some people who don't need it, which
  brings us to a different issue...


6.3 Linking Guile Modules & Features
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  In [a blog post], Andy Wingo lays out many tasks that he would like to
  see in a future Guile. One of the most important of these, for us, are
  under the headings "linking multiple modules together" and "linking a
  single executable". To grossly simplify, we want to be able to link
  various files into one single executable, which contains all and only
  the code we need for our application.

  As it stands, I included a simple script `guild jslink' that bundles
  various compiled JavaScript files into one file, but we would like it
  to be much more featureful: removing modules, functions, even types we
  don't need; and inferring which modules are required by our
  application and bundling them without requiring the information
  `jslink' does. This would allow us to minimise the amount of code that
  needs to be sent over the network, which is very important to web
  developers.

  This is a large task, and one I don't know enough about at the moment
  to attempt, but it is work that would benefit not just our JavaScript
  compiler, but people who want to deploy regular Guile applications.


[a blog post]
https://wingolog.org/archives/2016/02/04/guile-compiler-tasks


6.4 JavaScript Version
~~~~~~~~~~~~~~~~~~~~~~

  I am not an expert in JavaScript, in fact, before this summer I
  probably hadn't written it for two years, which means the code
  certainly does not match up with the current best practices and
  specifications. Further, all of my testing for this compiler was done
  on [Node.js] v.6.10.3 only (this was the version available in the
  Fedora 25 repositories).

  The code should be vetted to determine precisely which modern JS
  features are used (I believe proper tail calls, and ES6 Maps are the
  main ones), and it should be tested on all major browsers. If
  necessary, we should incorporate switches in the compiler to allow JS
  users to compile for particular implementations, taking advantage of
  particular modern JS features, or providing our own implementations of
  those that are not supported (e.g. Cheney on the MTA).


[Node.js] https://nodejs.org/en/


6.5 JS Integration
~~~~~~~~~~~~~~~~~~

  One of the strengths of Guile is that it allows people to integrate
  their Scheme and C code, and although it has not been a focus for this
  summer, we should aim to provide similar levels of integration between
  Scheme and JS. There are two cases to consider.


6.5.1 JS calling Scheme
-----------------------

  As it stands, you can perform some limited interaction from JavaScript
  in a similar manner to how you would interact with Guile from C. For
  instance, by using `scm_current_module', `scm_public_lookup', and the
  `scheme.Symbol' constructor, one could look up a scheme function, e.g.
  `iota', and then invoke it by `scheme.call'.

  That said, C idioms are not JS idioms, and so we should work to
  provide a much nicer API through the `scheme' object.


6.5.2 Scheme calling JS
-----------------------

  In the case of Scheme calling JavaScript, I think we should follow the
  example of `(system foreign)', which provides an API for linking to
  dynamic C libraries, and creating Scheme versions of C functions, and
  automatically marshalling/unmarshalling C types to Scheme types. One
  additional complication we would have with JS would be the presence of
  exceptions, but I think these could also be marshalled into Scheme
  ones without much trouble.


7 Lessons Learned
=================

  It goes without saying that a project like this teaches you a lot
  about the technical design of Guile, how to navigate the codebase,
  etc, but I want to highlight a few "softer" lessons from this summer.


7.1 Compilers are "Easy", Runtimes are Hard
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  When I first set out to write this project two summers ago, I
  naturally assumed that the majority of the effort would go into the
  compiler, and much less into the built-ins. In reality, the effort was
  reversed. Partly this was due to my experience in writing Scheme, and
  Functional Programming more generally, meant that the tree-traversing
  code typical of a compiler pass was relatively straightforward, and
  the compiler was not doing a lot of optimisation, mostly code
  generation.


7.2 Bootstrapping is Hard
~~~~~~~~~~~~~~~~~~~~~~~~~

  The last point leads into this one, bootstrapping is pretty tricky.
  With boot-9, you have several versions of the module system at
  different times. My own attempt to write module code that handled this
  ended up being abandoned for a rewrite that more closely followed the
  Guile C code. The size of the compiled boot-9 code, and the, at times,
  non-local consequences of implementing certain built-ins made it
  tricky to debug.


7.3 Don't Panic
~~~~~~~~~~~~~~~

  This is a much more personal one, and one that I think is very
  important for anyone who wants to take part in a program like the
  Summer of Code, where you are spending a lot of time mostly on your
  own. In a complex software project, things are not always going to go
  smoothly. You might spend weeks banging up against a difficult
  problem. Don't Panic! If it was easy it would have already been done.
  Keep in Contact with your Mentor! It is tempting to only check in when
  you think you have something of progress to report, but they are there
  to help you, and explaining your issues to someone else is often very
  useful when trying to overcome them, even if they don't have an answer
  for you.


8 Wrapping Up
=============

  If you are still with me, good on you. As the new semester is starting
  I will be devoting much less time to this, and that will likely be
  true till December, but I will make an effort to keep up with
  guile-user and be on the IRC Channel to help the daring souls who want
  to give this a go. My priorities will be documenting the ILs, filling
  in missing builtins, and improving jslink. I especially want to see
  basic IO and MiniKanren up and running, and for it to be convenient to
  use Guile's builtin libraries.


Happy Hacking, Ian Price

(This is a crosspost to guile-user of my blogpost [Summer of Code
Recap], but please comment on this list, rather than there)

[Summer of Code Recap]
https://shift-reset.com/blog/2017/8/28/Summer%20of%20Code%20Recap/


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-08-28 18:56 Summer of Code Recap Ian Price
@ 2017-08-28 19:25 ` Christopher Allan Webber
  2017-08-28 19:58   ` Nala Ginrut
  2017-08-28 21:39 ` Amirouche
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 16+ messages in thread
From: Christopher Allan Webber @ 2017-08-28 19:25 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user

Thanks again Ian for all your work on this; I've been very proud to be
your mentor.  Great work, and look forward to seeing this land in Guile
proper :)



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-08-28 19:25 ` Christopher Allan Webber
@ 2017-08-28 19:58   ` Nala Ginrut
  0 siblings, 0 replies; 16+ messages in thread
From: Nala Ginrut @ 2017-08-28 19:58 UTC (permalink / raw)
  To: Christopher Allan Webber; +Cc: Guile User, Ian Price

Thanks for all the work!
I will be the potential user since I need a Scheme way for web front-end
and an easy way to integrate it into Artanis. Biwascheme is one of the
ways, now we have JS back end for another choice. I expect there're good
interfaces to interact with DOMs.
I will keep my eyes on this project to see what's going on.
Thanks again!

Best regards.



2017年8月29日 上午3:25,"Christopher Allan Webber" <cwebber@dustycloud.org>写道:

Thanks again Ian for all your work on this; I've been very proud to be
your mentor.  Great work, and look forward to seeing this land in Guile
proper :)


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-08-28 18:56 Summer of Code Recap Ian Price
  2017-08-28 19:25 ` Christopher Allan Webber
@ 2017-08-28 21:39 ` Amirouche
  2017-08-29  3:27   ` Christopher Allan Webber
  2017-09-01 22:09 ` Amirouche Boubekki
  2021-05-11 14:45 ` Christopher Lemmer Webber
  3 siblings, 1 reply; 16+ messages in thread
From: Amirouche @ 2017-08-28 21:39 UTC (permalink / raw)
  To: guile-user

Héllo,


Le 28/08/2017 à 20:56, Ian Price a écrit :
> 1 Introduction
> ==============
>
>    As many of you are aware, I have been working on compiling Guile
>    Scheme to JavaScript this summer, as part of the Google Summer of
>    Code. This post serves to bookend my work for the year.
[...]
> 2 Project Aims
> ==============
>
>    For a full introduction to the project, you can of course refer back
>    to my [project proposal], but very briefly my hopes for this summer
>    were:
>
>    1. To rewrite the previous version of my compiler from the [previous
>       CPS representation] to use the new representation ["CPS Soup"]
>       representation.
>    2. To completely port ice-9/boot-9.scm (our basic "prelude") to
>       JavaScript, and in particular, to support the [Guile Module
>       system].
>    3. To handle Proper Tail Calls by use of the [Cheney on the MTA]
>       strategy.
>    4. To include a new `guild' script for bundling compiled JS files with
>       their dependencies.
>
>
> [project proposal] https://shift-reset.com/static/docs/gsoc-2017.pdf
>
> 3 What was Achieved
> ===================
[...]
> 3.1 Compile Guile CPS Soup to JavaScript
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[...]
> 3.3 A linking script for JavaScript
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[...]
> 4 What was not Achieved
> =======================
>
> 4.1 Cheney on the MTA
> ~~~~~~~~~~~~~~~~~~~~~
[...]
> 6 What is next?
> ===============
>
>    Having recapped what was and what was not achieved, the next question
>    is: where does the project go from here? I have been asked about my
>    plans for all sorts of features, e.g. support for [Web Assembly], but
>    I think the following things are the most important to think about.
>
>
> [Web Assembly] http://webassembly.org/
>
> 6.1 Inclusion into Guile
> ~~~~~~~~~~~~~~~~~~~~~~~~
>
>    The entire point of the project is to have something that can be
>    included in Guile proper. I have not spoken with Guile's maintainers
>    about incorporation into the main distribution, but I expect there
>    would be not be too many problems with moving the "official branch" to
>    the main repository.

Very important, but not before we know it can be useful.
> 6.2 All Guile built-ins in runtime.js
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>    Although I have included enough to get though boot-9.scm, this does
>    not include all of the built-ins we would want in our programs. Two
>    things I use very often which do not appear in runtime.js are ports
>    and bytevectors.
I don't mind not using ports and bytevectors. I am not using them in 
BiwaScheme.
>    We would like most, if not all, Guile built-ins to be available for
>    those who need them, so these will need to be implemented. However,
>    this is a lot of extra code for some people who don't need it, which
>    brings us to a different issue...

> 6.3 Linking Guile Modules & Features
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>    In [a blog post], Andy Wingo lays out many tasks that he would like to
>    see in a future Guile. One of the most important of these, for us, are
>    under the headings "linking multiple modules together" and "linking a
>    single executable". To grossly simplify, we want to be able to link
>    various files into one single executable, which contains all and only
>    the code we need for our application.

I told you before, I think that this was not an important matter
because it can be handled by Javascript tooling for the time being.
Even if, I agree that in a ideal world, it must not require javascript
third parties except maybe for optimizations.

>    This would allow us to minimise the amount of code that
>    needs to be sent over the network, which is very important to web
>    developers.

You seem concerned by javascript web developers but not about
me in particular. As the maintainer of the only Scheme frontend
web framework, as a Guile user and the creator *one* of the most
successful Python -> Javascript compiler you could have consulted
me on what is best for GNU Guile in the regard of Guile -> Javascript.

That said, I am not surprised of your behavior nor the behavior of
your mentor, since it's not the first time you ignore me by mistake
or on purpose.

I think you should have asked me what would be best to prototype
the compiler.

Sorry, I will do nonetheless.
>    This is a large task, and one I don't know enough about at the moment
>    to attempt, but it is work that would benefit not just our JavaScript
>    compiler, but people who want to deploy regular Guile applications.
At least for javascript tooling is doing that for a decade. So, in my 
opinion
this is not the most important matter. See "tree shaking".

> [a blog post]
> https://wingolog.org/archives/2016/02/04/guile-compiler-tasks
>
>
> 6.4 JavaScript Version
> ~~~~~~~~~~~~~~~~~~~~~~
>
>    I am not an expert in JavaScript, in fact, before this summer I
>    probably hadn't written it for two years, which means the code
>    certainly does not match up with the current best practices and
>    specifications. Further, all of my testing for this compiler was done
>    on [Node.js] v.6.10.3 only (this was the version available in the
>    Fedora 25 repositories).
>
>    The code should be vetted to determine precisely which modern JS
>    features are used (I believe proper tail calls, and ES6 Maps are the
>    main ones), and it should be tested on all major browsers. If
>    necessary, we should incorporate switches in the compiler to allow JS
>    users to compile for particular implementations, taking advantage of
>    particular modern JS features, or providing our own implementations of
>    those that are not supported (e.g. Cheney on the MTA).

This is overkill. babel [https://babeljs.io] or traceur [*] can do that 
for us.

[*] https://github.com/google/traceur-compile

> 6.5 JS Integration
> ~~~~~~~~~~~~~~~~~~
[...]
> 6.5.1 JS calling Scheme
> -----------------------
>
>    As it stands, you can perform some limited interaction from JavaScript
>    in a similar manner to how you would interact with Guile from C. For
>    instance, by using `scm_current_module', `scm_public_lookup', and the
>    `scheme.Symbol' constructor, one could look up a scheme function, e.g.
>    `iota', and then invoke it by `scheme.call'.
>
>    That said, C idioms are not JS idioms, and so we should work to
>    provide a much nicer API through the `scheme' object.

Not very useful, except if you achieve Javascript speed in Guile 
compiled code.

> 6.5.2 Scheme calling JS
> -----------------------
>
>    In the case of Scheme calling JavaScript, I think we should follow the
>    example of `(system foreign)', which provides an API for linking to
>    dynamic C libraries, and creating Scheme versions of C functions, and
>    automatically marshalling/unmarshalling C types to Scheme types. One
>    additional complication we would have with JS would be the presence of
>    exceptions, but I think these could also be marshalled into Scheme
>    ones without much trouble.

marshalling and unmarshalling types is a pain performance wise. I was 
hoping
that calling Javascript functions would be native with zero overhead. It 
seems
like it's not the case. I hope that at least string won't require 
complex code.

> 7 Lessons Learned
> =================
[...]
> 8 Wrapping Up
> =============
>
>    If you are still with me, good on you. As the new semester is starting
>    I will be devoting much less time to this, and that will likely be
>    true till December, but I will make an effort to keep up with
>    guile-user and be on the IRC Channel to help the daring souls who want
>    to give this a go. My priorities will be documenting the ILs, filling
>    in missing builtins, and improving jslink. I especially want to see
>    basic IO and MiniKanren up and running, and for it to be convenient to
>    use Guile's builtin libraries.
>
>
> Happy Hacking, Ian Price

I hope that in the future, you will be more inclusive.

Great achievement!



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-08-28 21:39 ` Amirouche
@ 2017-08-29  3:27   ` Christopher Allan Webber
  0 siblings, 0 replies; 16+ messages in thread
From: Christopher Allan Webber @ 2017-08-29  3:27 UTC (permalink / raw)
  To: Amirouche; +Cc: guile-user

Amirouche writes:

>>    This would allow us to minimise the amount of code that
>>    needs to be sent over the network, which is very important to web
>>    developers.
>
> You seem concerned by javascript web developers but not about
> me in particular. As the maintainer of the only Scheme frontend
> web framework, as a Guile user and the creator *one* of the most
> successful Python -> Javascript compiler you could have consulted
> me on what is best for GNU Guile in the regard of Guile -> Javascript.
>
> That said, I am not surprised of your behavior nor the behavior of
> your mentor, since it's not the first time you ignore me by mistake
> or on purpose.
>
> I think you should have asked me what would be best to prototype
> the compiler.

Ian put in a ton of work on a really difficult task, and the work was
out there and open to feedback.  Updates were posted to this list on
which you could have commented.  It is even *still* open for feedback
and input and etc.  But they're also the implementor, and thus have more
leeway on decisions.

I also know that tasks like these tend to be an enormous emotional
effort.  Please be considerate to not douse the accomplishment.



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-08-28 18:56 Summer of Code Recap Ian Price
  2017-08-28 19:25 ` Christopher Allan Webber
  2017-08-28 21:39 ` Amirouche
@ 2017-09-01 22:09 ` Amirouche Boubekki
  2017-09-02  6:58   ` Arne Babenhauserheide
  2017-09-06 18:25   ` Amirouche Boubekki
  2021-05-11 14:45 ` Christopher Lemmer Webber
  3 siblings, 2 replies; 16+ messages in thread
From: Amirouche Boubekki @ 2017-09-01 22:09 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user, guile-user

[-- Attachment #1: Type: text/plain, Size: 2933 bytes --]

On 2017-08-28 20:56, Ian Price wrote:

> 1 Introduction
> ==============
> 
>   As many of you are aware, I have been working on compiling Guile
>   Scheme to JavaScript this summer, as part of the Google Summer of
>   Code. This post serves to bookend my work for the year.
> 
>   Before I go any further, I have to give my thanks to my mentor [Chris
>   Webber], without whom this project would have fizzled out weeks ago;
>   Google and the Gnu Project, naturally, for providing the Summer of
>   Code and allowing me to work on this project; and our fearless 
> leader,
>   [Andy Wingo], for answering a wide variety of stupid questions.
> 

Sorry, for my last mail I was a bit upset a minimal scm file with "42"
will crash.

Anyway, I did a bit of exploration:

Apparently webkit does ship TCO and safari is the only browser [0] 
shipping it.
So it will happen soon enough for chrome, but chrome is not a solution 
for many
of us. I don't know when firefox with ship TCO.

[0] https://www.chromestatus.com/feature/5516876633341952

Anyway, TCO requires "use strict"; thing to work, so I prefixed the 
output
of guild jslink with that and it lead to discover a bug where there is 
two
times the same argument called `v_rest`. You can run the file with
the following command:

    node --harmony_tailcalls main.js

It will tell you where the error is.

Here is the fixed code:

     var k_27363 = function(v_19412, k_27499) {
       var k_27364 = function(v_19412, v_e, v_p, v_w, v_mod) {
         var v_19413 = v_e;
         var v_19414 = v_p;
         var v_19415 = v_w;
         var v_19416 = v_mod;
         if (scheme["is_true"](scheme["primitives"]["pair?"](v_19413))) {
           {
             var v_19417 = scheme["primitives"]["car"](v_19413);
             var v_19418 = scheme["EMPTY"];
             scheme["primitives"]["handle-interrupts"]();
             var k_27370 = function(v_first, v_rest) {
               var v_19419 = v_first;
               var v_19420 = v_rest;
               if (scheme["is_true"](v_19419)) {
                 {
                   var v_19421 = scheme["primitives"]["cdr"](v_19413);
                   scheme["primitives"]["handle-interrupts"]();
                   var k_27375 = function(v_rest, v_rest2) {  // The 
error is HERE!
                     var v_19422 = v_rest;
                     var v_19423 = v_rest2;

With that "fix" it run under nodejs v8.4.0 without increasing the 
callstack.
So it's good news!

I can't make it work with Google Chrome Version 60.0.3112.113 which use
a more recent version of v8 than my nodejs.

chrome://flags/#enable-javascript-harmony

The babel-plugin-tailcall-optimization doesn't work on my machine. It 
crash
with an OOM failure.

Also Google's traceur does not work.

Finally, there is this heavy page that is supposed to tell what is 
supported:

https://kangax.github.io/compat-table/es6/#test-proper_tail_calls_(tail_call_optimisation)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-add-a-few-ffi-procedures-inspired-from-biwascheme.patch --]
[-- Type: text/x-diff; name=0002-add-a-few-ffi-procedures-inspired-from-biwascheme.patch, Size: 1203 bytes --]

From 4ac41ad82300b5f35862d51ebac7c661f526809a Mon Sep 17 00:00:00 2001
From: amirouche <amirouche+dev@hypermove.net>
Date: Fri, 1 Sep 2017 21:58:45 +0200
Subject: [PATCH 2/2] add a few ffi procedures inspired from biwascheme

---
 module/language/js-il/runtime.js | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/module/language/js-il/runtime.js b/module/language/js-il/runtime.js
index 583f21d93..26090055e 100644
--- a/module/language/js-il/runtime.js
+++ b/module/language/js-il/runtime.js
@@ -926,6 +926,24 @@ function scm_list (self, cont) {
 };
 def_guile0("list", scm_list);
 
+def_guile0("js-eval", function(self, cont, string) {
+  return cont(eval(string.s))
+});
+
+def_guile0("js-ref", function(self, cont, js, attrname) {
+  return cont(js[attrname.s]);
+});
+
+def_guile0("js-invoke", function(self, cont, jsobj, methodname) {
+  var args = Array.prototype.slice.call(arguments).slice(4);
+  cont(jsobj[methodname.s].apply(jsobj, args));
+});
+
+def_guile0("js-call", function(self, cont, jsfunc) {
+  var args = Array.prototype.slice.call(arguments).slice(3);
+  cont(jsfunc.apply(undefined, args));
+});
+
 // Numbers
 function scm_add(self, cont) {
 
-- 
2.11.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-always-throw-an-Error-object-to-have-a-nice-tracebac.patch --]
[-- Type: text/x-diff; name=0001-always-throw-an-Error-object-to-have-a-nice-tracebac.patch, Size: 715 bytes --]

From 0f4fdac6347631b8e12d02a91f5ff40d056dfbd9 Mon Sep 17 00:00:00 2001
From: amirouche <amirouche+dev@hypermove.net>
Date: Fri, 1 Sep 2017 21:09:36 +0200
Subject: [PATCH 1/2] always throw an Error object to have a nice traceback

---
 module/language/js-il/runtime.js | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/module/language/js-il/runtime.js b/module/language/js-il/runtime.js
index 142e3db86..583f21d93 100644
--- a/module/language/js-il/runtime.js
+++ b/module/language/js-il/runtime.js
@@ -38,7 +38,7 @@ var scheme = {
 };
 
 function not_implemented_yet() {
-    throw "not implemented yet";
+    throw new Error("not implemented yet");
 };
 
 function coerce_bool(obj) {
-- 
2.11.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-09-01 22:09 ` Amirouche Boubekki
@ 2017-09-02  6:58   ` Arne Babenhauserheide
  2017-09-06 18:25   ` Amirouche Boubekki
  1 sibling, 0 replies; 16+ messages in thread
From: Arne Babenhauserheide @ 2017-09-02  6:58 UTC (permalink / raw)
  To: Amirouche Boubekki; +Cc: guile-user, guile-user, Ian Price

[-- Attachment #1: Type: text/plain, Size: 967 bytes --]


Amirouche Boubekki <amirouche@hypermove.net> writes:

> On 2017-08-28 20:56, Ian Price wrote:
> Apparently webkit does ship TCO and safari is the only browser [0] 
> shipping it.
> So it will happen soon enough for chrome, but chrome is not a solution 
> for many
> of us. I don't know when firefox with ship TCO.
>
> [0] https://www.chromestatus.com/feature/5516876633341952

Firefox tracks tail recursion support here:

https://bugzilla.mozilla.org/show_bug.cgi?id=723959

And I just found a JS library which turns functions into tail recursive
ones with 1-4x time overhead:

http://glat.info/pub/tailopt-js/

So it might be possible to check for recursion support and just insert
this as a fallback for browsers which do not support proper tail
recursion (i.e. by implementing it as an identity function in those
which support proper tail recursion).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-09-01 22:09 ` Amirouche Boubekki
  2017-09-02  6:58   ` Arne Babenhauserheide
@ 2017-09-06 18:25   ` Amirouche Boubekki
  2017-09-07  6:32     ` Amirouche Boubekki
  1 sibling, 1 reply; 16+ messages in thread
From: Amirouche Boubekki @ 2017-09-06 18:25 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user, guile-user

On 2017-09-02 00:09, Amirouche Boubekki wrote:
> On 2017-08-28 20:56, Ian Price wrote:
> 
>> 1 Introduction
>> ==============
>> 
>>   As many of you are aware, I have been working on compiling Guile
>>   Scheme to JavaScript this summer, as part of the Google Summer of
>>   Code. This post serves to bookend my work for the year.
>> 
>>   Before I go any further, I have to give my thanks to my mentor 
>> [Chris
>>   Webber], without whom this project would have fizzled out weeks ago;
>>   Google and the Gnu Project, naturally, for providing the Summer of
>>   Code and allowing me to work on this project; and our fearless 
>> leader,
>>   [Andy Wingo], for answering a wide variety of stupid questions.
>> 
> 
> Sorry, for my last mail I was a bit upset a minimal scm file with "42"
> will crash.
> 
> Anyway, I did a bit of exploration:
> 
> Apparently webkit does ship TCO and safari is the only browser [0] 
> shipping it.
> So it will happen soon enough for chrome, but chrome is not a solution 
> for many
> of us. I don't know when firefox with ship TCO.
> 
> [0] https://www.chromestatus.com/feature/5516876633341952
> 
> Anyway, TCO requires "use strict"; thing to work, so I prefixed the 
> output
> of guild jslink with that and it lead to discover a bug where there is 
> two
> times the same argument called `v_rest`. You can run the file with
> the following command:
> 
>    node --harmony_tailcalls main.js
> 
> It will tell you where the error is.
> 
> Here is the fixed code:
> 
>     var k_27363 = function(v_19412, k_27499) {
>       var k_27364 = function(v_19412, v_e, v_p, v_w, v_mod) {
>         var v_19413 = v_e;
>         var v_19414 = v_p;
>         var v_19415 = v_w;
>         var v_19416 = v_mod;
>         if (scheme["is_true"](scheme["primitives"]["pair?"](v_19413))) 
> {
>           {
>             var v_19417 = scheme["primitives"]["car"](v_19413);
>             var v_19418 = scheme["EMPTY"];
>             scheme["primitives"]["handle-interrupts"]();
>             var k_27370 = function(v_first, v_rest) {
>               var v_19419 = v_first;
>               var v_19420 = v_rest;
>               if (scheme["is_true"](v_19419)) {
>                 {
>                   var v_19421 = scheme["primitives"]["cdr"](v_19413);
>                   scheme["primitives"]["handle-interrupts"]();
>                   var k_27375 = function(v_rest, v_rest2) {  // The
> error is HERE!
>                     var v_19422 = v_rest;
>                     var v_19423 = v_rest2;
> 
> With that "fix" it run under nodejs v8.4.0 without increasing the 
> callstack.
> So it's good news!

With both patches applied and "use strict"; prefixed to the output of 
jslink
it works with chromium 60.0.3112.113 using the following command:

chromium-browser --js-flags="--harmony-tailcalls"

-- 
Amirouche ~ amz3 ~ http://www.hyperdev.fr



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-09-06 18:25   ` Amirouche Boubekki
@ 2017-09-07  6:32     ` Amirouche Boubekki
  2017-09-08 12:18       ` Amirouche Boubekki
  0 siblings, 1 reply; 16+ messages in thread
From: Amirouche Boubekki @ 2017-09-07  6:32 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user, guile-user

On 2017-09-06 20:25, Amirouche Boubekki wrote:
> On 2017-09-02 00:09, Amirouche Boubekki wrote:
>> On 2017-08-28 20:56, Ian Price wrote:
>> 
>>> 1 Introduction
>>> ==============
>>> 
>>>   As many of you are aware, I have been working on compiling Guile
>>>   Scheme to JavaScript this summer, as part of the Google Summer of
>>>   Code. This post serves to bookend my work for the year.
>>> 
>>>   Before I go any further, I have to give my thanks to my mentor 
>>> [Chris
>>>   Webber], without whom this project would have fizzled out weeks 
>>> ago;
>>>   Google and the Gnu Project, naturally, for providing the Summer of
>>>   Code and allowing me to work on this project; and our fearless 
>>> leader,
>>>   [Andy Wingo], for answering a wide variety of stupid questions.
>>> 
>> 
>> Sorry, for my last mail I was a bit upset a minimal scm file with "42"
>> will crash.
>> 
>> Anyway, I did a bit of exploration:
>> 
>> Apparently webkit does ship TCO and safari is the only browser [0] 
>> shipping it.
>> So it will happen soon enough for chrome, but chrome is not a solution 
>> for many
>> of us. I don't know when firefox with ship TCO.
>> 
>> [0] https://www.chromestatus.com/feature/5516876633341952
>> 
>> Anyway, TCO requires "use strict"; thing to work, so I prefixed the 
>> output
>> of guild jslink with that and it lead to discover a bug where there is 
>> two
>> times the same argument called `v_rest`. You can run the file with
>> the following command:
>> 
>>    node --harmony_tailcalls main.js
>> 
>> It will tell you where the error is.
>> 
>> Here is the fixed code:
>> 
>>     var k_27363 = function(v_19412, k_27499) {
>>       var k_27364 = function(v_19412, v_e, v_p, v_w, v_mod) {
>>         var v_19413 = v_e;
>>         var v_19414 = v_p;
>>         var v_19415 = v_w;
>>         var v_19416 = v_mod;
>>         if (scheme["is_true"](scheme["primitives"]["pair?"](v_19413))) 
>> {
>>           {
>>             var v_19417 = scheme["primitives"]["car"](v_19413);
>>             var v_19418 = scheme["EMPTY"];
>>             scheme["primitives"]["handle-interrupts"]();
>>             var k_27370 = function(v_first, v_rest) {
>>               var v_19419 = v_first;
>>               var v_19420 = v_rest;
>>               if (scheme["is_true"](v_19419)) {
>>                 {
>>                   var v_19421 = scheme["primitives"]["cdr"](v_19413);
>>                   scheme["primitives"]["handle-interrupts"]();
>>                   var k_27375 = function(v_rest, v_rest2) {  // The
>> error is HERE!
>>                     var v_19422 = v_rest;
>>                     var v_19423 = v_rest2;
>> 
>> With that "fix" it run under nodejs v8.4.0 without increasing the 
>> callstack.
>> So it's good news!
> 
> With both patches applied and "use strict"; prefixed to the output of 
> jslink
> it works with chromium 60.0.3112.113 using the following command:
> 
> chromium-browser --js-flags="--harmony-tailcalls"

I pushed my changes to 
https://gitlab.com/amirouche/guile/tree/compile-to-js-2017

I am only missing `js-cloure` procedure to be able to submit guile 
callbacks
to javascript world.

-- 
Amirouche ~ amz3 ~ http://www.hyperdev.fr



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-09-07  6:32     ` Amirouche Boubekki
@ 2017-09-08 12:18       ` Amirouche Boubekki
  0 siblings, 0 replies; 16+ messages in thread
From: Amirouche Boubekki @ 2017-09-08 12:18 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user, guile-user

Le 2017-09-07 08:32, Amirouche Boubekki a écrit :
> On 2017-09-06 20:25, Amirouche Boubekki wrote:
>> On 2017-09-02 00:09, Amirouche Boubekki wrote:
>>> On 2017-08-28 20:56, Ian Price wrote:
>>> 
>>>> 1 Introduction
>>>> ==============
>>>> 
>>>>   As many of you are aware, I have been working on compiling Guile
>>>>   Scheme to JavaScript this summer, as part of the Google Summer of
>>>>   Code. This post serves to bookend my work for the year.
>>>> 

[...]

>>> 
>>> With that "fix" it run under nodejs v8.4.0 without increasing the 
>>> callstack.
>>> So it's good news!
>> 
>> With both patches applied and "use strict"; prefixed to the output of 
>> jslink
>> it works with chromium 60.0.3112.113 using the following command:
>> 
>> chromium-browser --js-flags="--harmony-tailcalls"
> 
> I pushed my changes to
> https://gitlab.com/amirouche/guile/tree/compile-to-js-2017
> 
> I am only missing `js-closure` procedure to be able to submit guile 
> callbacks
> to javascript world.

Now my framework works with guile-js.

I pushed everything at 
https://gitlab.com/amirouche/guile/commits/compile-to-js-2017

Tx again ijp for the hard work :)

-- 
Amirouche ~ amz3 ~ http://www.hyperdev.fr



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2017-08-28 18:56 Summer of Code Recap Ian Price
                   ` (2 preceding siblings ...)
  2017-09-01 22:09 ` Amirouche Boubekki
@ 2021-05-11 14:45 ` Christopher Lemmer Webber
  2021-05-11 15:08   ` Christopher Lemmer Webber
  2021-05-11 15:50   ` Dr. Arne Babenhauserheide
  3 siblings, 2 replies; 16+ messages in thread
From: Christopher Lemmer Webber @ 2021-05-11 14:45 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user

Hi!

Ian did some great work here in the past... let's not let it go to
waste.  Let's try to merge it!

I've made a branch in my gitlab repo here:

  https://gitlab.com/dustyweb/guile.git

the branch is "compile-to-js-merge"

I've dealt with the merge conflicts and etc I've been able to identify,
but things have already started to bitrot... I'd like to prevent them
from bitrotting further.  I fixed some things, updating the code to
where it appears things have shuffled around to as best as I could.

Currently I can get a file as simple as "just-plus.scm" to compile:

  (+ 1 2)

This outputs to:

  function (unit_cont){var k_0 = function (v_0,k_4){var k_1 = function (v_0){var v_1 = 3;return k_4(v_1);};if ((arguments["length"])==(2)) {{return k_1(v_0);}} else {{return undefined;}}};return k_0(undefined,unit_cont);};

Progress!

However, the amb.scm file no longer works as described below.  I get the
following:

  In language/cps/intset.scm:
      472:6  3 (visit-branch #(4294967295 1073741823 #f #f #f #f #f #f (#f)) _ 0 # #)
      472:6  2 (visit-branch 4294967295 _ 0 _ _)
  In language/cps/split-rec.scm:
      78:22  1 (_ _ _ _)
  In ice-9/boot-9.scm:
    1685:16  0 (raise-exception _ #:continuable? _)
  
  ice-9/boot-9.scm:1685:16: In procedure raise-exception:
  Throw to key `match-error' with args `("match" "no matching pattern" #<cps (const-fun 62)>)'.

I guess that's something that probably changed.  I'm going to look into
it...

Anyway, is there support from the maintainers from getting this merged
if I can get things working again?  I'd really like to see this effort
not go to waste... I'd even like to write a few demos using it.


Ian Price writes:

> 1 Introduction
> ==============
>
>   As many of you are aware, I have been working on compiling Guile
>   Scheme to JavaScript this summer, as part of the Google Summer of
>   Code. This post serves to bookend my work for the year.
>
>   Before I go any further, I have to give my thanks to my mentor [Chris
>   Webber], without whom this project would have fizzled out weeks ago;
>   Google and the Gnu Project, naturally, for providing the Summer of
>   Code and allowing me to work on this project; and our fearless leader,
>   [Andy Wingo], for answering a wide variety of stupid questions.
>
>
> [Chris Webber] https://dustycloud.org/
>
> [Andy Wingo] https://wingolog.org/
>
>
> 2 Project Aims
> ==============
>
>   For a full introduction to the project, you can of course refer back
>   to my [project proposal], but very briefly my hopes for this summer
>   were:
>
>   1. To rewrite the previous version of my compiler from the [previous
>      CPS representation] to use the new representation ["CPS Soup"]
>      representation.
>   2. To completely port ice-9/boot-9.scm (our basic "prelude") to
>      JavaScript, and in particular, to support the [Guile Module
>      system].
>   3. To handle Proper Tail Calls by use of the [Cheney on the MTA]
>      strategy.
>   4. To include a new `guild' script for bundling compiled JS files with
>      their dependencies.
>
>
> [project proposal] https://shift-reset.com/static/docs/gsoc-2017.pdf
>
> [previous CPS representation]
> https://wingolog.org/archives/2014/01/12/a-continuation-passing-style-intermediate-language-for-guile
>
> ["CPS Soup"] https://wingolog.org/archives/2015/07/27/cps-soup
>
> [Guile Module system]
> https://www.gnu.org/software/guile/manual/html_node/Modules.html#Modules
>
> [Cheney on the MTA] http://www.pipeline.com/~hbaker1/CheneyMTA.html
>
>
> 3 What was Achieved
> ===================
>
>   You can find all of my work on the [compile-to-js-2017] branch of my
>   Gitlab. A full list of the commits can be found [here], but I will
>   summarise the changes now:
>
>
> [compile-to-js-2017]
> https://gitlab.com/ijp/guile/tree/compile-to-js-2017
>
> [here] https://gitlab.com/ijp/guile/compare/1b36a76e...gsoc-2017-end
>
> 3.1 Compile Guile CPS Soup to JavaScript
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>   When I was working on my initial attempt at compiling Guile to
>   JavaScript, two years ago, Guile used a different CPS representation
>   as its intermediate language. The initial experiments with the CPS
>   Soup representation occurred while that work was ongoing, but as it
>   was not considered "stable", the plan was not to move to this
>   representation until after I had completed my other objectives.
>
>   Now, however, CPS Soup is the IL of Guile, and so the first task that
>   was accomplished was to move to this representation. Since I had
>   already created my own JS-IL as a target, I did not need to make any
>   changes to the code generation side from JS-IL to JavaScript proper.
>   The main change was to reconstruct the nested scope structure that was
>   implicit in the dominator structure that Guile made available.
>
>   The full code for the compiler is split into several sections,
>   corresponding to different stages in the compiler pipeline.
>
>
> 3.1.1 CPS to JS-IL Compiler
> ---------------------------
>
>   - module/language/cps/compile-js.scm
>   - module/language/cps/spec.scm
>
>   These modules constitute the compiler from CPS to my JS-IL
>   intermediate language.
>
>
> 3.1.2 JS-IL to JavaScript Compiler
> ----------------------------------
>
>   - module/language/js-il.scm
>   - module/language/js-il/compile-javascript.scm
>   - module/language/js-il/inlining.scm
>   - module/language/js-il/spec.scm
>
>   These modules constitute a somewhat ad-hoc intermediate representation
>   as a target for the CPS compiler. It differs from JavaScript, e.g., by
>   continuing to separate continuations and functions, and a slightly
>   specialised function representation to handle Guile's complicated
>   notion of procedure arity.
>
>
> 3.1.3 JavaScript Representation
> -------------------------------
>
>   - module/language/javascript.scm
>   - module/language/javascript/simplify.scm
>   - module/language/javascript/spec.scm
>
>   This is primarily the representation of JavaScript as Scheme Records.
>   This is separate from the representation of JavaScript Guile already
>   has in the form of `(language ecmascript)' primarily to avoid a
>   circularity when Guile determines which compilers to run in the
>   pipeline, as recommended by Andy Wingo.
>
>
> 3.2 A pre-amble capable of running through boot-9
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>   In order to run Guile, it is not enough to be able to compile Scheme
>   (or indeed any other language supported by Guile) forms to JavaScript,
>   we also need to incorporate as much of Guile's runtime as possible.
>   This involves implementing VM primitives (such as you might see in
>   vm-engine.c); basic Guile types like Symbols, Pairs, and Structs; as
>   well as many of the functions that Guile implements in C rather than
>   Scheme.
>
>   Although I certainly did not implement all of the functionality Guile
>   achieves, I was able to implement sufficiently many (including what
>   amounts to a port of much of module.c) that one can successfully run
>   though ice-9/boot-9.scm from start to finish.
>
>   This took up the bulk of the time I spent on this project, due to the
>   size of the compiled output of boot-9.scm, and my own difficulties
>   debugging the bootstrap process. More on this below.
>
>   The code can be found at
>   - module/language/js-il/runtime.js
>
>
> 3.3 A linking script for JavaScript
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>   Since we are using the `(language ...)' infrastructure, we can take
>   advantage of the existing `guild compile' script for compiling to
>   JavaScript, we simply need to use the `--to' switch. However, this
>   does not produce a file which you can just load up without any
>   additional work, especially if you are working with multiple modules.
>
>   In order to make it easier to deal with this, I have included a `guild
>   jslink' script, which can be used to package up a "main" script along
>   with the `runtime.js' and its dependencies. See below for an example.
>
>   The code can be found at
>   - module/scripts/jslink.scm
>
>
> 4 What was not Achieved
> =======================
>
> 4.1 Cheney on the MTA
> ~~~~~~~~~~~~~~~~~~~~~
>
>   One of my regrets is that I did not implement Baker's "Cheney on the
>   MTA" (as seen in [Chicken Scheme]) for handling Proper Tail Calls in
>   JavaScript. Historically, JavaScript has not guaranteed that tail
>   position function calls do not grow the stack, and this is obviously
>   of fundamental importance for languages like Scheme. Fortunately, ES6
>   has added support for [proper tail calls] and we can expect to see
>   increased support for it in future JavaScript versions. (Indeed,
>   during testing on node v.6.10.3, I did not have to increase the stack
>   size until very late).
>
>
> [Chicken Scheme] https://www.call-cc.org/
>
> [proper tail calls]
> https://www.ecma-international.org/ecma-262/6.0/#sec-tail-position-calls
>
>
> 5 How to use it
> ===============
>
>   I've talked a lot about what I've did and didn't do, but what about
>   actually using this thing?
>
>
> 5.1 Obtaining the Code
> ~~~~~~~~~~~~~~~~~~~~~~
>
>   The code is not currently available from the main Guile repository,
>   but only the `compile-to-js-2017' branch on my [GitLab].
>
>   If you already have a checkout of guile, you can add my repo as a
>   remote with
>   ,----
>   | $ git remote add ijp https://gitlab.com/ijp/guile.git
>   `----
>   and fetch the branch with
>   ,----
>   | $ git fetch ijp
>   `----
>
>   You can then check out the `compile-to-js-2017' branch and build as
>   normal.
>
>
> [GitLab] https://gitlab.com/ijp/guile/tree/compile-to-js-2017
>
>
> 5.2 A Non-Trivial Example
> ~~~~~~~~~~~~~~~~~~~~~~~~~
>
>   As an example of how to use the JS Backend that is short, but
>   non-trivial, I am using John McCarthy's `amb' operator (see [A Basis
>   for a Mathematical Theory of Computation]) to search for Pythagorean
>   Triples.
>
>   First we have a module for the `amb' operator in amb.scm
>   ,----
>   | (define-module (amb)
>   |   #:export (amb fail))
>   |
>   | (define original-fail
>   |   (lambda _
>   |     (error 'amb "No more paths to search")))
>   |
>   | (define *amb-fail* original-fail)
>   |
>   | (define (fail)
>   |   (*amb-fail* #f))
>   |
>   | (define (amb-thunks . values)
>   |   (let ((failure *amb-fail*))
>   |     (call/cc (lambda (escape)
>   |                (for-each (lambda (value)
>   |                            (call/cc (lambda (continue)
>   |                                       (set! *amb-fail* continue)
>   |                                       (escape (value)))))
>   |                          values)
>   |                (failure #f)))))
>   |
>   | (define-syntax amb
>   |   (syntax-rules ()
>   |     ((amb exprs ...)
>   |      (amb-thunks (lambda () exprs) ...))))
>   `----
>
>   Next we have the code performs the search in triple.scm
>   ,----
>   | (use-modules (amb))
>   |
>   | (let ((a (amb 4 5 6 7 8 9 10))
>   |       (b (amb 4 5 6 7 8 9 10))
>   |       (c (amb 4 5 6 7 8 9 10)))
>   |   (if (= (* c c) (+ (* a a) (* b b)))
>   |       (list a b c)
>   |       (fail)))
>   `----
>
>   We compile the files in the usual manner, only now we specify the
>   `javascript' language (We make sure to add the current directory to
>   the load-path for triple.scm).
>
>   ,----
>   | $ guild compile amb.scm --to=javascript --output=amb.js
>   | $ guild compile -L . triple.scm --to=javascript --output=triple.js
>   `----
>
>   Next we link the two together into a file main.js, making sure to
>   specify amb.js as a dependency of triple.js. (This step will take a
>   little while, since it also compiles a bunch of dependencies)
>
>   ,----
>   | $ guild jslink triple.js -o main.js --depends="(\"amb\" . \"amb.scm\")"
>   `----
>
>   Finally, you can run it with `node', although as mentioned above you
>   may have to increase the stack size.
>
>   ,----
>   | $ node  --stack-size=2000 main.js
>   `----
>
>   Which should, fingers crossed, print out the triple 6,8,10.
>
>
> [A Basis for a Mathematical Theory of Computation]
> http://www-formal.stanford.edu/jmc/basis1.pdf
>
>
> 6 What is next?
> ===============
>
>   Having recapped what was and what was not achieved, the next question
>   is: where does the project go from here? I have been asked about my
>   plans for all sorts of features, e.g. support for [Web Assembly], but
>   I think the following things are the most important to think about.
>
>
> [Web Assembly] http://webassembly.org/
>
> 6.1 Inclusion into Guile
> ~~~~~~~~~~~~~~~~~~~~~~~~
>
>   The entire point of the project is to have something that can be
>   included in Guile proper. I have not spoken with Guile's maintainers
>   about incorporation into the main distribution, but I expect there
>   would be not be too many problems with moving the "official branch" to
>   the main repository.
>
>
> 6.2 All Guile built-ins in runtime.js
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>   Although I have included enough to get though boot-9.scm, this does
>   not include all of the built-ins we would want in our programs. Two
>   things I use very often which do not appear in runtime.js are ports
>   and bytevectors.
>
>   We would like most, if not all, Guile built-ins to be available for
>   those who need them, so these will need to be implemented. However,
>   this is a lot of extra code for some people who don't need it, which
>   brings us to a different issue...
>
>
> 6.3 Linking Guile Modules & Features
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>   In [a blog post], Andy Wingo lays out many tasks that he would like to
>   see in a future Guile. One of the most important of these, for us, are
>   under the headings "linking multiple modules together" and "linking a
>   single executable". To grossly simplify, we want to be able to link
>   various files into one single executable, which contains all and only
>   the code we need for our application.
>
>   As it stands, I included a simple script `guild jslink' that bundles
>   various compiled JavaScript files into one file, but we would like it
>   to be much more featureful: removing modules, functions, even types we
>   don't need; and inferring which modules are required by our
>   application and bundling them without requiring the information
>   `jslink' does. This would allow us to minimise the amount of code that
>   needs to be sent over the network, which is very important to web
>   developers.
>
>   This is a large task, and one I don't know enough about at the moment
>   to attempt, but it is work that would benefit not just our JavaScript
>   compiler, but people who want to deploy regular Guile applications.
>
>
> [a blog post]
> https://wingolog.org/archives/2016/02/04/guile-compiler-tasks
>
>
> 6.4 JavaScript Version
> ~~~~~~~~~~~~~~~~~~~~~~
>
>   I am not an expert in JavaScript, in fact, before this summer I
>   probably hadn't written it for two years, which means the code
>   certainly does not match up with the current best practices and
>   specifications. Further, all of my testing for this compiler was done
>   on [Node.js] v.6.10.3 only (this was the version available in the
>   Fedora 25 repositories).
>
>   The code should be vetted to determine precisely which modern JS
>   features are used (I believe proper tail calls, and ES6 Maps are the
>   main ones), and it should be tested on all major browsers. If
>   necessary, we should incorporate switches in the compiler to allow JS
>   users to compile for particular implementations, taking advantage of
>   particular modern JS features, or providing our own implementations of
>   those that are not supported (e.g. Cheney on the MTA).
>
>
> [Node.js] https://nodejs.org/en/
>
>
> 6.5 JS Integration
> ~~~~~~~~~~~~~~~~~~
>
>   One of the strengths of Guile is that it allows people to integrate
>   their Scheme and C code, and although it has not been a focus for this
>   summer, we should aim to provide similar levels of integration between
>   Scheme and JS. There are two cases to consider.
>
>
> 6.5.1 JS calling Scheme
> -----------------------
>
>   As it stands, you can perform some limited interaction from JavaScript
>   in a similar manner to how you would interact with Guile from C. For
>   instance, by using `scm_current_module', `scm_public_lookup', and the
>   `scheme.Symbol' constructor, one could look up a scheme function, e.g.
>   `iota', and then invoke it by `scheme.call'.
>
>   That said, C idioms are not JS idioms, and so we should work to
>   provide a much nicer API through the `scheme' object.
>
>
> 6.5.2 Scheme calling JS
> -----------------------
>
>   In the case of Scheme calling JavaScript, I think we should follow the
>   example of `(system foreign)', which provides an API for linking to
>   dynamic C libraries, and creating Scheme versions of C functions, and
>   automatically marshalling/unmarshalling C types to Scheme types. One
>   additional complication we would have with JS would be the presence of
>   exceptions, but I think these could also be marshalled into Scheme
>   ones without much trouble.
>
>
> 7 Lessons Learned
> =================
>
>   It goes without saying that a project like this teaches you a lot
>   about the technical design of Guile, how to navigate the codebase,
>   etc, but I want to highlight a few "softer" lessons from this summer.
>
>
> 7.1 Compilers are "Easy", Runtimes are Hard
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>   When I first set out to write this project two summers ago, I
>   naturally assumed that the majority of the effort would go into the
>   compiler, and much less into the built-ins. In reality, the effort was
>   reversed. Partly this was due to my experience in writing Scheme, and
>   Functional Programming more generally, meant that the tree-traversing
>   code typical of a compiler pass was relatively straightforward, and
>   the compiler was not doing a lot of optimisation, mostly code
>   generation.
>
>
> 7.2 Bootstrapping is Hard
> ~~~~~~~~~~~~~~~~~~~~~~~~~
>
>   The last point leads into this one, bootstrapping is pretty tricky.
>   With boot-9, you have several versions of the module system at
>   different times. My own attempt to write module code that handled this
>   ended up being abandoned for a rewrite that more closely followed the
>   Guile C code. The size of the compiled boot-9 code, and the, at times,
>   non-local consequences of implementing certain built-ins made it
>   tricky to debug.
>
>
> 7.3 Don't Panic
> ~~~~~~~~~~~~~~~
>
>   This is a much more personal one, and one that I think is very
>   important for anyone who wants to take part in a program like the
>   Summer of Code, where you are spending a lot of time mostly on your
>   own. In a complex software project, things are not always going to go
>   smoothly. You might spend weeks banging up against a difficult
>   problem. Don't Panic! If it was easy it would have already been done.
>   Keep in Contact with your Mentor! It is tempting to only check in when
>   you think you have something of progress to report, but they are there
>   to help you, and explaining your issues to someone else is often very
>   useful when trying to overcome them, even if they don't have an answer
>   for you.
>
>
> 8 Wrapping Up
> =============
>
>   If you are still with me, good on you. As the new semester is starting
>   I will be devoting much less time to this, and that will likely be
>   true till December, but I will make an effort to keep up with
>   guile-user and be on the IRC Channel to help the daring souls who want
>   to give this a go. My priorities will be documenting the ILs, filling
>   in missing builtins, and improving jslink. I especially want to see
>   basic IO and MiniKanren up and running, and for it to be convenient to
>   use Guile's builtin libraries.
>
>
> Happy Hacking, Ian Price
>
> (This is a crosspost to guile-user of my blogpost [Summer of Code
> Recap], but please comment on this list, rather than there)
>
> [Summer of Code Recap]
> https://shift-reset.com/blog/2017/8/28/Summer%20of%20Code%20Recap/




^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2021-05-11 14:45 ` Christopher Lemmer Webber
@ 2021-05-11 15:08   ` Christopher Lemmer Webber
  2021-05-11 15:19     ` Christopher Lemmer Webber
  2021-05-11 15:50   ` Dr. Arne Babenhauserheide
  1 sibling, 1 reply; 16+ messages in thread
From: Christopher Lemmer Webber @ 2021-05-11 15:08 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user

I've now verified that the place where things fall apart is fairly
simple.  The following file does not compile:

  (define (add x y)
    (+ x y))

  (add 1 2)

So yeah, it's just functions in general.

It looks like the stage where things are breaking is between the
cps -> js-il representations.

I figured since probably the changes need to happen in
module/language/cps/compile-js.scm, I should look at
the commit log in compile-bytecode.scm in that same directory.
It looks like a lot has changed since 2017!

I suspect I need help at this stage! :)


Christopher Lemmer Webber writes:

> Hi!
>
> Ian did some great work here in the past... let's not let it go to
> waste.  Let's try to merge it!
>
> I've made a branch in my gitlab repo here:
>
>   https://gitlab.com/dustyweb/guile.git
>
> the branch is "compile-to-js-merge"
>
> I've dealt with the merge conflicts and etc I've been able to identify,
> but things have already started to bitrot... I'd like to prevent them
> from bitrotting further.  I fixed some things, updating the code to
> where it appears things have shuffled around to as best as I could.
>
> Currently I can get a file as simple as "just-plus.scm" to compile:
>
>   (+ 1 2)
>
> This outputs to:
>
>   function (unit_cont){var k_0 = function (v_0,k_4){var k_1 = function (v_0){var v_1 = 3;return k_4(v_1);};if ((arguments["length"])==(2)) {{return k_1(v_0);}} else {{return undefined;}}};return k_0(undefined,unit_cont);};
>
> Progress!
>
> However, the amb.scm file no longer works as described below.  I get the
> following:
>
>   In language/cps/intset.scm:
>       472:6  3 (visit-branch #(4294967295 1073741823 #f #f #f #f #f #f (#f)) _ 0 # #)
>       472:6  2 (visit-branch 4294967295 _ 0 _ _)
>   In language/cps/split-rec.scm:
>       78:22  1 (_ _ _ _)
>   In ice-9/boot-9.scm:
>     1685:16  0 (raise-exception _ #:continuable? _)
>   
>   ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>   Throw to key `match-error' with args `("match" "no matching pattern" #<cps (const-fun 62)>)'.
>
> I guess that's something that probably changed.  I'm going to look into
> it...
>
> Anyway, is there support from the maintainers from getting this merged
> if I can get things working again?  I'd really like to see this effort
> not go to waste... I'd even like to write a few demos using it.
>
>
> Ian Price writes:
>
>> 1 Introduction
>> ==============
>>
>>   As many of you are aware, I have been working on compiling Guile
>>   Scheme to JavaScript this summer, as part of the Google Summer of
>>   Code. This post serves to bookend my work for the year.
>>
>>   Before I go any further, I have to give my thanks to my mentor [Chris
>>   Webber], without whom this project would have fizzled out weeks ago;
>>   Google and the Gnu Project, naturally, for providing the Summer of
>>   Code and allowing me to work on this project; and our fearless leader,
>>   [Andy Wingo], for answering a wide variety of stupid questions.
>>
>>
>> [Chris Webber] https://dustycloud.org/
>>
>> [Andy Wingo] https://wingolog.org/
>>
>>
>> 2 Project Aims
>> ==============
>>
>>   For a full introduction to the project, you can of course refer back
>>   to my [project proposal], but very briefly my hopes for this summer
>>   were:
>>
>>   1. To rewrite the previous version of my compiler from the [previous
>>      CPS representation] to use the new representation ["CPS Soup"]
>>      representation.
>>   2. To completely port ice-9/boot-9.scm (our basic "prelude") to
>>      JavaScript, and in particular, to support the [Guile Module
>>      system].
>>   3. To handle Proper Tail Calls by use of the [Cheney on the MTA]
>>      strategy.
>>   4. To include a new `guild' script for bundling compiled JS files with
>>      their dependencies.
>>
>>
>> [project proposal] https://shift-reset.com/static/docs/gsoc-2017.pdf
>>
>> [previous CPS representation]
>> https://wingolog.org/archives/2014/01/12/a-continuation-passing-style-intermediate-language-for-guile
>>
>> ["CPS Soup"] https://wingolog.org/archives/2015/07/27/cps-soup
>>
>> [Guile Module system]
>> https://www.gnu.org/software/guile/manual/html_node/Modules.html#Modules
>>
>> [Cheney on the MTA] http://www.pipeline.com/~hbaker1/CheneyMTA.html
>>
>>
>> 3 What was Achieved
>> ===================
>>
>>   You can find all of my work on the [compile-to-js-2017] branch of my
>>   Gitlab. A full list of the commits can be found [here], but I will
>>   summarise the changes now:
>>
>>
>> [compile-to-js-2017]
>> https://gitlab.com/ijp/guile/tree/compile-to-js-2017
>>
>> [here] https://gitlab.com/ijp/guile/compare/1b36a76e...gsoc-2017-end
>>
>> 3.1 Compile Guile CPS Soup to JavaScript
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   When I was working on my initial attempt at compiling Guile to
>>   JavaScript, two years ago, Guile used a different CPS representation
>>   as its intermediate language. The initial experiments with the CPS
>>   Soup representation occurred while that work was ongoing, but as it
>>   was not considered "stable", the plan was not to move to this
>>   representation until after I had completed my other objectives.
>>
>>   Now, however, CPS Soup is the IL of Guile, and so the first task that
>>   was accomplished was to move to this representation. Since I had
>>   already created my own JS-IL as a target, I did not need to make any
>>   changes to the code generation side from JS-IL to JavaScript proper.
>>   The main change was to reconstruct the nested scope structure that was
>>   implicit in the dominator structure that Guile made available.
>>
>>   The full code for the compiler is split into several sections,
>>   corresponding to different stages in the compiler pipeline.
>>
>>
>> 3.1.1 CPS to JS-IL Compiler
>> ---------------------------
>>
>>   - module/language/cps/compile-js.scm
>>   - module/language/cps/spec.scm
>>
>>   These modules constitute the compiler from CPS to my JS-IL
>>   intermediate language.
>>
>>
>> 3.1.2 JS-IL to JavaScript Compiler
>> ----------------------------------
>>
>>   - module/language/js-il.scm
>>   - module/language/js-il/compile-javascript.scm
>>   - module/language/js-il/inlining.scm
>>   - module/language/js-il/spec.scm
>>
>>   These modules constitute a somewhat ad-hoc intermediate representation
>>   as a target for the CPS compiler. It differs from JavaScript, e.g., by
>>   continuing to separate continuations and functions, and a slightly
>>   specialised function representation to handle Guile's complicated
>>   notion of procedure arity.
>>
>>
>> 3.1.3 JavaScript Representation
>> -------------------------------
>>
>>   - module/language/javascript.scm
>>   - module/language/javascript/simplify.scm
>>   - module/language/javascript/spec.scm
>>
>>   This is primarily the representation of JavaScript as Scheme Records.
>>   This is separate from the representation of JavaScript Guile already
>>   has in the form of `(language ecmascript)' primarily to avoid a
>>   circularity when Guile determines which compilers to run in the
>>   pipeline, as recommended by Andy Wingo.
>>
>>
>> 3.2 A pre-amble capable of running through boot-9
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   In order to run Guile, it is not enough to be able to compile Scheme
>>   (or indeed any other language supported by Guile) forms to JavaScript,
>>   we also need to incorporate as much of Guile's runtime as possible.
>>   This involves implementing VM primitives (such as you might see in
>>   vm-engine.c); basic Guile types like Symbols, Pairs, and Structs; as
>>   well as many of the functions that Guile implements in C rather than
>>   Scheme.
>>
>>   Although I certainly did not implement all of the functionality Guile
>>   achieves, I was able to implement sufficiently many (including what
>>   amounts to a port of much of module.c) that one can successfully run
>>   though ice-9/boot-9.scm from start to finish.
>>
>>   This took up the bulk of the time I spent on this project, due to the
>>   size of the compiled output of boot-9.scm, and my own difficulties
>>   debugging the bootstrap process. More on this below.
>>
>>   The code can be found at
>>   - module/language/js-il/runtime.js
>>
>>
>> 3.3 A linking script for JavaScript
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   Since we are using the `(language ...)' infrastructure, we can take
>>   advantage of the existing `guild compile' script for compiling to
>>   JavaScript, we simply need to use the `--to' switch. However, this
>>   does not produce a file which you can just load up without any
>>   additional work, especially if you are working with multiple modules.
>>
>>   In order to make it easier to deal with this, I have included a `guild
>>   jslink' script, which can be used to package up a "main" script along
>>   with the `runtime.js' and its dependencies. See below for an example.
>>
>>   The code can be found at
>>   - module/scripts/jslink.scm
>>
>>
>> 4 What was not Achieved
>> =======================
>>
>> 4.1 Cheney on the MTA
>> ~~~~~~~~~~~~~~~~~~~~~
>>
>>   One of my regrets is that I did not implement Baker's "Cheney on the
>>   MTA" (as seen in [Chicken Scheme]) for handling Proper Tail Calls in
>>   JavaScript. Historically, JavaScript has not guaranteed that tail
>>   position function calls do not grow the stack, and this is obviously
>>   of fundamental importance for languages like Scheme. Fortunately, ES6
>>   has added support for [proper tail calls] and we can expect to see
>>   increased support for it in future JavaScript versions. (Indeed,
>>   during testing on node v.6.10.3, I did not have to increase the stack
>>   size until very late).
>>
>>
>> [Chicken Scheme] https://www.call-cc.org/
>>
>> [proper tail calls]
>> https://www.ecma-international.org/ecma-262/6.0/#sec-tail-position-calls
>>
>>
>> 5 How to use it
>> ===============
>>
>>   I've talked a lot about what I've did and didn't do, but what about
>>   actually using this thing?
>>
>>
>> 5.1 Obtaining the Code
>> ~~~~~~~~~~~~~~~~~~~~~~
>>
>>   The code is not currently available from the main Guile repository,
>>   but only the `compile-to-js-2017' branch on my [GitLab].
>>
>>   If you already have a checkout of guile, you can add my repo as a
>>   remote with
>>   ,----
>>   | $ git remote add ijp https://gitlab.com/ijp/guile.git
>>   `----
>>   and fetch the branch with
>>   ,----
>>   | $ git fetch ijp
>>   `----
>>
>>   You can then check out the `compile-to-js-2017' branch and build as
>>   normal.
>>
>>
>> [GitLab] https://gitlab.com/ijp/guile/tree/compile-to-js-2017
>>
>>
>> 5.2 A Non-Trivial Example
>> ~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   As an example of how to use the JS Backend that is short, but
>>   non-trivial, I am using John McCarthy's `amb' operator (see [A Basis
>>   for a Mathematical Theory of Computation]) to search for Pythagorean
>>   Triples.
>>
>>   First we have a module for the `amb' operator in amb.scm
>>   ,----
>>   | (define-module (amb)
>>   |   #:export (amb fail))
>>   |
>>   | (define original-fail
>>   |   (lambda _
>>   |     (error 'amb "No more paths to search")))
>>   |
>>   | (define *amb-fail* original-fail)
>>   |
>>   | (define (fail)
>>   |   (*amb-fail* #f))
>>   |
>>   | (define (amb-thunks . values)
>>   |   (let ((failure *amb-fail*))
>>   |     (call/cc (lambda (escape)
>>   |                (for-each (lambda (value)
>>   |                            (call/cc (lambda (continue)
>>   |                                       (set! *amb-fail* continue)
>>   |                                       (escape (value)))))
>>   |                          values)
>>   |                (failure #f)))))
>>   |
>>   | (define-syntax amb
>>   |   (syntax-rules ()
>>   |     ((amb exprs ...)
>>   |      (amb-thunks (lambda () exprs) ...))))
>>   `----
>>
>>   Next we have the code performs the search in triple.scm
>>   ,----
>>   | (use-modules (amb))
>>   |
>>   | (let ((a (amb 4 5 6 7 8 9 10))
>>   |       (b (amb 4 5 6 7 8 9 10))
>>   |       (c (amb 4 5 6 7 8 9 10)))
>>   |   (if (= (* c c) (+ (* a a) (* b b)))
>>   |       (list a b c)
>>   |       (fail)))
>>   `----
>>
>>   We compile the files in the usual manner, only now we specify the
>>   `javascript' language (We make sure to add the current directory to
>>   the load-path for triple.scm).
>>
>>   ,----
>>   | $ guild compile amb.scm --to=javascript --output=amb.js
>>   | $ guild compile -L . triple.scm --to=javascript --output=triple.js
>>   `----
>>
>>   Next we link the two together into a file main.js, making sure to
>>   specify amb.js as a dependency of triple.js. (This step will take a
>>   little while, since it also compiles a bunch of dependencies)
>>
>>   ,----
>>   | $ guild jslink triple.js -o main.js --depends="(\"amb\" . \"amb.scm\")"
>>   `----
>>
>>   Finally, you can run it with `node', although as mentioned above you
>>   may have to increase the stack size.
>>
>>   ,----
>>   | $ node  --stack-size=2000 main.js
>>   `----
>>
>>   Which should, fingers crossed, print out the triple 6,8,10.
>>
>>
>> [A Basis for a Mathematical Theory of Computation]
>> http://www-formal.stanford.edu/jmc/basis1.pdf
>>
>>
>> 6 What is next?
>> ===============
>>
>>   Having recapped what was and what was not achieved, the next question
>>   is: where does the project go from here? I have been asked about my
>>   plans for all sorts of features, e.g. support for [Web Assembly], but
>>   I think the following things are the most important to think about.
>>
>>
>> [Web Assembly] http://webassembly.org/
>>
>> 6.1 Inclusion into Guile
>> ~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   The entire point of the project is to have something that can be
>>   included in Guile proper. I have not spoken with Guile's maintainers
>>   about incorporation into the main distribution, but I expect there
>>   would be not be too many problems with moving the "official branch" to
>>   the main repository.
>>
>>
>> 6.2 All Guile built-ins in runtime.js
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   Although I have included enough to get though boot-9.scm, this does
>>   not include all of the built-ins we would want in our programs. Two
>>   things I use very often which do not appear in runtime.js are ports
>>   and bytevectors.
>>
>>   We would like most, if not all, Guile built-ins to be available for
>>   those who need them, so these will need to be implemented. However,
>>   this is a lot of extra code for some people who don't need it, which
>>   brings us to a different issue...
>>
>>
>> 6.3 Linking Guile Modules & Features
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   In [a blog post], Andy Wingo lays out many tasks that he would like to
>>   see in a future Guile. One of the most important of these, for us, are
>>   under the headings "linking multiple modules together" and "linking a
>>   single executable". To grossly simplify, we want to be able to link
>>   various files into one single executable, which contains all and only
>>   the code we need for our application.
>>
>>   As it stands, I included a simple script `guild jslink' that bundles
>>   various compiled JavaScript files into one file, but we would like it
>>   to be much more featureful: removing modules, functions, even types we
>>   don't need; and inferring which modules are required by our
>>   application and bundling them without requiring the information
>>   `jslink' does. This would allow us to minimise the amount of code that
>>   needs to be sent over the network, which is very important to web
>>   developers.
>>
>>   This is a large task, and one I don't know enough about at the moment
>>   to attempt, but it is work that would benefit not just our JavaScript
>>   compiler, but people who want to deploy regular Guile applications.
>>
>>
>> [a blog post]
>> https://wingolog.org/archives/2016/02/04/guile-compiler-tasks
>>
>>
>> 6.4 JavaScript Version
>> ~~~~~~~~~~~~~~~~~~~~~~
>>
>>   I am not an expert in JavaScript, in fact, before this summer I
>>   probably hadn't written it for two years, which means the code
>>   certainly does not match up with the current best practices and
>>   specifications. Further, all of my testing for this compiler was done
>>   on [Node.js] v.6.10.3 only (this was the version available in the
>>   Fedora 25 repositories).
>>
>>   The code should be vetted to determine precisely which modern JS
>>   features are used (I believe proper tail calls, and ES6 Maps are the
>>   main ones), and it should be tested on all major browsers. If
>>   necessary, we should incorporate switches in the compiler to allow JS
>>   users to compile for particular implementations, taking advantage of
>>   particular modern JS features, or providing our own implementations of
>>   those that are not supported (e.g. Cheney on the MTA).
>>
>>
>> [Node.js] https://nodejs.org/en/
>>
>>
>> 6.5 JS Integration
>> ~~~~~~~~~~~~~~~~~~
>>
>>   One of the strengths of Guile is that it allows people to integrate
>>   their Scheme and C code, and although it has not been a focus for this
>>   summer, we should aim to provide similar levels of integration between
>>   Scheme and JS. There are two cases to consider.
>>
>>
>> 6.5.1 JS calling Scheme
>> -----------------------
>>
>>   As it stands, you can perform some limited interaction from JavaScript
>>   in a similar manner to how you would interact with Guile from C. For
>>   instance, by using `scm_current_module', `scm_public_lookup', and the
>>   `scheme.Symbol' constructor, one could look up a scheme function, e.g.
>>   `iota', and then invoke it by `scheme.call'.
>>
>>   That said, C idioms are not JS idioms, and so we should work to
>>   provide a much nicer API through the `scheme' object.
>>
>>
>> 6.5.2 Scheme calling JS
>> -----------------------
>>
>>   In the case of Scheme calling JavaScript, I think we should follow the
>>   example of `(system foreign)', which provides an API for linking to
>>   dynamic C libraries, and creating Scheme versions of C functions, and
>>   automatically marshalling/unmarshalling C types to Scheme types. One
>>   additional complication we would have with JS would be the presence of
>>   exceptions, but I think these could also be marshalled into Scheme
>>   ones without much trouble.
>>
>>
>> 7 Lessons Learned
>> =================
>>
>>   It goes without saying that a project like this teaches you a lot
>>   about the technical design of Guile, how to navigate the codebase,
>>   etc, but I want to highlight a few "softer" lessons from this summer.
>>
>>
>> 7.1 Compilers are "Easy", Runtimes are Hard
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   When I first set out to write this project two summers ago, I
>>   naturally assumed that the majority of the effort would go into the
>>   compiler, and much less into the built-ins. In reality, the effort was
>>   reversed. Partly this was due to my experience in writing Scheme, and
>>   Functional Programming more generally, meant that the tree-traversing
>>   code typical of a compiler pass was relatively straightforward, and
>>   the compiler was not doing a lot of optimisation, mostly code
>>   generation.
>>
>>
>> 7.2 Bootstrapping is Hard
>> ~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>   The last point leads into this one, bootstrapping is pretty tricky.
>>   With boot-9, you have several versions of the module system at
>>   different times. My own attempt to write module code that handled this
>>   ended up being abandoned for a rewrite that more closely followed the
>>   Guile C code. The size of the compiled boot-9 code, and the, at times,
>>   non-local consequences of implementing certain built-ins made it
>>   tricky to debug.
>>
>>
>> 7.3 Don't Panic
>> ~~~~~~~~~~~~~~~
>>
>>   This is a much more personal one, and one that I think is very
>>   important for anyone who wants to take part in a program like the
>>   Summer of Code, where you are spending a lot of time mostly on your
>>   own. In a complex software project, things are not always going to go
>>   smoothly. You might spend weeks banging up against a difficult
>>   problem. Don't Panic! If it was easy it would have already been done.
>>   Keep in Contact with your Mentor! It is tempting to only check in when
>>   you think you have something of progress to report, but they are there
>>   to help you, and explaining your issues to someone else is often very
>>   useful when trying to overcome them, even if they don't have an answer
>>   for you.
>>
>>
>> 8 Wrapping Up
>> =============
>>
>>   If you are still with me, good on you. As the new semester is starting
>>   I will be devoting much less time to this, and that will likely be
>>   true till December, but I will make an effort to keep up with
>>   guile-user and be on the IRC Channel to help the daring souls who want
>>   to give this a go. My priorities will be documenting the ILs, filling
>>   in missing builtins, and improving jslink. I especially want to see
>>   basic IO and MiniKanren up and running, and for it to be convenient to
>>   use Guile's builtin libraries.
>>
>>
>> Happy Hacking, Ian Price
>>
>> (This is a crosspost to guile-user of my blogpost [Summer of Code
>> Recap], but please comment on this list, rather than there)
>>
>> [Summer of Code Recap]
>> https://shift-reset.com/blog/2017/8/28/Summer%20of%20Code%20Recap/




^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2021-05-11 15:08   ` Christopher Lemmer Webber
@ 2021-05-11 15:19     ` Christopher Lemmer Webber
  2021-10-11  1:50       ` Christine Lemmer-Webber
  0 siblings, 1 reply; 16+ messages in thread
From: Christopher Lemmer Webber @ 2021-05-11 15:19 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user

I suspect what changed most of all is in commit
  4311dc9858ba7c6db50a851e95fc7c387b9381b2

Right now in compile-js.scm it does:

  (define lower-cps (@@ (language cps optimize) lower-cps))

  (define (compile-js exp env opts)
    ;; TODO: I should special case the compilation for the initial fun,
    ;; as this is the entry point for the program, and shouldn't get a
    ;; "self" argument, for now, I add "undefined" as the first
    ;; argument in the call to it.
    ;; see compile-exp in (language js-il compile-javascript)
    (define (intmap->program map)
      (intmap-fold-right (lambda (kfun body accum)
                           (acons (make-kid kfun)
                                  (compile-fun (intmap-select map body) kfun)
                                  accum))
                         (compute-reachable-functions map 0)
                         '()))
    (values (make-program (intmap->program (lower-cps exp opts))) env env))

That last line, with the lower-cps... well, it looks like assumptions
have changed.  Based on reading the commit history, it looks like this
step is now done *before* handing it over to
compile-bytecode/compile-js.

So my guess would be something like:

diff --git a/module/language/cps/compile-js.scm b/module/language/cps/compile-js.scm
index 128f5d64d..3c95c105f 100644
--- a/module/language/cps/compile-js.scm
+++ b/module/language/cps/compile-js.scm
@@ -44,7 +44,7 @@
                                 accum))
                        (compute-reachable-functions map 0)
                        '()))
-  (values (make-program (intmap->program (lower-cps exp opts))) env env))
+  (values (make-program (intmap->program exp)) env env))

This does not work, however.  Not sure why, or what really should be done...


Christopher Lemmer Webber writes:

> I've now verified that the place where things fall apart is fairly
> simple.  The following file does not compile:
>
>   (define (add x y)
>     (+ x y))
>
>   (add 1 2)
>
> So yeah, it's just functions in general.
>
> It looks like the stage where things are breaking is between the
> cps -> js-il representations.
>
> I figured since probably the changes need to happen in
> module/language/cps/compile-js.scm, I should look at
> the commit log in compile-bytecode.scm in that same directory.
> It looks like a lot has changed since 2017!
>
> I suspect I need help at this stage! :)
>
>
> Christopher Lemmer Webber writes:
>
>> Hi!
>>
>> Ian did some great work here in the past... let's not let it go to
>> waste.  Let's try to merge it!
>>
>> I've made a branch in my gitlab repo here:
>>
>>   https://gitlab.com/dustyweb/guile.git
>>
>> the branch is "compile-to-js-merge"
>>
>> I've dealt with the merge conflicts and etc I've been able to identify,
>> but things have already started to bitrot... I'd like to prevent them
>> from bitrotting further.  I fixed some things, updating the code to
>> where it appears things have shuffled around to as best as I could.
>>
>> Currently I can get a file as simple as "just-plus.scm" to compile:
>>
>>   (+ 1 2)
>>
>> This outputs to:
>>
>>   function (unit_cont){var k_0 = function (v_0,k_4){var k_1 = function (v_0){var v_1 = 3;return k_4(v_1);};if ((arguments["length"])==(2)) {{return k_1(v_0);}} else {{return undefined;}}};return k_0(undefined,unit_cont);};
>>
>> Progress!
>>
>> However, the amb.scm file no longer works as described below.  I get the
>> following:
>>
>>   In language/cps/intset.scm:
>>       472:6  3 (visit-branch #(4294967295 1073741823 #f #f #f #f #f #f (#f)) _ 0 # #)
>>       472:6  2 (visit-branch 4294967295 _ 0 _ _)
>>   In language/cps/split-rec.scm:
>>       78:22  1 (_ _ _ _)
>>   In ice-9/boot-9.scm:
>>     1685:16  0 (raise-exception _ #:continuable? _)
>>   
>>   ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>>   Throw to key `match-error' with args `("match" "no matching pattern" #<cps (const-fun 62)>)'.
>>
>> I guess that's something that probably changed.  I'm going to look into
>> it...
>>
>> Anyway, is there support from the maintainers from getting this merged
>> if I can get things working again?  I'd really like to see this effort
>> not go to waste... I'd even like to write a few demos using it.
>>
>>
>> Ian Price writes:
>>
>>> 1 Introduction
>>> ==============
>>>
>>>   As many of you are aware, I have been working on compiling Guile
>>>   Scheme to JavaScript this summer, as part of the Google Summer of
>>>   Code. This post serves to bookend my work for the year.
>>>
>>>   Before I go any further, I have to give my thanks to my mentor [Chris
>>>   Webber], without whom this project would have fizzled out weeks ago;
>>>   Google and the Gnu Project, naturally, for providing the Summer of
>>>   Code and allowing me to work on this project; and our fearless leader,
>>>   [Andy Wingo], for answering a wide variety of stupid questions.
>>>
>>>
>>> [Chris Webber] https://dustycloud.org/
>>>
>>> [Andy Wingo] https://wingolog.org/
>>>
>>>
>>> 2 Project Aims
>>> ==============
>>>
>>>   For a full introduction to the project, you can of course refer back
>>>   to my [project proposal], but very briefly my hopes for this summer
>>>   were:
>>>
>>>   1. To rewrite the previous version of my compiler from the [previous
>>>      CPS representation] to use the new representation ["CPS Soup"]
>>>      representation.
>>>   2. To completely port ice-9/boot-9.scm (our basic "prelude") to
>>>      JavaScript, and in particular, to support the [Guile Module
>>>      system].
>>>   3. To handle Proper Tail Calls by use of the [Cheney on the MTA]
>>>      strategy.
>>>   4. To include a new `guild' script for bundling compiled JS files with
>>>      their dependencies.
>>>
>>>
>>> [project proposal] https://shift-reset.com/static/docs/gsoc-2017.pdf
>>>
>>> [previous CPS representation]
>>> https://wingolog.org/archives/2014/01/12/a-continuation-passing-style-intermediate-language-for-guile
>>>
>>> ["CPS Soup"] https://wingolog.org/archives/2015/07/27/cps-soup
>>>
>>> [Guile Module system]
>>> https://www.gnu.org/software/guile/manual/html_node/Modules.html#Modules
>>>
>>> [Cheney on the MTA] http://www.pipeline.com/~hbaker1/CheneyMTA.html
>>>
>>>
>>> 3 What was Achieved
>>> ===================
>>>
>>>   You can find all of my work on the [compile-to-js-2017] branch of my
>>>   Gitlab. A full list of the commits can be found [here], but I will
>>>   summarise the changes now:
>>>
>>>
>>> [compile-to-js-2017]
>>> https://gitlab.com/ijp/guile/tree/compile-to-js-2017
>>>
>>> [here] https://gitlab.com/ijp/guile/compare/1b36a76e...gsoc-2017-end
>>>
>>> 3.1 Compile Guile CPS Soup to JavaScript
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   When I was working on my initial attempt at compiling Guile to
>>>   JavaScript, two years ago, Guile used a different CPS representation
>>>   as its intermediate language. The initial experiments with the CPS
>>>   Soup representation occurred while that work was ongoing, but as it
>>>   was not considered "stable", the plan was not to move to this
>>>   representation until after I had completed my other objectives.
>>>
>>>   Now, however, CPS Soup is the IL of Guile, and so the first task that
>>>   was accomplished was to move to this representation. Since I had
>>>   already created my own JS-IL as a target, I did not need to make any
>>>   changes to the code generation side from JS-IL to JavaScript proper.
>>>   The main change was to reconstruct the nested scope structure that was
>>>   implicit in the dominator structure that Guile made available.
>>>
>>>   The full code for the compiler is split into several sections,
>>>   corresponding to different stages in the compiler pipeline.
>>>
>>>
>>> 3.1.1 CPS to JS-IL Compiler
>>> ---------------------------
>>>
>>>   - module/language/cps/compile-js.scm
>>>   - module/language/cps/spec.scm
>>>
>>>   These modules constitute the compiler from CPS to my JS-IL
>>>   intermediate language.
>>>
>>>
>>> 3.1.2 JS-IL to JavaScript Compiler
>>> ----------------------------------
>>>
>>>   - module/language/js-il.scm
>>>   - module/language/js-il/compile-javascript.scm
>>>   - module/language/js-il/inlining.scm
>>>   - module/language/js-il/spec.scm
>>>
>>>   These modules constitute a somewhat ad-hoc intermediate representation
>>>   as a target for the CPS compiler. It differs from JavaScript, e.g., by
>>>   continuing to separate continuations and functions, and a slightly
>>>   specialised function representation to handle Guile's complicated
>>>   notion of procedure arity.
>>>
>>>
>>> 3.1.3 JavaScript Representation
>>> -------------------------------
>>>
>>>   - module/language/javascript.scm
>>>   - module/language/javascript/simplify.scm
>>>   - module/language/javascript/spec.scm
>>>
>>>   This is primarily the representation of JavaScript as Scheme Records.
>>>   This is separate from the representation of JavaScript Guile already
>>>   has in the form of `(language ecmascript)' primarily to avoid a
>>>   circularity when Guile determines which compilers to run in the
>>>   pipeline, as recommended by Andy Wingo.
>>>
>>>
>>> 3.2 A pre-amble capable of running through boot-9
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   In order to run Guile, it is not enough to be able to compile Scheme
>>>   (or indeed any other language supported by Guile) forms to JavaScript,
>>>   we also need to incorporate as much of Guile's runtime as possible.
>>>   This involves implementing VM primitives (such as you might see in
>>>   vm-engine.c); basic Guile types like Symbols, Pairs, and Structs; as
>>>   well as many of the functions that Guile implements in C rather than
>>>   Scheme.
>>>
>>>   Although I certainly did not implement all of the functionality Guile
>>>   achieves, I was able to implement sufficiently many (including what
>>>   amounts to a port of much of module.c) that one can successfully run
>>>   though ice-9/boot-9.scm from start to finish.
>>>
>>>   This took up the bulk of the time I spent on this project, due to the
>>>   size of the compiled output of boot-9.scm, and my own difficulties
>>>   debugging the bootstrap process. More on this below.
>>>
>>>   The code can be found at
>>>   - module/language/js-il/runtime.js
>>>
>>>
>>> 3.3 A linking script for JavaScript
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   Since we are using the `(language ...)' infrastructure, we can take
>>>   advantage of the existing `guild compile' script for compiling to
>>>   JavaScript, we simply need to use the `--to' switch. However, this
>>>   does not produce a file which you can just load up without any
>>>   additional work, especially if you are working with multiple modules.
>>>
>>>   In order to make it easier to deal with this, I have included a `guild
>>>   jslink' script, which can be used to package up a "main" script along
>>>   with the `runtime.js' and its dependencies. See below for an example.
>>>
>>>   The code can be found at
>>>   - module/scripts/jslink.scm
>>>
>>>
>>> 4 What was not Achieved
>>> =======================
>>>
>>> 4.1 Cheney on the MTA
>>> ~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   One of my regrets is that I did not implement Baker's "Cheney on the
>>>   MTA" (as seen in [Chicken Scheme]) for handling Proper Tail Calls in
>>>   JavaScript. Historically, JavaScript has not guaranteed that tail
>>>   position function calls do not grow the stack, and this is obviously
>>>   of fundamental importance for languages like Scheme. Fortunately, ES6
>>>   has added support for [proper tail calls] and we can expect to see
>>>   increased support for it in future JavaScript versions. (Indeed,
>>>   during testing on node v.6.10.3, I did not have to increase the stack
>>>   size until very late).
>>>
>>>
>>> [Chicken Scheme] https://www.call-cc.org/
>>>
>>> [proper tail calls]
>>> https://www.ecma-international.org/ecma-262/6.0/#sec-tail-position-calls
>>>
>>>
>>> 5 How to use it
>>> ===============
>>>
>>>   I've talked a lot about what I've did and didn't do, but what about
>>>   actually using this thing?
>>>
>>>
>>> 5.1 Obtaining the Code
>>> ~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   The code is not currently available from the main Guile repository,
>>>   but only the `compile-to-js-2017' branch on my [GitLab].
>>>
>>>   If you already have a checkout of guile, you can add my repo as a
>>>   remote with
>>>   ,----
>>>   | $ git remote add ijp https://gitlab.com/ijp/guile.git
>>>   `----
>>>   and fetch the branch with
>>>   ,----
>>>   | $ git fetch ijp
>>>   `----
>>>
>>>   You can then check out the `compile-to-js-2017' branch and build as
>>>   normal.
>>>
>>>
>>> [GitLab] https://gitlab.com/ijp/guile/tree/compile-to-js-2017
>>>
>>>
>>> 5.2 A Non-Trivial Example
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   As an example of how to use the JS Backend that is short, but
>>>   non-trivial, I am using John McCarthy's `amb' operator (see [A Basis
>>>   for a Mathematical Theory of Computation]) to search for Pythagorean
>>>   Triples.
>>>
>>>   First we have a module for the `amb' operator in amb.scm
>>>   ,----
>>>   | (define-module (amb)
>>>   |   #:export (amb fail))
>>>   |
>>>   | (define original-fail
>>>   |   (lambda _
>>>   |     (error 'amb "No more paths to search")))
>>>   |
>>>   | (define *amb-fail* original-fail)
>>>   |
>>>   | (define (fail)
>>>   |   (*amb-fail* #f))
>>>   |
>>>   | (define (amb-thunks . values)
>>>   |   (let ((failure *amb-fail*))
>>>   |     (call/cc (lambda (escape)
>>>   |                (for-each (lambda (value)
>>>   |                            (call/cc (lambda (continue)
>>>   |                                       (set! *amb-fail* continue)
>>>   |                                       (escape (value)))))
>>>   |                          values)
>>>   |                (failure #f)))))
>>>   |
>>>   | (define-syntax amb
>>>   |   (syntax-rules ()
>>>   |     ((amb exprs ...)
>>>   |      (amb-thunks (lambda () exprs) ...))))
>>>   `----
>>>
>>>   Next we have the code performs the search in triple.scm
>>>   ,----
>>>   | (use-modules (amb))
>>>   |
>>>   | (let ((a (amb 4 5 6 7 8 9 10))
>>>   |       (b (amb 4 5 6 7 8 9 10))
>>>   |       (c (amb 4 5 6 7 8 9 10)))
>>>   |   (if (= (* c c) (+ (* a a) (* b b)))
>>>   |       (list a b c)
>>>   |       (fail)))
>>>   `----
>>>
>>>   We compile the files in the usual manner, only now we specify the
>>>   `javascript' language (We make sure to add the current directory to
>>>   the load-path for triple.scm).
>>>
>>>   ,----
>>>   | $ guild compile amb.scm --to=javascript --output=amb.js
>>>   | $ guild compile -L . triple.scm --to=javascript --output=triple.js
>>>   `----
>>>
>>>   Next we link the two together into a file main.js, making sure to
>>>   specify amb.js as a dependency of triple.js. (This step will take a
>>>   little while, since it also compiles a bunch of dependencies)
>>>
>>>   ,----
>>>   | $ guild jslink triple.js -o main.js --depends="(\"amb\" . \"amb.scm\")"
>>>   `----
>>>
>>>   Finally, you can run it with `node', although as mentioned above you
>>>   may have to increase the stack size.
>>>
>>>   ,----
>>>   | $ node  --stack-size=2000 main.js
>>>   `----
>>>
>>>   Which should, fingers crossed, print out the triple 6,8,10.
>>>
>>>
>>> [A Basis for a Mathematical Theory of Computation]
>>> http://www-formal.stanford.edu/jmc/basis1.pdf
>>>
>>>
>>> 6 What is next?
>>> ===============
>>>
>>>   Having recapped what was and what was not achieved, the next question
>>>   is: where does the project go from here? I have been asked about my
>>>   plans for all sorts of features, e.g. support for [Web Assembly], but
>>>   I think the following things are the most important to think about.
>>>
>>>
>>> [Web Assembly] http://webassembly.org/
>>>
>>> 6.1 Inclusion into Guile
>>> ~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   The entire point of the project is to have something that can be
>>>   included in Guile proper. I have not spoken with Guile's maintainers
>>>   about incorporation into the main distribution, but I expect there
>>>   would be not be too many problems with moving the "official branch" to
>>>   the main repository.
>>>
>>>
>>> 6.2 All Guile built-ins in runtime.js
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   Although I have included enough to get though boot-9.scm, this does
>>>   not include all of the built-ins we would want in our programs. Two
>>>   things I use very often which do not appear in runtime.js are ports
>>>   and bytevectors.
>>>
>>>   We would like most, if not all, Guile built-ins to be available for
>>>   those who need them, so these will need to be implemented. However,
>>>   this is a lot of extra code for some people who don't need it, which
>>>   brings us to a different issue...
>>>
>>>
>>> 6.3 Linking Guile Modules & Features
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   In [a blog post], Andy Wingo lays out many tasks that he would like to
>>>   see in a future Guile. One of the most important of these, for us, are
>>>   under the headings "linking multiple modules together" and "linking a
>>>   single executable". To grossly simplify, we want to be able to link
>>>   various files into one single executable, which contains all and only
>>>   the code we need for our application.
>>>
>>>   As it stands, I included a simple script `guild jslink' that bundles
>>>   various compiled JavaScript files into one file, but we would like it
>>>   to be much more featureful: removing modules, functions, even types we
>>>   don't need; and inferring which modules are required by our
>>>   application and bundling them without requiring the information
>>>   `jslink' does. This would allow us to minimise the amount of code that
>>>   needs to be sent over the network, which is very important to web
>>>   developers.
>>>
>>>   This is a large task, and one I don't know enough about at the moment
>>>   to attempt, but it is work that would benefit not just our JavaScript
>>>   compiler, but people who want to deploy regular Guile applications.
>>>
>>>
>>> [a blog post]
>>> https://wingolog.org/archives/2016/02/04/guile-compiler-tasks
>>>
>>>
>>> 6.4 JavaScript Version
>>> ~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   I am not an expert in JavaScript, in fact, before this summer I
>>>   probably hadn't written it for two years, which means the code
>>>   certainly does not match up with the current best practices and
>>>   specifications. Further, all of my testing for this compiler was done
>>>   on [Node.js] v.6.10.3 only (this was the version available in the
>>>   Fedora 25 repositories).
>>>
>>>   The code should be vetted to determine precisely which modern JS
>>>   features are used (I believe proper tail calls, and ES6 Maps are the
>>>   main ones), and it should be tested on all major browsers. If
>>>   necessary, we should incorporate switches in the compiler to allow JS
>>>   users to compile for particular implementations, taking advantage of
>>>   particular modern JS features, or providing our own implementations of
>>>   those that are not supported (e.g. Cheney on the MTA).
>>>
>>>
>>> [Node.js] https://nodejs.org/en/
>>>
>>>
>>> 6.5 JS Integration
>>> ~~~~~~~~~~~~~~~~~~
>>>
>>>   One of the strengths of Guile is that it allows people to integrate
>>>   their Scheme and C code, and although it has not been a focus for this
>>>   summer, we should aim to provide similar levels of integration between
>>>   Scheme and JS. There are two cases to consider.
>>>
>>>
>>> 6.5.1 JS calling Scheme
>>> -----------------------
>>>
>>>   As it stands, you can perform some limited interaction from JavaScript
>>>   in a similar manner to how you would interact with Guile from C. For
>>>   instance, by using `scm_current_module', `scm_public_lookup', and the
>>>   `scheme.Symbol' constructor, one could look up a scheme function, e.g.
>>>   `iota', and then invoke it by `scheme.call'.
>>>
>>>   That said, C idioms are not JS idioms, and so we should work to
>>>   provide a much nicer API through the `scheme' object.
>>>
>>>
>>> 6.5.2 Scheme calling JS
>>> -----------------------
>>>
>>>   In the case of Scheme calling JavaScript, I think we should follow the
>>>   example of `(system foreign)', which provides an API for linking to
>>>   dynamic C libraries, and creating Scheme versions of C functions, and
>>>   automatically marshalling/unmarshalling C types to Scheme types. One
>>>   additional complication we would have with JS would be the presence of
>>>   exceptions, but I think these could also be marshalled into Scheme
>>>   ones without much trouble.
>>>
>>>
>>> 7 Lessons Learned
>>> =================
>>>
>>>   It goes without saying that a project like this teaches you a lot
>>>   about the technical design of Guile, how to navigate the codebase,
>>>   etc, but I want to highlight a few "softer" lessons from this summer.
>>>
>>>
>>> 7.1 Compilers are "Easy", Runtimes are Hard
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   When I first set out to write this project two summers ago, I
>>>   naturally assumed that the majority of the effort would go into the
>>>   compiler, and much less into the built-ins. In reality, the effort was
>>>   reversed. Partly this was due to my experience in writing Scheme, and
>>>   Functional Programming more generally, meant that the tree-traversing
>>>   code typical of a compiler pass was relatively straightforward, and
>>>   the compiler was not doing a lot of optimisation, mostly code
>>>   generation.
>>>
>>>
>>> 7.2 Bootstrapping is Hard
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>   The last point leads into this one, bootstrapping is pretty tricky.
>>>   With boot-9, you have several versions of the module system at
>>>   different times. My own attempt to write module code that handled this
>>>   ended up being abandoned for a rewrite that more closely followed the
>>>   Guile C code. The size of the compiled boot-9 code, and the, at times,
>>>   non-local consequences of implementing certain built-ins made it
>>>   tricky to debug.
>>>
>>>
>>> 7.3 Don't Panic
>>> ~~~~~~~~~~~~~~~
>>>
>>>   This is a much more personal one, and one that I think is very
>>>   important for anyone who wants to take part in a program like the
>>>   Summer of Code, where you are spending a lot of time mostly on your
>>>   own. In a complex software project, things are not always going to go
>>>   smoothly. You might spend weeks banging up against a difficult
>>>   problem. Don't Panic! If it was easy it would have already been done.
>>>   Keep in Contact with your Mentor! It is tempting to only check in when
>>>   you think you have something of progress to report, but they are there
>>>   to help you, and explaining your issues to someone else is often very
>>>   useful when trying to overcome them, even if they don't have an answer
>>>   for you.
>>>
>>>
>>> 8 Wrapping Up
>>> =============
>>>
>>>   If you are still with me, good on you. As the new semester is starting
>>>   I will be devoting much less time to this, and that will likely be
>>>   true till December, but I will make an effort to keep up with
>>>   guile-user and be on the IRC Channel to help the daring souls who want
>>>   to give this a go. My priorities will be documenting the ILs, filling
>>>   in missing builtins, and improving jslink. I especially want to see
>>>   basic IO and MiniKanren up and running, and for it to be convenient to
>>>   use Guile's builtin libraries.
>>>
>>>
>>> Happy Hacking, Ian Price
>>>
>>> (This is a crosspost to guile-user of my blogpost [Summer of Code
>>> Recap], but please comment on this list, rather than there)
>>>
>>> [Summer of Code Recap]
>>> https://shift-reset.com/blog/2017/8/28/Summer%20of%20Code%20Recap/




^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2021-05-11 14:45 ` Christopher Lemmer Webber
  2021-05-11 15:08   ` Christopher Lemmer Webber
@ 2021-05-11 15:50   ` Dr. Arne Babenhauserheide
  1 sibling, 0 replies; 16+ messages in thread
From: Dr. Arne Babenhauserheide @ 2021-05-11 15:50 UTC (permalink / raw)
  To: Christopher Lemmer Webber; +Cc: guile-user, Ian Price

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]


Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

> Anyway, is there support from the maintainers from getting this merged
> if I can get things working again?  I'd really like to see this effort
> not go to waste... I'd even like to write a few demos using it.

I’m no maintainer, but I would love having the js backend merged! I
tried biwascheme to write Scheme on the web[1] but found it lacking (it
combines the gotchas from javascript (no hygiene) with the gotchas from
transpiling), and I would love to have a canonical source of Guile-code
for Javascript-tools on my website and for small games.

Best wishes,
Arne

[1]: This pseudo-random language-assignment generator is written in biwascheme:
     https://www.draketo.de/software/vorlesung-netztechnik#nummer-zu-sprache
     https://www.draketo.de/software/matrikellanguage.scm
     <script src="biwascheme.js">
       (load "matrikellanguage.scm")
       (let loop ()
           (wait-for "#matrikelnummer" "input")
           (set-content! "#result" (matrikel->pair (get-content "#matrikelnummer")))
           (console-log "ok.")
           (loop))
     </script>
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2021-05-11 15:19     ` Christopher Lemmer Webber
@ 2021-10-11  1:50       ` Christine Lemmer-Webber
  2021-10-11  5:28         ` Dr. Arne Babenhauserheide
  0 siblings, 1 reply; 16+ messages in thread
From: Christine Lemmer-Webber @ 2021-10-11  1:50 UTC (permalink / raw)
  To: Ian Price; +Cc: guile-user

I have pushed one more merge with master to the compile-to-js-merge
branch.  I've taken the bold move of pushing this to origin... I think
it's a good idea to try to get this in, so it's worth it not just
sitting in my own personal repo.

Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

> I suspect what changed most of all is in commit
>   4311dc9858ba7c6db50a851e95fc7c387b9381b2
>
> Right now in compile-js.scm it does:
>
>   (define lower-cps (@@ (language cps optimize) lower-cps))
>
>   (define (compile-js exp env opts)
>     ;; TODO: I should special case the compilation for the initial fun,
>     ;; as this is the entry point for the program, and shouldn't get a
>     ;; "self" argument, for now, I add "undefined" as the first
>     ;; argument in the call to it.
>     ;; see compile-exp in (language js-il compile-javascript)
>     (define (intmap->program map)
>       (intmap-fold-right (lambda (kfun body accum)
>                            (acons (make-kid kfun)
>                                   (compile-fun (intmap-select map body) kfun)
>                                   accum))
>                          (compute-reachable-functions map 0)
>                          '()))
>     (values (make-program (intmap->program (lower-cps exp opts))) env env))
>
> That last line, with the lower-cps... well, it looks like assumptions
> have changed.  Based on reading the commit history, it looks like this
> step is now done *before* handing it over to
> compile-bytecode/compile-js.
>
> So my guess would be something like:
>
> diff --git a/module/language/cps/compile-js.scm b/module/language/cps/compile-js.scm
> index 128f5d64d..3c95c105f 100644
> --- a/module/language/cps/compile-js.scm
> +++ b/module/language/cps/compile-js.scm
> @@ -44,7 +44,7 @@
>                                  accum))
>                         (compute-reachable-functions map 0)
>                         '()))
> -  (values (make-program (intmap->program (lower-cps exp opts))) env env))
> +  (values (make-program (intmap->program exp)) env env))
>
> This does not work, however.  Not sure why, or what really should be done...
>
>
> Christopher Lemmer Webber writes:
>
>> I've now verified that the place where things fall apart is fairly
>> simple.  The following file does not compile:
>>
>>   (define (add x y)
>>     (+ x y))
>>
>>   (add 1 2)
>>
>> So yeah, it's just functions in general.
>>
>> It looks like the stage where things are breaking is between the
>> cps -> js-il representations.
>>
>> I figured since probably the changes need to happen in
>> module/language/cps/compile-js.scm, I should look at
>> the commit log in compile-bytecode.scm in that same directory.
>> It looks like a lot has changed since 2017!
>>
>> I suspect I need help at this stage! :)
>>
>>
>> Christopher Lemmer Webber writes:
>>
>>> Hi!
>>>
>>> Ian did some great work here in the past... let's not let it go to
>>> waste.  Let's try to merge it!
>>>
>>> I've made a branch in my gitlab repo here:
>>>
>>>   https://gitlab.com/dustyweb/guile.git
>>>
>>> the branch is "compile-to-js-merge"
>>>
>>> I've dealt with the merge conflicts and etc I've been able to identify,
>>> but things have already started to bitrot... I'd like to prevent them
>>> from bitrotting further.  I fixed some things, updating the code to
>>> where it appears things have shuffled around to as best as I could.
>>>
>>> Currently I can get a file as simple as "just-plus.scm" to compile:
>>>
>>>   (+ 1 2)
>>>
>>> This outputs to:
>>>
>>>   function (unit_cont){var k_0 = function (v_0,k_4){var k_1 = function (v_0){var v_1 = 3;return k_4(v_1);};if ((arguments["length"])==(2)) {{return k_1(v_0);}} else {{return undefined;}}};return k_0(undefined,unit_cont);};
>>>
>>> Progress!
>>>
>>> However, the amb.scm file no longer works as described below.  I get the
>>> following:
>>>
>>>   In language/cps/intset.scm:
>>>       472:6  3 (visit-branch #(4294967295 1073741823 #f #f #f #f #f #f (#f)) _ 0 # #)
>>>       472:6  2 (visit-branch 4294967295 _ 0 _ _)
>>>   In language/cps/split-rec.scm:
>>>       78:22  1 (_ _ _ _)
>>>   In ice-9/boot-9.scm:
>>>     1685:16  0 (raise-exception _ #:continuable? _)
>>>   
>>>   ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>>>   Throw to key `match-error' with args `("match" "no matching pattern" #<cps (const-fun 62)>)'.
>>>
>>> I guess that's something that probably changed.  I'm going to look into
>>> it...
>>>
>>> Anyway, is there support from the maintainers from getting this merged
>>> if I can get things working again?  I'd really like to see this effort
>>> not go to waste... I'd even like to write a few demos using it.
>>>
>>>
>>> Ian Price writes:
>>>
>>>> 1 Introduction
>>>> ==============
>>>>
>>>>   As many of you are aware, I have been working on compiling Guile
>>>>   Scheme to JavaScript this summer, as part of the Google Summer of
>>>>   Code. This post serves to bookend my work for the year.
>>>>
>>>>   Before I go any further, I have to give my thanks to my mentor [Chris
>>>>   Webber], without whom this project would have fizzled out weeks ago;
>>>>   Google and the Gnu Project, naturally, for providing the Summer of
>>>>   Code and allowing me to work on this project; and our fearless leader,
>>>>   [Andy Wingo], for answering a wide variety of stupid questions.
>>>>
>>>>
>>>> [Chris Webber] https://dustycloud.org/
>>>>
>>>> [Andy Wingo] https://wingolog.org/
>>>>
>>>>
>>>> 2 Project Aims
>>>> ==============
>>>>
>>>>   For a full introduction to the project, you can of course refer back
>>>>   to my [project proposal], but very briefly my hopes for this summer
>>>>   were:
>>>>
>>>>   1. To rewrite the previous version of my compiler from the [previous
>>>>      CPS representation] to use the new representation ["CPS Soup"]
>>>>      representation.
>>>>   2. To completely port ice-9/boot-9.scm (our basic "prelude") to
>>>>      JavaScript, and in particular, to support the [Guile Module
>>>>      system].
>>>>   3. To handle Proper Tail Calls by use of the [Cheney on the MTA]
>>>>      strategy.
>>>>   4. To include a new `guild' script for bundling compiled JS files with
>>>>      their dependencies.
>>>>
>>>>
>>>> [project proposal] https://shift-reset.com/static/docs/gsoc-2017.pdf
>>>>
>>>> [previous CPS representation]
>>>> https://wingolog.org/archives/2014/01/12/a-continuation-passing-style-intermediate-language-for-guile
>>>>
>>>> ["CPS Soup"] https://wingolog.org/archives/2015/07/27/cps-soup
>>>>
>>>> [Guile Module system]
>>>> https://www.gnu.org/software/guile/manual/html_node/Modules.html#Modules
>>>>
>>>> [Cheney on the MTA] http://www.pipeline.com/~hbaker1/CheneyMTA.html
>>>>
>>>>
>>>> 3 What was Achieved
>>>> ===================
>>>>
>>>>   You can find all of my work on the [compile-to-js-2017] branch of my
>>>>   Gitlab. A full list of the commits can be found [here], but I will
>>>>   summarise the changes now:
>>>>
>>>>
>>>> [compile-to-js-2017]
>>>> https://gitlab.com/ijp/guile/tree/compile-to-js-2017
>>>>
>>>> [here] https://gitlab.com/ijp/guile/compare/1b36a76e...gsoc-2017-end
>>>>
>>>> 3.1 Compile Guile CPS Soup to JavaScript
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   When I was working on my initial attempt at compiling Guile to
>>>>   JavaScript, two years ago, Guile used a different CPS representation
>>>>   as its intermediate language. The initial experiments with the CPS
>>>>   Soup representation occurred while that work was ongoing, but as it
>>>>   was not considered "stable", the plan was not to move to this
>>>>   representation until after I had completed my other objectives.
>>>>
>>>>   Now, however, CPS Soup is the IL of Guile, and so the first task that
>>>>   was accomplished was to move to this representation. Since I had
>>>>   already created my own JS-IL as a target, I did not need to make any
>>>>   changes to the code generation side from JS-IL to JavaScript proper.
>>>>   The main change was to reconstruct the nested scope structure that was
>>>>   implicit in the dominator structure that Guile made available.
>>>>
>>>>   The full code for the compiler is split into several sections,
>>>>   corresponding to different stages in the compiler pipeline.
>>>>
>>>>
>>>> 3.1.1 CPS to JS-IL Compiler
>>>> ---------------------------
>>>>
>>>>   - module/language/cps/compile-js.scm
>>>>   - module/language/cps/spec.scm
>>>>
>>>>   These modules constitute the compiler from CPS to my JS-IL
>>>>   intermediate language.
>>>>
>>>>
>>>> 3.1.2 JS-IL to JavaScript Compiler
>>>> ----------------------------------
>>>>
>>>>   - module/language/js-il.scm
>>>>   - module/language/js-il/compile-javascript.scm
>>>>   - module/language/js-il/inlining.scm
>>>>   - module/language/js-il/spec.scm
>>>>
>>>>   These modules constitute a somewhat ad-hoc intermediate representation
>>>>   as a target for the CPS compiler. It differs from JavaScript, e.g., by
>>>>   continuing to separate continuations and functions, and a slightly
>>>>   specialised function representation to handle Guile's complicated
>>>>   notion of procedure arity.
>>>>
>>>>
>>>> 3.1.3 JavaScript Representation
>>>> -------------------------------
>>>>
>>>>   - module/language/javascript.scm
>>>>   - module/language/javascript/simplify.scm
>>>>   - module/language/javascript/spec.scm
>>>>
>>>>   This is primarily the representation of JavaScript as Scheme Records.
>>>>   This is separate from the representation of JavaScript Guile already
>>>>   has in the form of `(language ecmascript)' primarily to avoid a
>>>>   circularity when Guile determines which compilers to run in the
>>>>   pipeline, as recommended by Andy Wingo.
>>>>
>>>>
>>>> 3.2 A pre-amble capable of running through boot-9
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   In order to run Guile, it is not enough to be able to compile Scheme
>>>>   (or indeed any other language supported by Guile) forms to JavaScript,
>>>>   we also need to incorporate as much of Guile's runtime as possible.
>>>>   This involves implementing VM primitives (such as you might see in
>>>>   vm-engine.c); basic Guile types like Symbols, Pairs, and Structs; as
>>>>   well as many of the functions that Guile implements in C rather than
>>>>   Scheme.
>>>>
>>>>   Although I certainly did not implement all of the functionality Guile
>>>>   achieves, I was able to implement sufficiently many (including what
>>>>   amounts to a port of much of module.c) that one can successfully run
>>>>   though ice-9/boot-9.scm from start to finish.
>>>>
>>>>   This took up the bulk of the time I spent on this project, due to the
>>>>   size of the compiled output of boot-9.scm, and my own difficulties
>>>>   debugging the bootstrap process. More on this below.
>>>>
>>>>   The code can be found at
>>>>   - module/language/js-il/runtime.js
>>>>
>>>>
>>>> 3.3 A linking script for JavaScript
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   Since we are using the `(language ...)' infrastructure, we can take
>>>>   advantage of the existing `guild compile' script for compiling to
>>>>   JavaScript, we simply need to use the `--to' switch. However, this
>>>>   does not produce a file which you can just load up without any
>>>>   additional work, especially if you are working with multiple modules.
>>>>
>>>>   In order to make it easier to deal with this, I have included a `guild
>>>>   jslink' script, which can be used to package up a "main" script along
>>>>   with the `runtime.js' and its dependencies. See below for an example.
>>>>
>>>>   The code can be found at
>>>>   - module/scripts/jslink.scm
>>>>
>>>>
>>>> 4 What was not Achieved
>>>> =======================
>>>>
>>>> 4.1 Cheney on the MTA
>>>> ~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   One of my regrets is that I did not implement Baker's "Cheney on the
>>>>   MTA" (as seen in [Chicken Scheme]) for handling Proper Tail Calls in
>>>>   JavaScript. Historically, JavaScript has not guaranteed that tail
>>>>   position function calls do not grow the stack, and this is obviously
>>>>   of fundamental importance for languages like Scheme. Fortunately, ES6
>>>>   has added support for [proper tail calls] and we can expect to see
>>>>   increased support for it in future JavaScript versions. (Indeed,
>>>>   during testing on node v.6.10.3, I did not have to increase the stack
>>>>   size until very late).
>>>>
>>>>
>>>> [Chicken Scheme] https://www.call-cc.org/
>>>>
>>>> [proper tail calls]
>>>> https://www.ecma-international.org/ecma-262/6.0/#sec-tail-position-calls
>>>>
>>>>
>>>> 5 How to use it
>>>> ===============
>>>>
>>>>   I've talked a lot about what I've did and didn't do, but what about
>>>>   actually using this thing?
>>>>
>>>>
>>>> 5.1 Obtaining the Code
>>>> ~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   The code is not currently available from the main Guile repository,
>>>>   but only the `compile-to-js-2017' branch on my [GitLab].
>>>>
>>>>   If you already have a checkout of guile, you can add my repo as a
>>>>   remote with
>>>>   ,----
>>>>   | $ git remote add ijp https://gitlab.com/ijp/guile.git
>>>>   `----
>>>>   and fetch the branch with
>>>>   ,----
>>>>   | $ git fetch ijp
>>>>   `----
>>>>
>>>>   You can then check out the `compile-to-js-2017' branch and build as
>>>>   normal.
>>>>
>>>>
>>>> [GitLab] https://gitlab.com/ijp/guile/tree/compile-to-js-2017
>>>>
>>>>
>>>> 5.2 A Non-Trivial Example
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   As an example of how to use the JS Backend that is short, but
>>>>   non-trivial, I am using John McCarthy's `amb' operator (see [A Basis
>>>>   for a Mathematical Theory of Computation]) to search for Pythagorean
>>>>   Triples.
>>>>
>>>>   First we have a module for the `amb' operator in amb.scm
>>>>   ,----
>>>>   | (define-module (amb)
>>>>   |   #:export (amb fail))
>>>>   |
>>>>   | (define original-fail
>>>>   |   (lambda _
>>>>   |     (error 'amb "No more paths to search")))
>>>>   |
>>>>   | (define *amb-fail* original-fail)
>>>>   |
>>>>   | (define (fail)
>>>>   |   (*amb-fail* #f))
>>>>   |
>>>>   | (define (amb-thunks . values)
>>>>   |   (let ((failure *amb-fail*))
>>>>   |     (call/cc (lambda (escape)
>>>>   |                (for-each (lambda (value)
>>>>   |                            (call/cc (lambda (continue)
>>>>   |                                       (set! *amb-fail* continue)
>>>>   |                                       (escape (value)))))
>>>>   |                          values)
>>>>   |                (failure #f)))))
>>>>   |
>>>>   | (define-syntax amb
>>>>   |   (syntax-rules ()
>>>>   |     ((amb exprs ...)
>>>>   |      (amb-thunks (lambda () exprs) ...))))
>>>>   `----
>>>>
>>>>   Next we have the code performs the search in triple.scm
>>>>   ,----
>>>>   | (use-modules (amb))
>>>>   |
>>>>   | (let ((a (amb 4 5 6 7 8 9 10))
>>>>   |       (b (amb 4 5 6 7 8 9 10))
>>>>   |       (c (amb 4 5 6 7 8 9 10)))
>>>>   |   (if (= (* c c) (+ (* a a) (* b b)))
>>>>   |       (list a b c)
>>>>   |       (fail)))
>>>>   `----
>>>>
>>>>   We compile the files in the usual manner, only now we specify the
>>>>   `javascript' language (We make sure to add the current directory to
>>>>   the load-path for triple.scm).
>>>>
>>>>   ,----
>>>>   | $ guild compile amb.scm --to=javascript --output=amb.js
>>>>   | $ guild compile -L . triple.scm --to=javascript --output=triple.js
>>>>   `----
>>>>
>>>>   Next we link the two together into a file main.js, making sure to
>>>>   specify amb.js as a dependency of triple.js. (This step will take a
>>>>   little while, since it also compiles a bunch of dependencies)
>>>>
>>>>   ,----
>>>>   | $ guild jslink triple.js -o main.js --depends="(\"amb\" . \"amb.scm\")"
>>>>   `----
>>>>
>>>>   Finally, you can run it with `node', although as mentioned above you
>>>>   may have to increase the stack size.
>>>>
>>>>   ,----
>>>>   | $ node  --stack-size=2000 main.js
>>>>   `----
>>>>
>>>>   Which should, fingers crossed, print out the triple 6,8,10.
>>>>
>>>>
>>>> [A Basis for a Mathematical Theory of Computation]
>>>> http://www-formal.stanford.edu/jmc/basis1.pdf
>>>>
>>>>
>>>> 6 What is next?
>>>> ===============
>>>>
>>>>   Having recapped what was and what was not achieved, the next question
>>>>   is: where does the project go from here? I have been asked about my
>>>>   plans for all sorts of features, e.g. support for [Web Assembly], but
>>>>   I think the following things are the most important to think about.
>>>>
>>>>
>>>> [Web Assembly] http://webassembly.org/
>>>>
>>>> 6.1 Inclusion into Guile
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   The entire point of the project is to have something that can be
>>>>   included in Guile proper. I have not spoken with Guile's maintainers
>>>>   about incorporation into the main distribution, but I expect there
>>>>   would be not be too many problems with moving the "official branch" to
>>>>   the main repository.
>>>>
>>>>
>>>> 6.2 All Guile built-ins in runtime.js
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   Although I have included enough to get though boot-9.scm, this does
>>>>   not include all of the built-ins we would want in our programs. Two
>>>>   things I use very often which do not appear in runtime.js are ports
>>>>   and bytevectors.
>>>>
>>>>   We would like most, if not all, Guile built-ins to be available for
>>>>   those who need them, so these will need to be implemented. However,
>>>>   this is a lot of extra code for some people who don't need it, which
>>>>   brings us to a different issue...
>>>>
>>>>
>>>> 6.3 Linking Guile Modules & Features
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   In [a blog post], Andy Wingo lays out many tasks that he would like to
>>>>   see in a future Guile. One of the most important of these, for us, are
>>>>   under the headings "linking multiple modules together" and "linking a
>>>>   single executable". To grossly simplify, we want to be able to link
>>>>   various files into one single executable, which contains all and only
>>>>   the code we need for our application.
>>>>
>>>>   As it stands, I included a simple script `guild jslink' that bundles
>>>>   various compiled JavaScript files into one file, but we would like it
>>>>   to be much more featureful: removing modules, functions, even types we
>>>>   don't need; and inferring which modules are required by our
>>>>   application and bundling them without requiring the information
>>>>   `jslink' does. This would allow us to minimise the amount of code that
>>>>   needs to be sent over the network, which is very important to web
>>>>   developers.
>>>>
>>>>   This is a large task, and one I don't know enough about at the moment
>>>>   to attempt, but it is work that would benefit not just our JavaScript
>>>>   compiler, but people who want to deploy regular Guile applications.
>>>>
>>>>
>>>> [a blog post]
>>>> https://wingolog.org/archives/2016/02/04/guile-compiler-tasks
>>>>
>>>>
>>>> 6.4 JavaScript Version
>>>> ~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   I am not an expert in JavaScript, in fact, before this summer I
>>>>   probably hadn't written it for two years, which means the code
>>>>   certainly does not match up with the current best practices and
>>>>   specifications. Further, all of my testing for this compiler was done
>>>>   on [Node.js] v.6.10.3 only (this was the version available in the
>>>>   Fedora 25 repositories).
>>>>
>>>>   The code should be vetted to determine precisely which modern JS
>>>>   features are used (I believe proper tail calls, and ES6 Maps are the
>>>>   main ones), and it should be tested on all major browsers. If
>>>>   necessary, we should incorporate switches in the compiler to allow JS
>>>>   users to compile for particular implementations, taking advantage of
>>>>   particular modern JS features, or providing our own implementations of
>>>>   those that are not supported (e.g. Cheney on the MTA).
>>>>
>>>>
>>>> [Node.js] https://nodejs.org/en/
>>>>
>>>>
>>>> 6.5 JS Integration
>>>> ~~~~~~~~~~~~~~~~~~
>>>>
>>>>   One of the strengths of Guile is that it allows people to integrate
>>>>   their Scheme and C code, and although it has not been a focus for this
>>>>   summer, we should aim to provide similar levels of integration between
>>>>   Scheme and JS. There are two cases to consider.
>>>>
>>>>
>>>> 6.5.1 JS calling Scheme
>>>> -----------------------
>>>>
>>>>   As it stands, you can perform some limited interaction from JavaScript
>>>>   in a similar manner to how you would interact with Guile from C. For
>>>>   instance, by using `scm_current_module', `scm_public_lookup', and the
>>>>   `scheme.Symbol' constructor, one could look up a scheme function, e.g.
>>>>   `iota', and then invoke it by `scheme.call'.
>>>>
>>>>   That said, C idioms are not JS idioms, and so we should work to
>>>>   provide a much nicer API through the `scheme' object.
>>>>
>>>>
>>>> 6.5.2 Scheme calling JS
>>>> -----------------------
>>>>
>>>>   In the case of Scheme calling JavaScript, I think we should follow the
>>>>   example of `(system foreign)', which provides an API for linking to
>>>>   dynamic C libraries, and creating Scheme versions of C functions, and
>>>>   automatically marshalling/unmarshalling C types to Scheme types. One
>>>>   additional complication we would have with JS would be the presence of
>>>>   exceptions, but I think these could also be marshalled into Scheme
>>>>   ones without much trouble.
>>>>
>>>>
>>>> 7 Lessons Learned
>>>> =================
>>>>
>>>>   It goes without saying that a project like this teaches you a lot
>>>>   about the technical design of Guile, how to navigate the codebase,
>>>>   etc, but I want to highlight a few "softer" lessons from this summer.
>>>>
>>>>
>>>> 7.1 Compilers are "Easy", Runtimes are Hard
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   When I first set out to write this project two summers ago, I
>>>>   naturally assumed that the majority of the effort would go into the
>>>>   compiler, and much less into the built-ins. In reality, the effort was
>>>>   reversed. Partly this was due to my experience in writing Scheme, and
>>>>   Functional Programming more generally, meant that the tree-traversing
>>>>   code typical of a compiler pass was relatively straightforward, and
>>>>   the compiler was not doing a lot of optimisation, mostly code
>>>>   generation.
>>>>
>>>>
>>>> 7.2 Bootstrapping is Hard
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>>   The last point leads into this one, bootstrapping is pretty tricky.
>>>>   With boot-9, you have several versions of the module system at
>>>>   different times. My own attempt to write module code that handled this
>>>>   ended up being abandoned for a rewrite that more closely followed the
>>>>   Guile C code. The size of the compiled boot-9 code, and the, at times,
>>>>   non-local consequences of implementing certain built-ins made it
>>>>   tricky to debug.
>>>>
>>>>
>>>> 7.3 Don't Panic
>>>> ~~~~~~~~~~~~~~~
>>>>
>>>>   This is a much more personal one, and one that I think is very
>>>>   important for anyone who wants to take part in a program like the
>>>>   Summer of Code, where you are spending a lot of time mostly on your
>>>>   own. In a complex software project, things are not always going to go
>>>>   smoothly. You might spend weeks banging up against a difficult
>>>>   problem. Don't Panic! If it was easy it would have already been done.
>>>>   Keep in Contact with your Mentor! It is tempting to only check in when
>>>>   you think you have something of progress to report, but they are there
>>>>   to help you, and explaining your issues to someone else is often very
>>>>   useful when trying to overcome them, even if they don't have an answer
>>>>   for you.
>>>>
>>>>
>>>> 8 Wrapping Up
>>>> =============
>>>>
>>>>   If you are still with me, good on you. As the new semester is starting
>>>>   I will be devoting much less time to this, and that will likely be
>>>>   true till December, but I will make an effort to keep up with
>>>>   guile-user and be on the IRC Channel to help the daring souls who want
>>>>   to give this a go. My priorities will be documenting the ILs, filling
>>>>   in missing builtins, and improving jslink. I especially want to see
>>>>   basic IO and MiniKanren up and running, and for it to be convenient to
>>>>   use Guile's builtin libraries.
>>>>
>>>>
>>>> Happy Hacking, Ian Price
>>>>
>>>> (This is a crosspost to guile-user of my blogpost [Summer of Code
>>>> Recap], but please comment on this list, rather than there)
>>>>
>>>> [Summer of Code Recap]
>>>> https://shift-reset.com/blog/2017/8/28/Summer%20of%20Code%20Recap/




^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Summer of Code Recap
  2021-10-11  1:50       ` Christine Lemmer-Webber
@ 2021-10-11  5:28         ` Dr. Arne Babenhauserheide
  0 siblings, 0 replies; 16+ messages in thread
From: Dr. Arne Babenhauserheide @ 2021-10-11  5:28 UTC (permalink / raw)
  To: Christine Lemmer-Webber; +Cc: guile-user, Ian Price

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]


Christine Lemmer-Webber <cwebber@dustycloud.org> writes:

> I have pushed one more merge with master to the compile-to-js-merge
> branch.  I've taken the bold move of pushing this to origin... I think
> it's a good idea to try to get this in, so it's worth it not just
> sitting in my own personal repo.

wow — thank you!

Is there a short example on how to use this branch for the features that
already work?

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2021-10-11  5:28 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-28 18:56 Summer of Code Recap Ian Price
2017-08-28 19:25 ` Christopher Allan Webber
2017-08-28 19:58   ` Nala Ginrut
2017-08-28 21:39 ` Amirouche
2017-08-29  3:27   ` Christopher Allan Webber
2017-09-01 22:09 ` Amirouche Boubekki
2017-09-02  6:58   ` Arne Babenhauserheide
2017-09-06 18:25   ` Amirouche Boubekki
2017-09-07  6:32     ` Amirouche Boubekki
2017-09-08 12:18       ` Amirouche Boubekki
2021-05-11 14:45 ` Christopher Lemmer Webber
2021-05-11 15:08   ` Christopher Lemmer Webber
2021-05-11 15:19     ` Christopher Lemmer Webber
2021-10-11  1:50       ` Christine Lemmer-Webber
2021-10-11  5:28         ` Dr. Arne Babenhauserheide
2021-05-11 15:50   ` Dr. Arne Babenhauserheide

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).