How best to produce a standalone executable with Guile?

How best to produce a standalone executable with Guile?

[James Cooper]

Hi Guile folks,

I am currently experimenting with a handful of languages that include
modern implementations of Concurrent ML, to determine which one appears
best able to handle computationally- and message-heavy workloads (at least
for my purposes, Computer Vision).  Guile is one of the languages I am
trying out with the Fibers library.

My question is, how can I best go about creating Linux executables from
Guile, where said programs benefit from the most ahead-of-time compilation
and optimisation possible?  I don't believe they strictly need to be
standalone, as the environment I will be running the programs in will
likely be the same as the development environment. So they can undoubtedly
have dependencies on external libraries.  Furthermore, I am perfectly happy
with the process being a bit convoluted, as I can easily automate it with a
makefile.  I would, however, prefer if at all possible that the option to
target other processor architectures remains available down the line (I
assume using GCC can enable this, but I'm not sure).

I have looked at all the sections of the Guile manual that appeared
relevant, but to be honest I'm still a bit confused about what the best
approach would be.  *s* *9.4.5 Bytecode* in the manual seems to suggest
that one can use guild to produce what will be a standalone executable that
looks like an ELF binary to the system and runs on the Guile VM, but I'm
entirely sure if I am reading it correctly.  *s 6.18.5 Compiling Scheme
Code* also discusses compilation but seems to denounce the practice
explicitly.  *s 9.3.8 Just-In-Time Native Code* talks about JIT compilation
to native code for a given target.  Is it possible to apply this process
ahead-of-time?

A potential alternative approach I can see is using the capabilities of
running Guile code in a C program (compiled with GCC).  This would
presumably involve creating a template C program that just calls the
appropriate Guile functions and passes the relevant command line parameters
in.

Any thoughts on what would likely produce the most efficient executable in
the end?  The target programs do not need to modify themselves at run-time
(or really do anything dynamic) if that makes a difference.  And does
anyone know of pre-existing examples they could point me to?  Please do let
me know if there is further information I could provide that would help.
Thanks in advance.

Regards,

James

Re: How best to produce a standalone executable with Guile?

[Matt Wette]

On 4/4/20 10:28 PM, James Cooper wrote:
> Hi Guile folks,
>
> I am currently experimenting with a handful of languages that include
> modern implementations of Concurrent ML, to determine which one appears
> best able to handle computationally- and message-heavy workloads (at least
> for my purposes, Computer Vision).  Guile is one of the languages I am
> trying out with the Fibers library.
>
> My question is, how can I best go about creating Linux executables from
> Guile, where said programs benefit from the most ahead-of-time compilation
> and optimisation possible?  I don't believe they strictly need to be
> standalone, as the environment I will be running the programs in will
> likely be the same as the development environment. So they can undoubtedly
> have dependencies on external libraries.  Furthermore, I am perfectly happy
> with the process being a bit convoluted, as I can easily automate it with a
> makefile.  I would, however, prefer if at all possible that the option to
> target other processor architectures remains available down the line (I
> assume using GCC can enable this, but I'm not sure).
>
> [...]
You can't produce a pre-compiled machine code binary of your guile
program.  I believe you can use the environment variable
GUILE_JIT_THRESHOLD set to zero to have your code compiled to
machine code on first use.

$ GUILE_JIT_THRESHOLD=0 guile my-program.scm

You can also start execution of your guile program from C using
the techniques described in the section "Programming in C" in the
Guile Reference Manual.

Matt

How best to produce a standalone executable with Guile?

[Christopher Howard]

Are you a Guix user? One you come up with a Guix package definition, it
is I think a one line command to generate an exportable executable with
a full dependency chain included, or a docker image.

-- 
Christopher Howard
p: +1 (907) 374-0257
w: https://librehacker.com
social: https://gnusocial.club/librehacker
gpg: ADDEAADE5D607C8D (keys.gnupg.net)

On Sun, 2020-04-05 at 12:00 -0400, guile-user-request@gnu.org wrote:
> Send guile-user mailing list submissions to
> 	guile-user@gnu.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://lists.gnu.org/mailman/listinfo/guile-user
> or, via email, send a message with subject or body 'help' to
> 	guile-user-request@gnu.org
> 
> You can reach the person managing the list at
> 	guile-user-owner@gnu.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of guile-user digest..."
> 
> 
> Today's Topics:
> 
>    1. How best to produce a standalone executable with Guile?
>       (James Cooper)
>    2. Re: How best to produce a standalone executable with Guile?
>       (Matt Wette)
> 
> 
> -------------------------------------------------------------------
> ---
> 
> Message: 1
> Date: Sun, 5 Apr 2020 05:28:10 +0000
> From: James Cooper <jcoo092@aucklanduni.ac.nz>
> To: guile-user@gnu.org
> Subject: How best to produce a standalone executable with Guile?
> Message-ID:
> 	<
> CAB_RW2E5s7Wd4yRA81OcaGLtGoFsvjOR1hCGhD=W7EC4FHr8HA@mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
> 
> Hi Guile folks,
> 
> I am currently experimenting with a handful of languages that include
> modern implementations of Concurrent ML, to determine which one
> appears
> best able to handle computationally- and message-heavy workloads (at
> least
> for my purposes, Computer Vision).  Guile is one of the languages I
> am
> trying out with the Fibers library.
> 
> My question is, how can I best go about creating Linux executables
> from
> Guile, where said programs benefit from the most ahead-of-time
> compilation
> and optimisation possible?  I don't believe they strictly need to be
> standalone, as the environment I will be running the programs in will
> likely be the same as the development environment. So they can
> undoubtedly
> have dependencies on external libraries.  Furthermore, I am perfectly
> happy
> with the process being a bit convoluted, as I can easily automate it
> with a
> makefile.  I would, however, prefer if at all possible that the
> option to
> target other processor architectures remains available down the line
> (I
> assume using GCC can enable this, but I'm not sure).
> 
> I have looked at all the sections of the Guile manual that appeared
> relevant, but to be honest I'm still a bit confused about what the
> best
> approach would be.  *s* *9.4.5 Bytecode* in the manual seems to
> suggest
> that one can use guild to produce what will be a standalone
> executable that
> looks like an ELF binary to the system and runs on the Guile VM, but
> I'm
> entirely sure if I am reading it correctly.  *s 6.18.5 Compiling
> Scheme
> Code* also discusses compilation but seems to denounce the practice
> explicitly.  *s 9.3.8 Just-In-Time Native Code* talks about JIT
> compilation
> to native code for a given target.  Is it possible to apply this
> process
> ahead-of-time?
> 
> A potential alternative approach I can see is using the capabilities
> of
> running Guile code in a C program (compiled with GCC).  This would
> presumably involve creating a template C program that just calls the
> appropriate Guile functions and passes the relevant command line
> parameters
> in.
> 
> Any thoughts on what would likely produce the most efficient
> executable in
> the end?  The target programs do not need to modify themselves at
> run-time
> (or really do anything dynamic) if that makes a difference.  And does
> anyone know of pre-existing examples they could point me to?  Please
> do let
> me know if there is further information I could provide that would
> help.
> Thanks in advance.
> 
> Regards,
> 
> James
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Sun, 5 Apr 2020 07:44:18 -0700
> From: Matt Wette <matt.wette@gmail.com>
> To: guile-user@gnu.org
> Subject: Re: How best to produce a standalone executable with Guile?
> Message-ID: <a0853855-b588-ab59-0ad6-6fd5c9728d2a@gmail.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
> 
> On 4/4/20 10:28 PM, James Cooper wrote:
> > Hi Guile folks,
> > 
> > I am currently experimenting with a handful of languages that
> > include
> > modern implementations of Concurrent ML, to determine which one
> > appears
> > best able to handle computationally- and message-heavy workloads
> > (at least
> > for my purposes, Computer Vision).  Guile is one of the languages I
> > am
> > trying out with the Fibers library.
> > 
> > My question is, how can I best go about creating Linux executables
> > from
> > Guile, where said programs benefit from the most ahead-of-time
> > compilation
> > and optimisation possible?  I don't believe they strictly need to
> > be
> > standalone, as the environment I will be running the programs in
> > will
> > likely be the same as the development environment. So they can
> > undoubtedly
> > have dependencies on external libraries.  Furthermore, I am
> > perfectly happy
> > with the process being a bit convoluted, as I can easily automate
> > it with a
> > makefile.  I would, however, prefer if at all possible that the
> > option to
> > target other processor architectures remains available down the
> > line (I
> > assume using GCC can enable this, but I'm not sure).
> > 
> > [...]
> You can't produce a pre-compiled machine code binary of your guile
> program.  I believe you can use the environment variable
> GUILE_JIT_THRESHOLD set to zero to have your code compiled to
> machine code on first use.
> 
> $ GUILE_JIT_THRESHOLD=0 guile my-program.scm
> 
> You can also start execution of your guile program from C using
> the techniques described in the section "Programming in C" in the
> Guile Reference Manual.
> 
> Matt
> 
> 
> 
> 
> ------------------------------
> 
> Subject: Digest Footer
> 
> _______________________________________________
> guile-user mailing list
> guile-user@gnu.org
> https://lists.gnu.org/mailman/listinfo/guile-user
> 
> 
> ------------------------------
> 
> End of guile-user Digest, Vol 209, Issue 4
> ******************************************

Re: How best to produce a standalone executable with Guile?

[James Cooper]

Hi Matt and Christopher,

Thank you for your responses.  Apologies for replying to my email - for
some reason I never received either of yours into my inbox, I only saw them
when I looked up my email on the mailing list archive online.

I'm afraid I am not (currently) a Guix user.  I did try to install it on my
working system but had no end of issues, which led me to leave it for the
time being and instead simply build Guile from source.  That's a story for
another time, however, and it certainly sounds like something I should look
into.

Am I correct in understanding that guild essentially is what the Guile
runtime will use to compile my source files before executing them?  And
that the way to create a traditional executable would be to create a C
program that includes libguile.h and wraps my actual Guile program
(probably using scm_boot_guile)?  I'll try out this approach as soon as I
can.  If you're wondering, the reason why I'm interested in having a
traditional executable is that I'm planning to run comparative benchmarking
directly on equivalent versions of the same program in different languages,
and I *think *that will work a lot better if I can specify an executable
name for the Guile implementation.

Thanks again.

Regards,

James

On Sun, 5 Apr 2020 at 05:28, James Cooper <jcoo092@aucklanduni.ac.nz> wrote:

> Hi Guile folks,
>
> I am currently experimenting with a handful of languages that include
> modern implementations of Concurrent ML, to determine which one appears
> best able to handle computationally- and message-heavy workloads (at least
> for my purposes, Computer Vision).  Guile is one of the languages I am
> trying out with the Fibers library.
>
> My question is, how can I best go about creating Linux executables from
> Guile, where said programs benefit from the most ahead-of-time compilation
> and optimisation possible?  I don't believe they strictly need to be
> standalone, as the environment I will be running the programs in will
> likely be the same as the development environment. So they can undoubtedly
> have dependencies on external libraries.  Furthermore, I am perfectly happy
> with the process being a bit convoluted, as I can easily automate it with a
> makefile.  I would, however, prefer if at all possible that the option to
> target other processor architectures remains available down the line (I
> assume using GCC can enable this, but I'm not sure).
>
> I have looked at all the sections of the Guile manual that appeared
> relevant, but to be honest I'm still a bit confused about what the best
> approach would be.  *s* *9.4.5 Bytecode* in the manual seems to suggest
> that one can use guild to produce what will be a standalone executable
> that looks like an ELF binary to the system and runs on the Guile VM, but
> I'm entirely sure if I am reading it correctly.  *s 6.18.5 Compiling
> Scheme Code* also discusses compilation but seems to denounce the
> practice explicitly.  *s 9.3.8 Just-In-Time Native Code* talks about JIT
> compilation to native code for a given target.  Is it possible to apply
> this process ahead-of-time?
>
> A potential alternative approach I can see is using the capabilities of
> running Guile code in a C program (compiled with GCC).  This would
> presumably involve creating a template C program that just calls the
> appropriate Guile functions and passes the relevant command line parameters
> in.
>
> Any thoughts on what would likely produce the most efficient executable in
> the end?  The target programs do not need to modify themselves at run-time
> (or really do anything dynamic) if that makes a difference.  And does
> anyone know of pre-existing examples they could point me to?  Please do let
> me know if there is further information I could provide that would help.
> Thanks in advance.
>
> Regards,
>
> James
>

Re: How best to produce a standalone executable with Guile?

[Arne Babenhauserheide]

Hi James,

James Cooper <jcoo092@aucklanduni.ac.nz> writes:
> Am I correct in understanding that guild essentially is what the Guile
> runtime will use to compile my source files before executing them?

Not quite. Guile does create bytecode, and guild can create it, but this
also happens during normal execution.

For some tasty details see https://wingolog.org/archives/2014/01/19/elf-in-guile

> traditional executable is that I'm planning to run comparative benchmarking
> directly on equivalent versions of the same program in different languages,
> and I *think *that will work a lot better if I can specify an executable

This sounds pretty interesting. Before you run this, though, please get
the help of experienced Guile programmers. I have an example of a
string-handling procedure which got more than 160x faster thanks to help
from others here. No typo there: It really got more than
hundred-and-sixty-times faster.

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken