relocatable guile on windows

relocatable guile on windows

[Mike Gran]

Hello Guile,

The Lisp Game Jam was a few weeks ago, and in prep, I took a stab an
making Guile work better on Windows, using MinGW MSys64 UCRT64 for
compilation. Little did I imagine that it would take a couple months
to make something servicable, so I never got around to actually making
a game.

This Guile is a fork with a lot of hacks for Windows: lack of
POSIX shell, DLL nonsense, Windows filesystem instead of FHS,
yadda yadda yadda.  The biggest hack is that of making
all the file paths relative to the guile executable, so that
it can be distributed as a single zip file or MSIX install file.

There is an MSIX installer and a Zip file of a compiled Guile at

  https://github.com/spk121/guile/releases/tag/3.0.8.1

For reasons why you shouldn't use Microsoft Windows ever, see:
https://www.gnu.org/philosophy/upgrade-windows.html

There is a README-win.txt file that explains how you might use it to
distribute a game jam game on Windows.

  https://github.com/spk121/guile/blob/reloc-package/README-win.txt

-------

This is just a hack. I am not promising to maintain it, and I'm not
sure how much of this nonsense can or should be upstreamed.  But if
you do find a problem, I guess just email me directly so we don't clog
the mailing list with discussion of non-free software.

Also, my name shows up on the MSIX installer. I'm not taking credit,
it is just that the installer requires that the publisher match the
code signing certificate.

-------

CHANGES FROM CORE GUILE

This fork from core Guile has features necessary to distribute a Guile
app on Windows.  These are some of the changes.

Almost all instances of the long integer are replaced with intptr_t,
since on 64-bit Windows, sizeof(long) < sizeof(void *), in violation
of Guile's expectations.  Largely janneke did this work, but, I added
a couple more.

To help deliver a game as a single zip or tar file, a new configure
option --enable-relative-paths is added.  This installs all of guile
into the $(prefix)/app directory, and modifies the loading logic to
search for scheme, compiled scheme, and extension files relative to
the location of the executable.  The applications and all shared
object library files are in the root of that directory.  $(prefix)/app
is just a staging directory, and that directory can be renamed and
relocated.

Since the presence of a POSIX shell is not guaranteed, the guild
script may not be usable. A binary executable version of guild, called
guile-tools, is added.  Previously, guile-tools was an alias to guild
but now it is a compiled program.

Since unpacking zip or MSIX files may not preserve timestamps,
unpacked scheme files may have more recent file timestamps than their
associated compiled scheme (*.go) files, causing a recompilation.  A
feature is added such that if a file named FINAL is present in
ccache-dir or site-ccache-dir, the files therein are not recompiled
when they have file timestamps older than their associated scheme
files.

On Windows, the fallback directory is set to a more canonical Windows
app directory

  <HOME>\AppData\Local\guile

rather than

  <HOME>\AppData\Local\.cache\guile

On Windows, before a binary extension is loaded, the $libdir and
$extensiondir paths are added to the DLL search directories.  Also the
paths in the LTDL_LIBRARY_PATH and SCM_EXTENSIONS_PATH environment
variables are added to the DLL search directories.  This helps improve
the problem with a DLL failing to load because it depends on a DLL in
a directory that has not yet been added to the DLL search directories.

On Windows, there is a new function add-dll-search-directories.

On Windows, UTF-8 locales are used and tested.  When using the UCRT
library instead of the deprecated MSVCRT libary, guile on Windows has
some UTF-8 support.

read-line now handles the alternate line endings CRLF, CR, LS and PS.
To handle CRLF, read-line may return a string line terminator.  I
guess theoretically NEL is also a line terminator, but, I ignored
that.

The HTTP read-header-line is simplfied to take advantage of CRLF
support in read-line.

By default, cmd and powershell scripts require security elevation on
Windows.  As a workaround to avoid having to make a batch file to
launch a game, if guile is called without any command line arguments,
it now searches for a file named "cmdargs.txt" in the current working
directory.  When present, it will be used as command line arguments
for executing guile.  The purpose of this is to allow someone to just
double-click on the Guile executable to launch an app without
requiring a shell script.

In "cmdargs.txt", a line that begins with '#' in the first column is
treated as a comment.  If you need a command-line argument that begins
with a '#', begin the line with a space or tab.  The first non-comment
line of "command-args.txt" is stripped of initial and terminal
whitespace and is used as the first command-line argument (argv[1]).
The 2nd non-comment line is the 2nd command-line argument (argv[2]),
etc.  If the file does not end with newline, the last line is ignored.

Regards,
Mike Gran

Re: relocatable guile on windows

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]


Mike Gran <spk121@yahoo.com> writes:
> There is a README-win.txt file that explains how you might use it to
> distribute a game jam game on Windows.
>
>   https://github.com/spk121/guile/blob/reloc-package/README-win.txt

This may be just a hack, but it is absolutely awesome!

It finally solves the problem of getting Guile-tools to Windows-using
friends. (now I just have to learn how to follow that)

Thank you very much!

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: relocatable guile on windows

[Vivien Kraus]

Hello,

Le samedi 03 décembre 2022 à 18:48 -0800, Mike Gran a écrit :
> To help deliver a game as a single zip or tar file, a new configure
> option --enable-relative-paths is added.  This installs all of guile
> into the $(prefix)/app directory, and modifies the loading logic to
> search for scheme, compiled scheme, and extension files relative to
> the location of the executable.  The applications and all shared
> object library files are in the root of that directory. 
> $(prefix)/app
> is just a staging directory, and that directory can be renamed and
> relocated.

This is a common problem for all programs, and gnulib has a working
solution for that:
https://www.gnu.org/software/gnulib/manual/html_node/Supporting-Relocation.html

Also, this chapter describes a few more things gnulib can do for
windows:
https://www.gnu.org/software/gnulib/manual/html_node/Native-Windows-Support.html

I hope this can help make your work more maintainable.

Best regards,

Vivien

Re: relocatable guile on windows

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]


"Dr. Arne Babenhauserheide" <arne_bab@web.de> writes:

> Mike Gran <spk121@yahoo.com> writes:
>> There is a README-win.txt file that explains how you might use it to
>> distribute a game jam game on Windows.
>>
>>   https://github.com/spk121/guile/blob/reloc-package/README-win.txt
>
> This may be just a hack, but it is absolutely awesome!

> It finally solves the problem of getting Guile-tools to Windows-using
> friends. (now I just have to learn how to follow that)

And this sounds like it could make games actually feel like standalone
games:

> The purpose of this is to allow someone to just double-click on the
> Guile executable to launch an app without requiring a shell script.

https://github.com/spk121/guile/blob/reloc-package/README-win.txt

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: [EXT] relocatable guile on windows

[Thompson, David]

Hi Mike,

On Sat, Dec 3, 2022 at 9:49 PM Mike Gran <spk121@yahoo.com> wrote:
>
> Hello Guile,
>
> The Lisp Game Jam was a few weeks ago, and in prep, I took a stab an
> making Guile work better on Windows, using MinGW MSys64 UCRT64 for
> compilation. Little did I imagine that it would take a couple months
> to make something servicable, so I never got around to actually making
> a game.
>
> This Guile is a fork with a lot of hacks for Windows: lack of
> POSIX shell, DLL nonsense, Windows filesystem instead of FHS,
> yadda yadda yadda.  The biggest hack is that of making
> all the file paths relative to the guile executable, so that
> it can be distributed as a single zip file or MSIX install file.

I haven't had a chance to try this out yet but I wanted to say that
this is great work!  I'd really like to have a good story to tell for
shipping games built with Guile and Chickadee on Windows.  In the past
I've tried to cross-compile Guile for Windows from Linux and made it
as far as getting Guile 3 to boot, but the JIT didn't work and I think
threads, though enabled, were broken.  I haven't used Windows for
development in over 10 years so I'm somewhat helpless when things go
wrong.

It would be really great if we could organize a bit around upstreaming
patches for proper Windows support and come up with some documentation
with official recommendations for how to build for Windows.  I never
know what the best approach is: cygwin? mingw? native build (more work
than the other two)?  A lot of other Scheme implementations run on
Windows, so Guile should, too.

- Dave

Re: relocatable guile on windows

[Janneke Nieuwenhuizen]

Mike Gran writes:

Hi!

> For reasons why you shouldn't use Microsoft Windows ever, see:
> https://www.gnu.org/philosophy/upgrade-windows.html

Right.  No-one in their right mind should ever use Windows.  Sadly,
people even pay for this stuff.

Having said that, great work!

> Almost all instances of the long integer are replaced with intptr_t,
> since on 64-bit Windows, sizeof(long) < sizeof(void *), in violation
> of Guile's expectations.  Largely janneke did this work, but, I added
> a couple more.

Now that you found more, do you know if JIT works?  I couldn't get it to
work on wip-mingw, so I've been using --disable-jit.

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Re: relocatable guile on windows

[Mike Gran]

On Thu, Dec 08, 2022 at 06:56:44AM +0100, Janneke Nieuwenhuizen wrote:
> Mike Gran writes:
> 
> Hi!
> 
> > Almost all instances of the long integer are replaced with intptr_t,
> > since on 64-bit Windows, sizeof(long) < sizeof(void *), in violation
> > of Guile's expectations.  Largely janneke did this work, but, I added
> > a couple more.
> 
> Now that you found more, do you know if JIT works?  I couldn't get it to
> work on wip-mingw, so I've been using --disable-jit.

Sorry, I didn't even look into the lightening directory or try to
patch it.  I just assumed that JIT would be too compilicated
for me to understand.

A recompile with JIT enabled seems to fail almost immediately.

Regards,
Mike Gran

Re: [EXT] relocatable guile on windows

[Mike Gran]

On Wed, Dec 07, 2022 at 08:55:10AM -0500, Thompson, David wrote:
> Hi Mike,
> 
> On Sat, Dec 3, 2022 at 9:49 PM Mike Gran <spk121@yahoo.com> wrote:
> >
> It would be really great if we could organize a bit around upstreaming
> patches for proper Windows support and come up with some documentation
> with official recommendations for how to build for Windows.  I never
> know what the best approach is: cygwin? mingw? native build (more work
> than the other two)?  A lot of other Scheme implementations run on
> Windows, so Guile should, too.

Cygwin is basically and emulated POSIX layer using a C library
(newlib) that is fairly similar to GNU libc.  Guile builds and works
fine with Cygwin already.  The disadvange to Cygwin is that graphical
programs in cygwin require an X server, so they are a bit slow to run
and quite difficult to package up without requiring a downstream user
to get all of Cygwin.

MinGW is basically GCC plus a C library that is definitely not POSIX:
UCRT or MSVCRT, the native Windows libraries.  The result is a native
Windows app.  It comes with just enough of a shell to run autotools.

Native windows using Microsoft's C compiler is always tough because
autotools doesn't support it without extreme gymnastics.  It works
better for CMake programs. I haven't attempted it. It has some quirks
like the lack of #include_next that gnulib depends on.

The long integer patch is the most important and most disruptive if we
want 64-bit windows support.  We have to start there, and everything
descends from that. I'm not sure how to gather support for that.

Regards,
Mike Gran