The problem of packaging Minetest mods/games

The problem of packaging Minetest mods/games

[Jan]

Hello,

Recently I decided to update the Minetest package, which isn't a problem
itself, but I discovered the package fails to provide the default
minetest game. For those who don't know, Minetest is extensible by
desing - all you do is you put a modification/game into a folder -
usually /share/minetest/games or ~/.minetest/games and it is run by the
game engine.
I checked the source code and there's a non-public minetest-data
package, which provides the minetest_game and it contains this:

(arguments
     `(#:modules ((guix build utils))
       #:builder (begin
                   (use-modules (guix build utils))
                   (let ((install-dir (string-append
                                       %output
                                       "/share/minetest/games/minetest_game")))
                     (mkdir-p install-dir)
                     (copy-recursively
                       (assoc-ref %build-inputs "source")
                       install-dir)
                     #t))))

And this package is in the propagated-inputs fiend of the minetest
package, but it doesn't work.

I would like to understand why it doesn't work, fix it and learn
something new about Guix by the way :)

The second problem I encountered during examining the package was every
time I changed the path of the minetest_game in the minetest-data
package minetest was also recompiled, which took long time. (I thought
the path is improper)
My question is, what is the Guix way of dealing with such packages?
Imagine we have like 100 packaged minetest mods/games. Say I wanted to
add one, would I have to recompile the whole minetest package (C++),
despite the fact all mods are just Lua scripts put into folder and
interpreted by the engine?

Could we for example place the mods in the ~/.minetest/games folder?


Jan Wielkiewicz

Re: The problem of packaging Minetest mods/games

[Julien Lepiller]

Le 19 mai 2020 17:05:49 GMT-04:00, Jan <tona_kosmicznego_smiecia@interia.pl> a écrit :
>Hello,
>
>Recently I decided to update the Minetest package, which isn't a
>problem
>itself, but I discovered the package fails to provide the default
>minetest game. For those who don't know, Minetest is extensible by
>desing - all you do is you put a modification/game into a folder -
>usually /share/minetest/games or ~/.minetest/games and it is run by the
>game engine.
>I checked the source code and there's a non-public minetest-data
>package, which provides the minetest_game and it contains this:
>
>(arguments
>     `(#:modules ((guix build utils))
>       #:builder (begin
>                   (use-modules (guix build utils))
>                   (let ((install-dir (string-append
>                                       %output
>                               "/share/minetest/games/minetest_game")))
>                     (mkdir-p install-dir)
>                     (copy-recursively
>                       (assoc-ref %build-inputs "source")
>                       install-dir)
>                     #t))))
>
>And this package is in the propagated-inputs fiend of the minetest
>package, but it doesn't work.
>
>I would like to understand why it doesn't work, fix it and learn
>something new about Guix by the way :)
>
>The second problem I encountered during examining the package was every
>time I changed the path of the minetest_game in the minetest-data
>package minetest was also recompiled, which took long time. (I thought
>the path is improper)
>My question is, what is the Guix way of dealing with such packages?
>Imagine we have like 100 packaged minetest mods/games. Say I wanted to
>add one, would I have to recompile the whole minetest package (C++),
>despite the fact all mods are just Lua scripts put into folder and
>interpreted by the engine?
>
>Could we for example place the mods in the ~/.minetest/games folder?
>
>
>Jan Wielkiewicz

The minetest package declares MINETEST_SUBGAME_PATH. This means that minetest reads (or used to read) that environment variable to find its games. Can you check it is set in your system? Guix should have told you about it (and relogin should set it automatically). There might be a similar variable for mods we need to declare.

Minetest gets rebuilt because one of its inputs is different. That's the way guix works, because we cannot be sure the package will be the same with a different input. If we didn't, we would not guarantee reproducibility.

I think simply installing mods/games in share/minetest/games should work. Installing the additional package in your profile will make them available to the game.

Setting MINETEST_SUBGAME_PATH to a local directory in your home might work as well.

HTH!

The problem of packaging Minetest mods/games

[Leo Prikler]

Hello Jan,

> And this package is in the propagated-inputs fiend of the minetest
> package, but it doesn't work.
> 
> I would like to understand why it doesn't work, fix it and learn
> something new about Guix by the way :)
My guess is, that minetest searches in 
  /gnu/store/<hash>-minetest/share/...
when the actual path is
  /gnu/store/<other-hash>-minetest-data/share/...
If you just want to get the base game to work, it should probably be
okay to add a symlink instead of propagating the input, but this
obviously won't work for third party extensions.

> My question is, what is the Guix way of dealing with such packages?
> Imagine we have like 100 packaged minetest mods/games. Say I wanted
> to
> add one, would I have to recompile the whole minetest package (C++),
> despite the fact all mods are just Lua scripts put into folder and
> interpreted by the engine?
The canonical Guix way is to look for environment variables to set
(usually called STUFF_WERE_LOOKING_FOR_PATH) and add a search path
specification for that in the original package.  Take gstreamer as an
example, for which we define GST_PLUGIN_SYSTEM_PATH.

If the game does not yet support such variables (it appears, Minetest
does indeed have MINETEST_SUBGAME_PATH, but no MINETEST_MOD_PATH in
case you need the latter), consider patching the game itself so that it
does.  If you're lucky, you can even convince people to accept your
patch upstream.  If not or if it's really Guix-specific (as in the case
of GUIX_GTK3_PATH for example), we'll have to carry around that patch
in the Guix tree and keep it updated.

> Could we for example place the mods in the ~/.minetest/games folder?
That's not very functional of you.  In theory, you can put stuff there,
but not using Guix.  As an example, Stepmania reads all its
configuration, data and cache from ~/.stepmania-<version>, so that's of
course where I put the data – data, that is already unfit to be managed
by Guix due to licensing issues, mind you.  In the case of Minetest
mods/games, that folder quickly gets stale however, so I'd not suggest
doing so unless you really need to.

Regards,
Leo

PS: minetest-data sounds like a good candidate for copy-build-system.
PPS: Ignore the older (non-public) mail I sent you mentioning
MINETEST_GAME_PATH instead of MINETEST_SUBGAME_PATH.  At that point, I
had not yet encountered that variable in my research.

Re: The problem of packaging Minetest mods/games

[Ricardo Wurmus]

Jan <tona_kosmicznego_smiecia@interia.pl> writes:

> The second problem I encountered during examining the package was every
> time I changed the path of the minetest_game in the minetest-data
> package minetest was also recompiled, which took long time. (I thought
> the path is improper)
> My question is, what is the Guix way of dealing with such packages?
> Imagine we have like 100 packaged minetest mods/games. Say I wanted to
> add one, would I have to recompile the whole minetest package (C++),
> despite the fact all mods are just Lua scripts put into folder and
> interpreted by the engine?

Is there a search path?

> Could we for example place the mods in the ~/.minetest/games folder?

If minetest looks in ~/.minetest/games then it probably can also be made
to look in a list of directories specified via an environment variable.
That would be the Guixy way.  Let minetest specify a search path and
install the data files and plugins in a well-known directory suffix.

-- 
Ricardo

Re: The problem of packaging Minetest mods/games

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 923 bytes --]

On Wed, May 20, 2020 at 00:36:44 +0200, Leo Prikler wrote:
>> Could we for example place the mods in the ~/.minetest/games folder?
> That's not very functional of you.  In theory, you can put stuff there,
> but not using Guix.  As an example, Stepmania reads all its
> configuration, data and cache from ~/.stepmania-<version>, so that's of
> course where I put the data – data, that is already unfit to be managed
> by Guix due to licensing issues, mind you.  In the case of Minetest
> mods/games, that folder quickly gets stale however, so I'd not suggest
> doing so unless you really need to.

I still think it ought to search ~/.minetest/games, though, for those
things that may not be installed using guix.  For example, minetest has
a built-in means of downloading games/mods.  While it's best to use a
package manager, it also breaks functionality if it doesn't work both
ways.

-- 
Mike Gerwitz

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: The problem of packaging Minetest mods/games

[Jan]

Hi,

Thanks everyone for suggestions, I will check how it works in practice
and look for the environment variables.

Surprisingly I had an idea yesterday - what if we threat the minetest
package as an interpreter, which it actually is, because it interprets
lua scripts, while treating mods and games as files it
interprets. This way minetest doesn't require minetest-data as
a dependency anymore, on the other hand minetest-data requires minetest
to run, which is actually how it is in reality.

Following this idea, I removed the minetest-data dependency from the
minetest package and added minetest as a propagated input of
minetest-data. I installed the new minetest-data package and it worked!

This way minetest doesn't get rebuilt every time a modification/game is
added.

I could rename the packages:
minetest -> minetest-base/core (non-public package)
minetest-data -> minetest (public package)

This way installing minetest installs both the game's core and the
default game.

What do you think about this solution?



Jan Wielkiewicz