[PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Leo Famulari]

* gnu/packages/lua.scm (lua-5.2)[arguments]: Rewrite make-flags so that
  Lua is built with platform-specific instructions for shared library
  loading (dlopen).
---
 gnu/packages/lua.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index 6bedde3..bbb1b8c 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2014 Raimon Grau <raimonster@gmail.com>
 ;;; Copyright © 2014 Mark H Weaver <mhw@netris.org>
 ;;; Copyright © 2014 Andreas Enge <andreas@enge.fr>
+;;; Copyright © 2015 Leo Famulari <leo@famulari.name>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -47,7 +48,10 @@
        #:test-target "test"
        #:phases (alist-replace
                  'build
-                 (lambda _ (zero? (system* "make" "CFLAGS=-fPIC" "linux")))
+                 (lambda _ (zero? (system* "make"
+                                           "PLAT=linux"
+                                           "MYCFLAGS=-fPIC"
+                                           "MYLDFLAGS=-fPIC")))
                  (alist-replace
                   'install
                   (lambda* (#:key outputs #:allow-other-keys)
-- 
2.6.1

Re: [PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> * gnu/packages/lua.scm (lua-5.2)[arguments]: Rewrite make-flags so that
>   Lua is built with platform-specific instructions for shared library
>   loading (dlopen).

Please move the explanations as a comment in the code.

>         #:phases (alist-replace
>                   'build
> -                 (lambda _ (zero? (system* "make" "CFLAGS=-fPIC" "linux")))
> +                 (lambda _ (zero? (system* "make"
> +                                           "PLAT=linux"
> +                                           "MYCFLAGS=-fPIC"
> +                                           "MYLDFLAGS=-fPIC")))

How can I test whether this works as expected?  I tried this, but I’m
unsure whether it’s supposed to load OpenSSL’s libssl.so directly or
not:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix environment --ad-hoc lua-5.2 openssl 
[env]$ echo $PATH
/gnu/store/zmqhwsl9vvxr4ihdnhwwpc3dpgmpsgsy-openssl-1.0.2d/bin:/gnu/store/9wmsrx7s9ydln5vgddms3cfavaiigwq2-lua-5.2.3/bin:/home/ludo/src/guix/scripts:/home/ludo/src/guix:/home/ludo/soft/bin:/home/ludo/.opam/system/bin:/home/ludo/.guix-profile/bin:/home/ludo/.guix-profile/sbin:/run/setuid-programs:/run/current-system/profile/bin:/run/current-system/profile/sbin
[env]$ export LD_LIBRARY_PATH=/gnu/store/zmqhwsl9vvxr4ihdnhwwpc3dpgmpsgsy-openssl-1.0.2d/lib
[env]$ lua -lssl
lua: module 'ssl' not found:
	no field package.preload['ssl']
	no file '/usr/local/share/lua/5.2/ssl.lua'
	no file '/usr/local/share/lua/5.2/ssl/init.lua'
	no file '/usr/local/lib/lua/5.2/ssl.lua'
	no file '/usr/local/lib/lua/5.2/ssl/init.lua'
	no file './ssl.lua'
	no file '/usr/local/lib/lua/5.2/ssl.so'
	no file '/usr/local/lib/lua/5.2/loadall.so'
	no file './ssl.so'
stack traceback:
	[C]: in function 'require'
	[C]: in ?
--8<---------------cut here---------------end--------------->8---

Apologies in advance if I’m just asking stupid questions!

Besides, it would be nice to get rid of these hard-coded /usr/local in a
subsequent patch.

Ludo’.

Re: [PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Leo Famulari]

Forgot to send this to the list...

On Sun, Nov 8, 2015, at 16:08, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > * gnu/packages/lua.scm (lua-5.2)[arguments]: Rewrite make-flags so that
> >   Lua is built with platform-specific instructions for shared library
> >   loading (dlopen).
> 
> Please move the explanations as a comment in the code.

Okay.
 
> >         #:phases (alist-replace
> >                   'build
> > -                 (lambda _ (zero? (system* "make" "CFLAGS=-fPIC" "linux")))
> > +                 (lambda _ (zero? (system* "make"
> > +                                           "PLAT=linux"
> > +                                           "MYCFLAGS=-fPIC"
> > +                                           "MYLDFLAGS=-fPIC")))
> 
> How can I test whether this works as expected?  I tried this, but I’m
> unsure whether it’s supposed to load OpenSSL’s libssl.so directly or
> not:

There is a Lua library named luasec that provides SSL. Luasec only
supports lua-5.1 except in an alpha release (luasec-0.6). [1]

I am able to load luasec by telling Lua where to look. I have lua-5.1
and luasec in my profile:

$ LUA_PATH=~/.guix-profile/share/lua/5.1/\?.lua
LUA_CPATH=~/.guix-profile/lib/lua/5.1/\?.so lua -lssl
Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
> 
^ That is the Lua prompt, indicating that the Lua interpreter has
successfully imported the SSL libraries. The '?' character is replaced
by the component that Lua is currently searching for. So, if you invoke
`lua -ssl`,  the "?.so" in LUA_CPATH is replaced by ssl.so.

> Apologies in advance if I’m just asking stupid questions!

Not at all, everything I know about Lua I have learned trying to get
Prosody packaged for Guix. I am about 1 week ahead of you in terms of
Lua knowledge ;)

> Besides, it would be nice to get rid of these hard-coded /usr/local in a
> subsequent patch.

This is related to my next step and I need some advice.

We need to set LUA_PATH and LUA_CPATH properly in order for programs to
load external Lua libraries. I have been installing Lua libraries into
${out}/lib/lua/${LUA_VERSION} and ${out}/share/lua/${LUA_VERSION}, so
that they end up at, for example, ~/.guix-profile/lib/lua/5.1/ssl.so [2]

Nixpkgs makes it all work by setting the paths in wrappers around
programs that need to find Lua libraries. I don't fully understand how
they generate the paths but I get the idea. Some examples:
https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/xmpp/prosody/default.nix
https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/video/quvi/tool.nix

What do you think?

Leo 

[1] I have some of these packages defined in a WIP branch:
https://github.com/lfam/guix/commits/contrib-lua

[2]
bourne shell variable syntax (don't have paredit in my email client)

Re: [PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Sun, Nov 8, 2015, at 16:08, Ludovic Courtès wrote:
>> Leo Famulari <leo@famulari.name> skribis:

[...]

>> How can I test whether this works as expected?  I tried this, but I’m
>> unsure whether it’s supposed to load OpenSSL’s libssl.so directly or
>> not:
>
> There is a Lua library named luasec that provides SSL. Luasec only
> supports lua-5.1 except in an alpha release (luasec-0.6). [1]

Okay.

> I am able to load luasec by telling Lua where to look. I have lua-5.1
> and luasec in my profile:
>
> $ LUA_PATH=~/.guix-profile/share/lua/5.1/\?.lua
> LUA_CPATH=~/.guix-profile/lib/lua/5.1/\?.so lua -lssl
> Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
>> 
> ^ That is the Lua prompt, indicating that the Lua interpreter has
> successfully imported the SSL libraries. The '?' character is replaced
> by the component that Lua is currently searching for. So, if you invoke
> `lua -ssl`,  the "?.so" in LUA_CPATH is replaced by ssl.so.

OK.  So in LUA_PATH you need to list the actual .lua files (not just
their dirname), and in LUA_CPATH the .so files (not just their dirname),
right?

>> Besides, it would be nice to get rid of these hard-coded /usr/local in a
>> subsequent patch.
>
> This is related to my next step and I need some advice.
>
> We need to set LUA_PATH and LUA_CPATH properly in order for programs to
> load external Lua libraries. I have been installing Lua libraries into
> ${out}/lib/lua/${LUA_VERSION} and ${out}/share/lua/${LUA_VERSION}, so
> that they end up at, for example, ~/.guix-profile/lib/lua/5.1/ssl.so [2]

It seems we need a ‘native-search-paths’ thingie here, along these
lines:

  (search-path-specification
    (variable "LUA_PATH")
    (files '("share/lua/5.1"))
    (file-pattern "\\.lua$")
    (file-type 'regular))

See the ‘libxml2’ package for an example of how this works.

Something similar is needed for ‘LUA_CPATH’.  And then we need to make
sure to replace “5.1” with the actual version number.

That way, when Lua and luasec are in the same profile, ‘guix package
--search-paths’ will suggest the right environment variable settings
(and likewise within build environments.)

> Nixpkgs makes it all work by setting the paths in wrappers around
> programs that need to find Lua libraries. I don't fully understand how
> they generate the paths but I get the idea. Some examples:
> https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/xmpp/prosody/default.nix
> https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/video/quvi/tool.nix
>
> What do you think?

That sounds like the right thing.  In Guix this is achieved with
‘wrap-program’; there are several examples of packages doing that.

HTH!

Ludo’.

Re: [PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Leo Famulari]

On Sun, Nov 08, 2015 at 10:08:23PM +0100, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > * gnu/packages/lua.scm (lua-5.2)[arguments]: Rewrite make-flags so that
> >   Lua is built with platform-specific instructions for shared library
> >   loading (dlopen).
> 
> Please move the explanations as a comment in the code.

Done. Updated patches forthcoming...

> 
> >         #:phases (alist-replace
> >                   'build
> > -                 (lambda _ (zero? (system* "make" "CFLAGS=-fPIC" "linux")))
> > +                 (lambda _ (zero? (system* "make"
> > +                                           "PLAT=linux"
> > +                                           "MYCFLAGS=-fPIC"
> > +                                           "MYLDFLAGS=-fPIC")))
> 
> How can I test whether this works as expected?  I tried this, but I’m
> unsure whether it’s supposed to load OpenSSL’s libssl.so directly or
> not:
> 
> --8<---------------cut here---------------start------------->8---
> $ ./pre-inst-env guix environment --ad-hoc lua-5.2 openssl 
> [env]$ echo $PATH
> /gnu/store/zmqhwsl9vvxr4ihdnhwwpc3dpgmpsgsy-openssl-1.0.2d/bin:/gnu/store/9wmsrx7s9ydln5vgddms3cfavaiigwq2-lua-5.2.3/bin:/home/ludo/src/guix/scripts:/home/ludo/src/guix:/home/ludo/soft/bin:/home/ludo/.opam/system/bin:/home/ludo/.guix-profile/bin:/home/ludo/.guix-profile/sbin:/run/setuid-programs:/run/current-system/profile/bin:/run/current-system/profile/sbin
> [env]$ export LD_LIBRARY_PATH=/gnu/store/zmqhwsl9vvxr4ihdnhwwpc3dpgmpsgsy-openssl-1.0.2d/lib
> [env]$ lua -lssl
> lua: module 'ssl' not found:
> 	no field package.preload['ssl']
> 	no file '/usr/local/share/lua/5.2/ssl.lua'
> 	no file '/usr/local/share/lua/5.2/ssl/init.lua'
> 	no file '/usr/local/lib/lua/5.2/ssl.lua'
> 	no file '/usr/local/lib/lua/5.2/ssl/init.lua'
> 	no file './ssl.lua'
> 	no file '/usr/local/lib/lua/5.2/ssl.so'
> 	no file '/usr/local/lib/lua/5.2/loadall.so'
> 	no file './ssl.so'
> stack traceback:
> 	[C]: in function 'require'
> 	[C]: in ?
> --8<---------------cut here---------------end--------------->8---
> 
> Apologies in advance if I’m just asking stupid questions!
> 
> Besides, it would be nice to get rid of these hard-coded /usr/local in a
> subsequent patch.
> 
> Ludo’.