[PATCH 0/5] Lua: enable shared library loading, build liblua.so

[PATCH 0/5] Lua: enable shared library loading, build liblua.so

[Leo Famulari]

These patches do 4 things:
1) Build lua-5.1 and lua-5.2 so that they can load shared libraries.
2) Make the build process for lua-5.1 and lua-5.2 emit liblua.so.
3) Rewrite the build arguments to lua-5.1 and lua-5.2 for clarity.
4) Replace some tabs with spaces in the luajit package definition.

Things that still need to be done for Lua:
* Make a pkg-config .pc file for lua-5.1, or figure out if lua-5.2's can
  be re-used.
* Figure out the best way to find shared Lua libraries at runtime (nix
  uses 'makeWrapper', which I think is like 'wrap-program'. See the
  nixpkg for Prosody).

Details about changes:
1) Build lua-5.1 and lua-5.2 so that they can load shared libraries.

This involved making sure that the compiler gets some platform specific
instructions on how to load shared libraries. This wasn't happening
before.

2) Make the build process for lua-5.1 and lua-5.2 emit a working
liblua.so.

This involved adding the patch lua52-liblua-so.patch and applying it
from the lua-5.2 package definition. Lua-5.2's make-flags also had to be
altered to pass "-fPIC" to the linker.

For lua-5.1, I set the execute bit of liblua.so (it seems a convention
that Guix makes shared libraries executable). Also, I moved the
instruction that sets "-fPIC" from lua51-liblua-so.patch into the
package definition's make-flags. The lua-5.1 Makefile is not as general
as lua-5.2's and so the make-flags are a little messier.

Also, for lua-5.1, I regenerated lua51-liblua-so.patch rather than
patching the patch. A patched patch seems too difficult for humans to
understand.

3) Rewrite the build arguments to lua-5.1 and lua-5.2 to use
modify-phases and #:make-flags, which I think is much clearer than the
'alist-' procedures.

For lua-5.2, this change is separated into its own commit (49aea9d). I
didn't do this for lua-5.1 (3056c20) because I needed to alter
lua-5.1-liblua-so.patch, and git wouldn't separate those "hunks"
properly. If you want me to spend more time trying to split that commit
up, I will try.

4) Replace some tabs with spaces.

Leo Famulari (5):
  gnu: Remove tabulation from luajit.
  gnu: Build lua-5.2 with dynamic library support.
  gnu: Build lua-5.2 with a dynamic library
  gnu: Use make-flags and modify-phases for lua-5.2.
  gnu: Build lua-5.1 with dynamic library support and a dynamic library.

 gnu/packages/lua.scm                       | 46 ++++++++++--------
 gnu/packages/patches/lua51-liblua-so.patch | 53 ++++++++++++--------
 gnu/packages/patches/lua52-liblua-so.patch | 78 ++++++++++++++++++++++++++++++
 3 files changed, 139 insertions(+), 38 deletions(-)
 create mode 100644 gnu/packages/patches/lua52-liblua-so.patch

-- 
2.6.1

[PATCH 1/5] gnu: Remove tabulation from luajit.

[Leo Famulari]

* gnu/packages/lua.scm (luajit): Remove tabs.
---
 gnu/packages/lua.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index 87f53d1..6bedde3 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -89,8 +89,8 @@ for configuration, scripting, and rapid prototyping.")
                                   version ".tar.gz"))
               (sha256
                (base32 "0ydxpqkmsn2c341j4r2v6r5r0ig3kbwv3i9jran3iv81s6r6rgjm"))
-	      (patches (list (search-patch "luajit-symlinks.patch")
-			     (search-patch "luajit-no_ldconfig.patch")))))
+              (patches (list (search-patch "luajit-symlinks.patch")
+                             (search-patch "luajit-no_ldconfig.patch")))))
     (build-system gnu-build-system)
     (arguments
      '(#:tests? #f                      ;luajit is distributed without tests
-- 
2.6.1

[PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Leo Famulari]

* gnu/packages/lua.scm (lua-5.2)[arguments]: Rewrite make-flags so that
  Lua is built with platform-specific instructions for shared library
  loading (dlopen).
---
 gnu/packages/lua.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index 6bedde3..bbb1b8c 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2014 Raimon Grau <raimonster@gmail.com>
 ;;; Copyright © 2014 Mark H Weaver <mhw@netris.org>
 ;;; Copyright © 2014 Andreas Enge <andreas@enge.fr>
+;;; Copyright © 2015 Leo Famulari <leo@famulari.name>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -47,7 +48,10 @@
        #:test-target "test"
        #:phases (alist-replace
                  'build
-                 (lambda _ (zero? (system* "make" "CFLAGS=-fPIC" "linux")))
+                 (lambda _ (zero? (system* "make"
+                                           "PLAT=linux"
+                                           "MYCFLAGS=-fPIC"
+                                           "MYLDFLAGS=-fPIC")))
                  (alist-replace
                   'install
                   (lambda* (#:key outputs #:allow-other-keys)
-- 
2.6.1

[PATCH 3/5] gnu: Build lua-5.2 with a dynamic library

[Leo Famulari]

* gnu/packages/patches/lua52-liblua-so.patch: New file.
* gnu/packages/lua.scm (lua-5.2): Use lua52-liblua-so.patch.
---
 gnu/packages/lua.scm                       |  3 +-
 gnu/packages/patches/lua52-liblua-so.patch | 78 ++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/lua52-liblua-so.patch

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index bbb1b8c..bb65070 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -38,7 +38,8 @@
                                  version ".tar.gz"))
              (sha256
               (base32 "0b8034v1s82n4dg5rzcn12067ha3nxaylp2vdp8gg08kjsbzphhk"))
-             (patches (list (search-patch "lua-pkgconfig.patch")))))
+             (patches (list (search-patch "lua-pkgconfig.patch")
+                            (search-patch "lua52-liblua-so.patch")))))
     (build-system gnu-build-system)
     (inputs `(("readline", readline)))
     (arguments
diff --git a/gnu/packages/patches/lua52-liblua-so.patch b/gnu/packages/patches/lua52-liblua-so.patch
new file mode 100644
index 0000000..2e6b39a
--- /dev/null
+++ b/gnu/packages/patches/lua52-liblua-so.patch
@@ -0,0 +1,78 @@
+From 7a1b4e0829124976878ca2765a8e546667a92ceb Mon Sep 17 00:00:00 2001
+From: Leo Famulari <leo@famulari.name>
+Date: Fri, 30 Oct 2015 19:11:31 -0400
+Subject: [PATCH] apply existing non-executable library patch and install
+ libraries as executable
+
+---
+ Makefile     | 6 +++---
+ src/Makefile | 8 +++++++-
+ 2 files changed, 10 insertions(+), 4 deletions(-)
+
+diff --git a/Makefile b/Makefile
+index 1aea8e6..bfcfed9 100644
+--- a/Makefile
++++ b/Makefile
+@@ -44,7 +44,7 @@ INTERPRETER= lua
+ COMPILER= luac
+ TO_BIN= $(INTERPRETER) $(COMPILER)
+ TO_INC= lua.h luaconf.h lualib.h lauxlib.h lua.hpp
+-TO_LIB= liblua.a
++TO_LIB= liblua.a liblua.so liblua.so.${V}
+ TO_PC=  lua-$(V).pc
+ TO_MAN= lua.1 luac.1
+ 
+@@ -56,7 +56,7 @@ R= $V.3
+ all:	$(PLAT)
+ 
+ $(PLATS):
+-	cd src && $(MAKE) $@
++	cd src && $(MAKE) $@ V=$(V) R=$(R)
+ 
+ clean:
+ 	cd src && $(MAKE) $@
+@@ -69,7 +69,7 @@ install: dummy $(TO_PC)
+ 	cd src && $(MKDIR) $(INSTALL_BIN) $(INSTALL_INC) $(INSTALL_LIB) $(INSTALL_PC) $(INSTALL_MAN) $(INSTALL_LMOD) $(INSTALL_CMOD)
+ 	cd src && $(INSTALL_EXEC) $(TO_BIN) $(INSTALL_BIN)
+ 	cd src && $(INSTALL_DATA) $(TO_INC) $(INSTALL_INC)
+-	cd src && $(INSTALL_DATA) $(TO_LIB) $(INSTALL_LIB)
++	cd src && $(INSTALL_EXEC) $(TO_LIB) $(INSTALL_LIB)
+ 	cd src && $(INSTALL_DATA) ../$(TO_PC) $(INSTALL_PC)
+ 	cd doc && $(INSTALL_DATA) $(TO_MAN) $(INSTALL_MAN)
+ 
+diff --git a/src/Makefile b/src/Makefile
+index 7b4b2b7..c86476e 100644
+--- a/src/Makefile
++++ b/src/Makefile
+@@ -29,6 +29,7 @@ MYOBJS=
+ PLATS= aix ansi bsd freebsd generic linux macosx mingw posix solaris
+ 
+ LUA_A=	liblua.a
++LUA_SO= liblua.so
+ CORE_O=	lapi.o lcode.o lctype.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o \
+ 	lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o \
+ 	ltm.o lundump.o lvm.o lzio.o
+@@ -43,7 +44,7 @@ LUAC_T=	luac
+ LUAC_O=	luac.o
+ 
+ ALL_O= $(BASE_O) $(LUA_O) $(LUAC_O)
+-ALL_T= $(LUA_A) $(LUA_T) $(LUAC_T)
++ALL_T= $(LUA_A) $(LUA_T) $(LUAC_T) $(LUA_SO)
+ ALL_A= $(LUA_A)
+ 
+ # Targets start here.
+@@ -55,6 +56,11 @@ o:	$(ALL_O)
+ 
+ a:	$(ALL_A)
+ 
++$(LUA_SO): $(CORE_O) $(LIB_O)
++	$(CC) -shared -ldl -Wl,-soname,$(LUA_SO).$(V) -o $@.$(R) $? -lm $(MYLDFLAGS)
++	ln -sf $(LUA_SO).$(R) $(LUA_SO).$(V)
++	ln -sf $(LUA_SO).$(R) $(LUA_SO)
++
+ $(LUA_A): $(BASE_O)
+ 	$(AR) $@ $(BASE_O)
+ 	$(RANLIB) $@
+-- 
+2.6.1
+
-- 
2.6.1

[PATCH 4/5] gnu: Use make-flags and modify-phases for lua-5.2.

[Leo Famulari]

* gnu/packages/lua.scm (lua-5.2)[arguments]: Use make-flags and
  modify-phases to control build process, replacing use of
  'alist-' procedures.
---
 gnu/packages/lua.scm | 27 +++++++++------------------
 1 file changed, 9 insertions(+), 18 deletions(-)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index bb65070..1f58751 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -43,25 +43,16 @@
     (build-system gnu-build-system)
     (inputs `(("readline", readline)))
     (arguments
-     '(#:modules ((guix build gnu-build-system)
-                    (guix build utils)
-                    (srfi srfi-1))
+     '(#:phases (modify-phases %standard-phases
+                  (delete 'configure))
        #:test-target "test"
-       #:phases (alist-replace
-                 'build
-                 (lambda _ (zero? (system* "make"
-                                           "PLAT=linux"
-                                           "MYCFLAGS=-fPIC"
-                                           "MYLDFLAGS=-fPIC")))
-                 (alist-replace
-                  'install
-                  (lambda* (#:key outputs #:allow-other-keys)
-                    (let ((out (assoc-ref outputs "out")))
-                      (zero? (system* "make" "install"
-                                      (string-append "INSTALL_TOP=" out)
-                                      (string-append "INSTALL_MAN=" out
-                                                     "/share/man/man1")))))
-                  (alist-delete 'configure %standard-phases)))))
+       #:make-flags (list "PLAT= linux"
+                          "MYCFLAGS= -fPIC"
+                          "MYLDFLAGS= -fPIC"
+                          (string-append "INSTALL_TOP= "
+                                         (assoc-ref %outputs "out"))
+                          (string-append "INSTALL_MAN= "
+                                         (assoc-ref %outputs "out")))))
     (home-page "http://www.lua.org/")
     (synopsis "Embeddable scripting language")
     (description
-- 
2.6.1

[PATCH 5/5] gnu: Build lua-5.1 with dynamic library support and a dynamic library.

[Leo Famulari]

* gnu/packages/lua.scm (lua-5.1)[arguments]: Rewrite make-flags so that
  Lua is built with platform-specific instructions for shared library
  loading (dlopen).
* gnu/packages/patches/lua51-liblua-so.patch: Install liblua.so with
  execute bit set. Move "-fPIC" flag from patch to package definition.
---
 gnu/packages/lua.scm                       | 14 +++++++-
 gnu/packages/patches/lua51-liblua-so.patch | 53 +++++++++++++++++++-----------
 2 files changed, 47 insertions(+), 20 deletions(-)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index 1f58751..3f45b33 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -73,7 +73,19 @@ for configuration, scripting, and rapid prototyping.")
                                  version ".tar.gz"))
              (sha256
               (base32 "0cskd4w0g6rdm2q8q3i4n1h3j8kylhs3rq8mxwl9vwlmlxbgqh16"))
-             (patches (list (search-patch "lua51-liblua-so.patch")))))))
+             (patches (list (search-patch "lua51-liblua-so.patch")))))
+    (arguments
+     '(#:phases (modify-phases %standard-phases
+                  (delete 'configure))
+       #:test-target "test"
+       #:make-flags (list "PLAT= linux"
+                          "CFLAGS= -O2 -Wall -fPIC $(MYCFLAGS)"
+                          "MYLDFLAGS= -fPIC"
+                          (string-append "INSTALL_TOP= "
+                                         (assoc-ref %outputs "out"))
+                          (string-append "INSTALL_MAN= "
+                                         (assoc-ref %outputs "out")
+                                         "/share/man/man1"))))))
 
 (define-public luajit
   (package
diff --git a/gnu/packages/patches/lua51-liblua-so.patch b/gnu/packages/patches/lua51-liblua-so.patch
index 6795f10..6b029de 100644
--- a/gnu/packages/patches/lua51-liblua-so.patch
+++ b/gnu/packages/patches/lua51-liblua-so.patch
@@ -1,13 +1,24 @@
+From 115e612016fe615e6c895af8df7db646114a9860 Mon Sep 17 00:00:00 2001
+From: Leo Famulari <leo@famulari.name>
+Date: Mon, 2 Nov 2015 03:29:47 -0500
+Subject: [PATCH] lua-5.1: Changes to Makefile patches
+
+Install liblua.so with execute bit.
+Don't set -fPIC from here. It will be set in the make flags.
 
 Patch the two Makefile to also create liblua.so
 Original patch by Allan McRae <allan@archlinux.org>
 for Archlinux
+---
+ Makefile     |  6 +++---
+ src/Makefile | 10 +++++++++-
+ 2 files changed, 12 insertions(+), 4 deletions(-)
 
-
-diff -ruN lua-5.1.5/Makefile lua-5.1.5-new/Makefile
---- lua-5.1.5/Makefile	2012-02-10 10:50:23.000000000 +0100
-+++ lua-5.1.5-new/Makefile	2014-09-10 20:17:28.913951433 +0200
-@@ -43,7 +43,7 @@
+diff --git a/Makefile b/Makefile
+index 209a132..653dbed 100644
+--- a/Makefile
++++ b/Makefile
+@@ -43,7 +43,7 @@ PLATS= aix ansi bsd freebsd generic linux macosx mingw posix solaris
  # What to install.
  TO_BIN= lua luac
  TO_INC= lua.h luaconf.h lualib.h lauxlib.h ../etc/lua.hpp
@@ -16,7 +27,7 @@ diff -ruN lua-5.1.5/Makefile lua-5.1.5-new/Makefile
  TO_MAN= lua.1 luac.1
  
  # Lua version and release.
-@@ -53,7 +53,7 @@
+@@ -53,7 +53,7 @@ R= 5.1.5
  all:	$(PLAT)
  
  $(PLATS) clean:
@@ -25,19 +36,20 @@ diff -ruN lua-5.1.5/Makefile lua-5.1.5-new/Makefile
  
  test:	dummy
  	src/lua test/hello.lua
-diff -ruN lua-5.1.5/src/Makefile lua-5.1.5-new/src/Makefile
---- lua-5.1.5/src/Makefile	2012-02-13 21:41:22.000000000 +0100
-+++ lua-5.1.5-new/src/Makefile	2014-09-10 20:16:09.982952152 +0200
-@@ -8,7 +8,7 @@
- PLAT= none
+@@ -62,7 +62,7 @@ install: dummy
+ 	cd src && $(MKDIR) $(INSTALL_BIN) $(INSTALL_INC) $(INSTALL_LIB) $(INSTALL_MAN) $(INSTALL_LMOD) $(INSTALL_CMOD)
+ 	cd src && $(INSTALL_EXEC) $(TO_BIN) $(INSTALL_BIN)
+ 	cd src && $(INSTALL_DATA) $(TO_INC) $(INSTALL_INC)
+-	cd src && $(INSTALL_DATA) $(TO_LIB) $(INSTALL_LIB)
++	cd src && $(INSTALL_EXEC) $(TO_LIB) $(INSTALL_LIB)
+ 	cd doc && $(INSTALL_DATA) $(TO_MAN) $(INSTALL_MAN)
  
- CC= gcc
--CFLAGS= -O2 -Wall $(MYCFLAGS)
-+CFLAGS= -O2 -Wall $(MYCFLAGS) -fPIC
- AR= ar rcu
- RANLIB= ranlib
- RM= rm -f
-@@ -34,9 +34,10 @@
+ ranlib:
+diff --git a/src/Makefile b/src/Makefile
+index e0d4c9f..ebc17e9 100644
+--- a/src/Makefile
++++ b/src/Makefile
+@@ -34,9 +34,10 @@ LUA_O=	lua.o
  
  LUAC_T=	luac
  LUAC_O=	luac.o print.o
@@ -49,7 +61,7 @@ diff -ruN lua-5.1.5/src/Makefile lua-5.1.5-new/src/Makefile
  ALL_A= $(LUA_A)
  
  default: $(PLAT)
-@@ -57,6 +58,13 @@
+@@ -57,6 +58,13 @@ $(LUA_T): $(LUA_O) $(LUA_A)
  $(LUAC_T): $(LUAC_O) $(LUA_A)
  	$(CC) -o $@ $(MYLDFLAGS) $(LUAC_O) $(LUA_A) $(LIBS)
  
@@ -63,3 +75,6 @@ diff -ruN lua-5.1.5/src/Makefile lua-5.1.5-new/src/Makefile
  clean:
  	$(RM) $(ALL_T) $(ALL_O)
  
+-- 
+2.6.1
+
-- 
2.6.1

Re: [PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> * gnu/packages/lua.scm (lua-5.2)[arguments]: Rewrite make-flags so that
>   Lua is built with platform-specific instructions for shared library
>   loading (dlopen).

Please move the explanations as a comment in the code.

>         #:phases (alist-replace
>                   'build
> -                 (lambda _ (zero? (system* "make" "CFLAGS=-fPIC" "linux")))
> +                 (lambda _ (zero? (system* "make"
> +                                           "PLAT=linux"
> +                                           "MYCFLAGS=-fPIC"
> +                                           "MYLDFLAGS=-fPIC")))

How can I test whether this works as expected?  I tried this, but I’m
unsure whether it’s supposed to load OpenSSL’s libssl.so directly or
not:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix environment --ad-hoc lua-5.2 openssl 
[env]$ echo $PATH
/gnu/store/zmqhwsl9vvxr4ihdnhwwpc3dpgmpsgsy-openssl-1.0.2d/bin:/gnu/store/9wmsrx7s9ydln5vgddms3cfavaiigwq2-lua-5.2.3/bin:/home/ludo/src/guix/scripts:/home/ludo/src/guix:/home/ludo/soft/bin:/home/ludo/.opam/system/bin:/home/ludo/.guix-profile/bin:/home/ludo/.guix-profile/sbin:/run/setuid-programs:/run/current-system/profile/bin:/run/current-system/profile/sbin
[env]$ export LD_LIBRARY_PATH=/gnu/store/zmqhwsl9vvxr4ihdnhwwpc3dpgmpsgsy-openssl-1.0.2d/lib
[env]$ lua -lssl
lua: module 'ssl' not found:
	no field package.preload['ssl']
	no file '/usr/local/share/lua/5.2/ssl.lua'
	no file '/usr/local/share/lua/5.2/ssl/init.lua'
	no file '/usr/local/lib/lua/5.2/ssl.lua'
	no file '/usr/local/lib/lua/5.2/ssl/init.lua'
	no file './ssl.lua'
	no file '/usr/local/lib/lua/5.2/ssl.so'
	no file '/usr/local/lib/lua/5.2/loadall.so'
	no file './ssl.so'
stack traceback:
	[C]: in function 'require'
	[C]: in ?
--8<---------------cut here---------------end--------------->8---

Apologies in advance if I’m just asking stupid questions!

Besides, it would be nice to get rid of these hard-coded /usr/local in a
subsequent patch.

Ludo’.

Re: [PATCH 3/5] gnu: Build lua-5.2 with a dynamic library

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> * gnu/packages/patches/lua52-liblua-so.patch: New file.
> * gnu/packages/lua.scm (lua-5.2): Use lua52-liblua-so.patch.

Applied, thanks.

This is not optimal though because:

  1. We’re still providing liblua.a, but that archives actually contains
     PIC code.

  2. The ‘lua’ and ‘luac’ executables are not linked against liblua.so.

If you have time, it would be nice to discuss this with upstream; it’s
not something we should fix ourselves IMO.

Ludo’.

Re: [PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Leo Famulari]

Forgot to send this to the list...

On Sun, Nov 8, 2015, at 16:08, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > * gnu/packages/lua.scm (lua-5.2)[arguments]: Rewrite make-flags so that
> >   Lua is built with platform-specific instructions for shared library
> >   loading (dlopen).
> 
> Please move the explanations as a comment in the code.

Okay.
 
> >         #:phases (alist-replace
> >                   'build
> > -                 (lambda _ (zero? (system* "make" "CFLAGS=-fPIC" "linux")))
> > +                 (lambda _ (zero? (system* "make"
> > +                                           "PLAT=linux"
> > +                                           "MYCFLAGS=-fPIC"
> > +                                           "MYLDFLAGS=-fPIC")))
> 
> How can I test whether this works as expected?  I tried this, but I’m
> unsure whether it’s supposed to load OpenSSL’s libssl.so directly or
> not:

There is a Lua library named luasec that provides SSL. Luasec only
supports lua-5.1 except in an alpha release (luasec-0.6). [1]

I am able to load luasec by telling Lua where to look. I have lua-5.1
and luasec in my profile:

$ LUA_PATH=~/.guix-profile/share/lua/5.1/\?.lua
LUA_CPATH=~/.guix-profile/lib/lua/5.1/\?.so lua -lssl
Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
> 
^ That is the Lua prompt, indicating that the Lua interpreter has
successfully imported the SSL libraries. The '?' character is replaced
by the component that Lua is currently searching for. So, if you invoke
`lua -ssl`,  the "?.so" in LUA_CPATH is replaced by ssl.so.

> Apologies in advance if I’m just asking stupid questions!

Not at all, everything I know about Lua I have learned trying to get
Prosody packaged for Guix. I am about 1 week ahead of you in terms of
Lua knowledge ;)

> Besides, it would be nice to get rid of these hard-coded /usr/local in a
> subsequent patch.

This is related to my next step and I need some advice.

We need to set LUA_PATH and LUA_CPATH properly in order for programs to
load external Lua libraries. I have been installing Lua libraries into
${out}/lib/lua/${LUA_VERSION} and ${out}/share/lua/${LUA_VERSION}, so
that they end up at, for example, ~/.guix-profile/lib/lua/5.1/ssl.so [2]

Nixpkgs makes it all work by setting the paths in wrappers around
programs that need to find Lua libraries. I don't fully understand how
they generate the paths but I get the idea. Some examples:
https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/xmpp/prosody/default.nix
https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/video/quvi/tool.nix

What do you think?

Leo 

[1] I have some of these packages defined in a WIP branch:
https://github.com/lfam/guix/commits/contrib-lua

[2]
bourne shell variable syntax (don't have paredit in my email client)

Re: [PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Sun, Nov 8, 2015, at 16:08, Ludovic Courtès wrote:
>> Leo Famulari <leo@famulari.name> skribis:

[...]

>> How can I test whether this works as expected?  I tried this, but I’m
>> unsure whether it’s supposed to load OpenSSL’s libssl.so directly or
>> not:
>
> There is a Lua library named luasec that provides SSL. Luasec only
> supports lua-5.1 except in an alpha release (luasec-0.6). [1]

Okay.

> I am able to load luasec by telling Lua where to look. I have lua-5.1
> and luasec in my profile:
>
> $ LUA_PATH=~/.guix-profile/share/lua/5.1/\?.lua
> LUA_CPATH=~/.guix-profile/lib/lua/5.1/\?.so lua -lssl
> Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
>> 
> ^ That is the Lua prompt, indicating that the Lua interpreter has
> successfully imported the SSL libraries. The '?' character is replaced
> by the component that Lua is currently searching for. So, if you invoke
> `lua -ssl`,  the "?.so" in LUA_CPATH is replaced by ssl.so.

OK.  So in LUA_PATH you need to list the actual .lua files (not just
their dirname), and in LUA_CPATH the .so files (not just their dirname),
right?

>> Besides, it would be nice to get rid of these hard-coded /usr/local in a
>> subsequent patch.
>
> This is related to my next step and I need some advice.
>
> We need to set LUA_PATH and LUA_CPATH properly in order for programs to
> load external Lua libraries. I have been installing Lua libraries into
> ${out}/lib/lua/${LUA_VERSION} and ${out}/share/lua/${LUA_VERSION}, so
> that they end up at, for example, ~/.guix-profile/lib/lua/5.1/ssl.so [2]

It seems we need a ‘native-search-paths’ thingie here, along these
lines:

  (search-path-specification
    (variable "LUA_PATH")
    (files '("share/lua/5.1"))
    (file-pattern "\\.lua$")
    (file-type 'regular))

See the ‘libxml2’ package for an example of how this works.

Something similar is needed for ‘LUA_CPATH’.  And then we need to make
sure to replace “5.1” with the actual version number.

That way, when Lua and luasec are in the same profile, ‘guix package
--search-paths’ will suggest the right environment variable settings
(and likewise within build environments.)

> Nixpkgs makes it all work by setting the paths in wrappers around
> programs that need to find Lua libraries. I don't fully understand how
> they generate the paths but I get the idea. Some examples:
> https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/xmpp/prosody/default.nix
> https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/video/quvi/tool.nix
>
> What do you think?

That sounds like the right thing.  In Guix this is achieved with
‘wrap-program’; there are several examples of packages doing that.

HTH!

Ludo’.

Re: [PATCH 4/5] gnu: Use make-flags and modify-phases for lua-5.2.

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> * gnu/packages/lua.scm (lua-5.2)[arguments]: Use make-flags and
>   modify-phases to control build process, replacing use of
>   'alist-' procedures.

Sorry for dropping the ball.  This patch LGTM, but I think it depends on
a previous one, no?  At least it doesn’t apply to current master.

Thanks,
Ludo’.

Re: [PATCH 5/5] gnu: Build lua-5.1 with dynamic library support and a dynamic library.

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> * gnu/packages/lua.scm (lua-5.1)[arguments]: Rewrite make-flags so that
>   Lua is built with platform-specific instructions for shared library
>   loading (dlopen).
> * gnu/packages/patches/lua51-liblua-so.patch: Install liblua.so with
>   execute bit set. Move "-fPIC" flag from patch to package definition.

[...]

> +       #:make-flags (list "PLAT= linux"
> +                          "CFLAGS= -O2 -Wall -fPIC $(MYCFLAGS)"
> +                          "MYLDFLAGS= -fPIC"

Why not just:

  MYCFLAGS=-fPIC

instead of CFLAGS=…?  This would be consistent with how MYLDFLAGS is
used.

> +                          (string-append "INSTALL_TOP= "
> +                                         (assoc-ref %outputs "out"))
> +                          (string-append "INSTALL_MAN= "
> +                                         (assoc-ref %outputs "out")
> +                                         "/share/man/man1"))))))

I’m under the impression that these two variables aren’t needed since
things already get installed in the right place, no?

Thanks,
Ludo’.

Re: [PATCH 5/5] gnu: Build lua-5.1 with dynamic library support and a dynamic library.

[Leo Famulari]

On Sun, Dec 13, 2015 at 05:27:36PM +0100, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > * gnu/packages/lua.scm (lua-5.1)[arguments]: Rewrite make-flags so that
> >   Lua is built with platform-specific instructions for shared library
> >   loading (dlopen).
> > * gnu/packages/patches/lua51-liblua-so.patch: Install liblua.so with
> >   execute bit set. Move "-fPIC" flag from patch to package definition.
> 
> [...]
> 
> > +       #:make-flags (list "PLAT= linux"
> > +                          "CFLAGS= -O2 -Wall -fPIC $(MYCFLAGS)"
> > +                          "MYLDFLAGS= -fPIC"
> 
> Why not just:
> 
>   MYCFLAGS=-fPIC
> 
> instead of CFLAGS=…?  This would be consistent with how MYLDFLAGS is
> used.

Because lua-5.1's Makefile is broken in a tricky way.

When building Lua you are supposed to set PLAT in order to tailor the
build to your platform. The problem is that MYCFLAGS is clobbered and
redefined based on the value of PLAT. So, MYCFLAGS isn't actually
available for customizing the in the way that MYLDFLAGS is. [0]

CFLAGS, on the other hand, never gets redefined, so we can use it to
pass -fPIC. [1] The other CFLAGS options in the patch are default values
in the upstream Makefile that I chose to preserve.

> 
> > +                          (string-append "INSTALL_TOP= "
> > +                                         (assoc-ref %outputs "out"))
> > +                          (string-append "INSTALL_MAN= "
> > +                                         (assoc-ref %outputs "out")
> > +                                         "/share/man/man1"))))))
> 
> I’m under the impression that these two variables aren’t needed since
> things already get installed in the right place, no?

The current lua-5.1 package definition is inheriting from lua-5.2, where
these are set. Since this patch series gives lua-5.1 its own
(arguments), I reproduced these variables.

I've rebased the unmerged parts of the patch series on master, and made
the requested changes. I'll send them shortly.

[0] Actually, MYLDFLAGS gets clobbered as well if you build for the
platforms 'aix' or 'mingw'.

[1] Again, 'aix' clobbers CFLAGS.

Re: [PATCH 4/5] gnu: Use make-flags and modify-phases for lua-5.2.

[Leo Famulari]

On Sun, Dec 13, 2015 at 05:24:14PM +0100, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > * gnu/packages/lua.scm (lua-5.2)[arguments]: Use make-flags and
> >   modify-phases to control build process, replacing use of
> >   'alist-' procedures.
> 
> Sorry for dropping the ball.  This patch LGTM, but I think it depends on
> a previous one, no?  At least it doesn’t apply to current master.

I've rebased against master and merged any requested changes to prior
patches. Updated patch forthcoming...

Re: [PATCH 2/5] gnu: Build lua-5.2 with dynamic library support.

[Leo Famulari]

On Sun, Nov 08, 2015 at 10:08:23PM +0100, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > * gnu/packages/lua.scm (lua-5.2)[arguments]: Rewrite make-flags so that
> >   Lua is built with platform-specific instructions for shared library
> >   loading (dlopen).
> 
> Please move the explanations as a comment in the code.

Done. Updated patches forthcoming...

> 
> >         #:phases (alist-replace
> >                   'build
> > -                 (lambda _ (zero? (system* "make" "CFLAGS=-fPIC" "linux")))
> > +                 (lambda _ (zero? (system* "make"
> > +                                           "PLAT=linux"
> > +                                           "MYCFLAGS=-fPIC"
> > +                                           "MYLDFLAGS=-fPIC")))
> 
> How can I test whether this works as expected?  I tried this, but I’m
> unsure whether it’s supposed to load OpenSSL’s libssl.so directly or
> not:
> 
> --8<---------------cut here---------------start------------->8---
> $ ./pre-inst-env guix environment --ad-hoc lua-5.2 openssl 
> [env]$ echo $PATH
> /gnu/store/zmqhwsl9vvxr4ihdnhwwpc3dpgmpsgsy-openssl-1.0.2d/bin:/gnu/store/9wmsrx7s9ydln5vgddms3cfavaiigwq2-lua-5.2.3/bin:/home/ludo/src/guix/scripts:/home/ludo/src/guix:/home/ludo/soft/bin:/home/ludo/.opam/system/bin:/home/ludo/.guix-profile/bin:/home/ludo/.guix-profile/sbin:/run/setuid-programs:/run/current-system/profile/bin:/run/current-system/profile/sbin
> [env]$ export LD_LIBRARY_PATH=/gnu/store/zmqhwsl9vvxr4ihdnhwwpc3dpgmpsgsy-openssl-1.0.2d/lib
> [env]$ lua -lssl
> lua: module 'ssl' not found:
> 	no field package.preload['ssl']
> 	no file '/usr/local/share/lua/5.2/ssl.lua'
> 	no file '/usr/local/share/lua/5.2/ssl/init.lua'
> 	no file '/usr/local/lib/lua/5.2/ssl.lua'
> 	no file '/usr/local/lib/lua/5.2/ssl/init.lua'
> 	no file './ssl.lua'
> 	no file '/usr/local/lib/lua/5.2/ssl.so'
> 	no file '/usr/local/lib/lua/5.2/loadall.so'
> 	no file './ssl.so'
> stack traceback:
> 	[C]: in function 'require'
> 	[C]: in ?
> --8<---------------cut here---------------end--------------->8---
> 
> Apologies in advance if I’m just asking stupid questions!
> 
> Besides, it would be nice to get rid of these hard-coded /usr/local in a
> subsequent patch.
> 
> Ludo’.