Re: [PATCH 1/1] gnu: lua: Build with dynamic library support.

[Leo Famulari <leo@famulari.name>]

[-- Attachment #1: Type: text/plain, Size: 3211 bytes --]

I'm still working on this but I found something interesting that I want
to ask the list about. Attached is a diff of the build logs for lua-5.1
from master and lua-5.1 for the patch I sent upthread.

You may notice that they share the `make` invocation, but that gcc is
being invoked with different flags. Does anyone know why that is
happening?

I decided to compare the log files after analyzing the lua-5.1 Makefile
and realizing that my original patch should make no difference for
lua-5.1 (it would make a difference for lua-5.2). And if that is the
case, why am I getting different results?

The missing '-DLUA_USE_LINUX' is the issue for the lua-5.1 in master.
That flag specifies the use of dlopen, among other things.

On Tue, Oct 27, 2015, at 23:32, Leo Famulari wrote:
> Calling all Lua experts! Or even somebody that has used Lua before! ;)
> 
> I studied the lua-5.1 and lua-5.2 Makefiles more carefully and I will
> split this up into separate packages for each version, with some
> changes.
> 
> On Tue, Oct 27, 2015, at 11:48, Ludovic Courtès wrote:
> > By “fixing the flags”, do you mean that the .so that we have for 5.1
> > does not actually contain position-independent code?  That CFLAGS was
> > ignored?
> 
> I'm not sure about the state of our lua-5.1's liblua.so. How to test it?
> 
> These changes are about building Lua so that it can load Lua libraries
> dynamically. Currently, the lua-5.1 we package lacks this feature.
> Possibly the lua-5.2 is the same way.
> 
> > > I will have to look into it again. I tested the results by starting the
> > > Lua interpreter while linking a shared library:
> > > $ lua -lssl
> > 
> > I’m not familiar with Lua; what does this command do?
> 
> `lua` starts the interactive lua interpreter. Giving it the argument
> `-lname` calls "require('name') before executing script.  Typically used
> to load libraries." (quoted from them Lua man page). Some helpful people
> in the Prosody chatroom suggested it as a simple way to test if a Lua
> interpreter can load dynamic libraries or not. Our lua-5.1 can't:
> $ ll `which lua`
> lrwxrwxrwx 7 root guixbuild 61 Dec 31  1969
> /var/guix/profiles/per-user/leo/guix-profile/bin/lua ->
> /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/bin/lua
> $ lua -lssl
> lua: error loading module 'ssl.core' from file
> '/home/leo/.guix-profile/usr/lib/lua/5.1/ssl.so':
> 	dynamic libraries not enabled; check your Lua installation
> stack traceback:
> 	[C]: ?
> 	[C]: in function 'require'
> 	/home/leo/.guix-profile/usr/share/lua/5.1/ssl.lua:7: in main
> 	chunk
> 	[C]: ?
> 	[C]: ?
> 
> > I thought the goal was to get liblua.so for 5.2, but this seems to be
> > testing something else?
> 
> By default, building Lua does not create liblua.so, which is why the
> lua51-liblua-so.patch is there. I can look into a similar patch for 5.2. 
> 
> Also, we need to set up the search paths to help Lua find Lua libraries.
> That can be configured at build time. I figured I would try to get a
> working Lua interpreter in master before working on those things, but I
> am open to guidance.
> 
> Revised patches forthcoming.
> 
> Leo

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: master-contrib.diff --]
[-- Type: text/x-diff; name="master-contrib.diff", Size: 11616 bytes --]

--- lua-5.1-master.log	2015-10-28 00:55:35.056412401 -0400
+++ lua-5.1-contrib.log	2015-10-28 18:17:29.127392079 -0400
@@ -1,6 +1,3 @@
-The following derivation will be built:
-   /gnu/store/djjyfzi09p8ikjq03vv7rkscjmg9sxq8-lua-5.1.5.drv
-@ build-started /gnu/store/djjyfzi09p8ikjq03vv7rkscjmg9sxq8-lua-5.1.5.drv - x86_64-linux /usr/local/var/log/guix/drvs/dj//jyfzi09p8ikjq03vv7rkscjmg9sxq8-lua-5.1.5.drv.bz2
 starting phase `set-paths'
 environment variable `PATH' set to `/gnu/store/xz8c9ihn3p1d8xqkhzf54bjvjqzvqkc0-tar-1.28/bin:/gnu/store/8m7mgixi5539vqrr1vd1nrvh96vjl66j-gzip-1.6/bin:/gnu/store/yq9vkx7f4zw17gs6b09bg4arcgmmjgxn-bzip2-1.0.6/bin:/gnu/store/hk31a1xlgf6n3yi1yzbdyd2bnjmxc60m-xz-5.0.4/bin:/gnu/store/wl84nj6x2mz7zf5r0ajx5gpi0p689ags-file-5.22/bin:/gnu/store/p1h463zxk0f56jl7snw7s8wxvz1k542q-diffutils-3.3/bin:/gnu/store/0xm4q87qi07gcig4riflrf65vyk1k0mp-patch-2.7.5/bin:/gnu/store/g55pwwrmgi58n1yr57m7vc0y69v1ba89-sed-4.2.2/bin:/gnu/store/7fk9v37a5q4r5sygywq0q0qg7iqrj517-findutils-4.4.2/bin:/gnu/store/k8qgvgwn5anbfy8r70h938kxgd46cyxx-gawk-4.1.3/bin:/gnu/store/021rg9bs7xivw0acp1yr0rxw6mf3h64d-grep-2.21/bin:/gnu/store/mnwjrkbfzkb5ifhqf8hssf3cxfvg11l6-coreutils-8.24/bin:/gnu/store/22k2zmc74dvnri91ma0lwxyyhfi1srk0-make-4.1/bin:/gnu/store/311nvir0pz1mhf0mgsmfrw00qfj7yq0j-bash-4.3.39/bin:/gnu/store/k6r37137lfpg3l3igi50c7lj2za7kqly-ld-wrapper-0/bin:/gnu/store/y5psndwpbbkjrf856x757psb708y62dn-binutils-2.25.1/bin:/gnu/store/hddjjpkfvwaf1j1q3qwpvby0rid3k8by-gcc-4.9.3/bin:/gnu/store/qv7bk62c22ms9i11dhfl71hnivyc82k2-glibc-2.22/bin:/gnu/store/qv7bk62c22ms9i11dhfl71hnivyc82k2-glibc-2.22/sbin:/gnu/store/ksgpmjqi9l8z012n18zbac1bijs1jdrn-readline-6.3/bin:/gnu/store/mahpzasbp7i0v7aqq8970rl7paq5mwln-ncurses-6.0/bin'
 environment variable `CPATH' set to `/gnu/store/yq9vkx7f4zw17gs6b09bg4arcgmmjgxn-bzip2-1.0.6/include:/gnu/store/hk31a1xlgf6n3yi1yzbdyd2bnjmxc60m-xz-5.0.4/include:/gnu/store/wl84nj6x2mz7zf5r0ajx5gpi0p689ags-file-5.22/include:/gnu/store/k8qgvgwn5anbfy8r70h938kxgd46cyxx-gawk-4.1.3/include:/gnu/store/22k2zmc74dvnri91ma0lwxyyhfi1srk0-make-4.1/include:/gnu/store/y5psndwpbbkjrf856x757psb708y62dn-binutils-2.25.1/include:/gnu/store/hddjjpkfvwaf1j1q3qwpvby0rid3k8by-gcc-4.9.3/include:/gnu/store/qv7bk62c22ms9i11dhfl71hnivyc82k2-glibc-2.22/include:/gnu/store/ksgpmjqi9l8z012n18zbac1bijs1jdrn-readline-6.3/include:/gnu/store/lyn2331ilik14yy2jqhndshvxmv9r6w5-linux-libre-headers-3.14.37/include:/gnu/store/mahpzasbp7i0v7aqq8970rl7paq5mwln-ncurses-6.0/include'
@@ -127,83 +124,77 @@
 phase `patch-generated-file-shebangs' succeeded after 0.0 seconds
 starting phase `build'
 cd src && make linux V=5.1 R=5.1.5
-make[1]: Entering directory '/tmp/nix-build-lua-5.1.5.drv-0/lua-5.1.5/src'
+make[1]: Entering directory '/home/leo/tmp/nix-build-lua-5.1.5.drv-0/lua-5.1.5/src'
 make all MYCFLAGS=-DLUA_USE_LINUX MYLIBS="-Wl,-E -ldl -lreadline -lhistory -lncurses"
-make[2]: Entering directory '/tmp/nix-build-lua-5.1.5.drv-0/lua-5.1.5/src'
-gcc -fPIC   -c -o lapi.o lapi.c
-gcc -fPIC   -c -o lcode.o lcode.c
-gcc -fPIC   -c -o ldebug.o ldebug.c
-gcc -fPIC   -c -o ldo.o ldo.c
-gcc -fPIC   -c -o ldump.o ldump.c
-gcc -fPIC   -c -o lfunc.o lfunc.c
-gcc -fPIC   -c -o lgc.o lgc.c
-gcc -fPIC   -c -o llex.o llex.c
-gcc -fPIC   -c -o lmem.o lmem.c
-gcc -fPIC   -c -o lobject.o lobject.c
-gcc -fPIC   -c -o lopcodes.o lopcodes.c
-gcc -fPIC   -c -o lparser.o lparser.c
-gcc -fPIC   -c -o lstate.o lstate.c
-gcc -fPIC   -c -o lstring.o lstring.c
-gcc -fPIC   -c -o ltable.o ltable.c
-gcc -fPIC   -c -o ltm.o ltm.c
-gcc -fPIC   -c -o lundump.o lundump.c
-gcc -fPIC   -c -o lvm.o lvm.c
-gcc -fPIC   -c -o lzio.o lzio.c
-gcc -fPIC   -c -o lauxlib.o lauxlib.c
-gcc -fPIC   -c -o lbaselib.o lbaselib.c
-gcc -fPIC   -c -o ldblib.o ldblib.c
-gcc -fPIC   -c -o liolib.o liolib.c
-gcc -fPIC   -c -o lmathlib.o lmathlib.c
-gcc -fPIC   -c -o loslib.o loslib.c
-gcc -fPIC   -c -o ltablib.o ltablib.c
-gcc -fPIC   -c -o lstrlib.o lstrlib.c
-gcc -fPIC   -c -o loadlib.o loadlib.c
-gcc -fPIC   -c -o linit.o linit.c
+make[2]: Entering directory '/home/leo/tmp/nix-build-lua-5.1.5.drv-0/lua-5.1.5/src'
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lapi.o lapi.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lcode.o lcode.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o ldebug.o ldebug.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o ldo.o ldo.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o ldump.o ldump.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lfunc.o lfunc.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lgc.o lgc.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o llex.o llex.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lmem.o lmem.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lobject.o lobject.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lopcodes.o lopcodes.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lparser.o lparser.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lstate.o lstate.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lstring.o lstring.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o ltable.o ltable.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o ltm.o ltm.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lundump.o lundump.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lvm.o lvm.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lzio.o lzio.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lauxlib.o lauxlib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lbaselib.o lbaselib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o ldblib.o ldblib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o liolib.o liolib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lmathlib.o lmathlib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o loslib.o loslib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o ltablib.o ltablib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lstrlib.o lstrlib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o loadlib.o loadlib.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o linit.o linit.c
 ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o	# DLL needs all object files
 ar: `u' modifier ignored since `D' is the default (see `U')
 ranlib liblua.a
-gcc -fPIC   -c -o lua.o lua.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o lua.o lua.c
 gcc -o lua  lua.o liblua.a -lm -Wl,-E -ldl -lreadline -lhistory -lncurses
-liblua.a(loslib.o): In function `os_tmpname':
-loslib.c:(.text+0x1a5): warning: the use of `tmpnam' is dangerous, better use `mkstemp'
-gcc -fPIC   -c -o luac.o luac.c
-gcc -fPIC   -c -o print.o print.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o luac.o luac.c
+gcc -O2 -Wall -DLUA_USE_LINUX -fPIC   -c -o print.o print.c
 gcc -o luac  luac.o print.o liblua.a -lm -Wl,-E -ldl -lreadline -lhistory -lncurses
 gcc -shared -ldl -Wl,-soname,liblua.so.5.1 -o liblua.so.5.1.5 lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o -lm 
-loslib.o: In function `os_tmpname':
-loslib.c:(.text+0x1a5): warning: the use of `tmpnam' is dangerous, better use `mkstemp'
 ln -sf liblua.so.5.1.5 liblua.so.5.1
 ln -sf liblua.so.5.1.5 liblua.so
-make[2]: Leaving directory '/tmp/nix-build-lua-5.1.5.drv-0/lua-5.1.5/src'
-make[1]: Leaving directory '/tmp/nix-build-lua-5.1.5.drv-0/lua-5.1.5/src'
-phase `build' succeeded after 1.6 seconds
+make[2]: Leaving directory '/home/leo/tmp/nix-build-lua-5.1.5.drv-0/lua-5.1.5/src'
+make[1]: Leaving directory '/home/leo/tmp/nix-build-lua-5.1.5.drv-0/lua-5.1.5/src'
+phase `build' succeeded after 5.1 seconds
 starting phase `check'
 src/lua test/hello.lua
 Hello world, from Lua 5.1!
 phase `check' succeeded after 0.0 seconds
 starting phase `install'
-cd src && mkdir -p /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/bin /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/include /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/lib /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/share/man/man1 /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/share/lua/5.1 /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/lib/lua/5.1
-cd src && install -p -m 0755 lua luac /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/bin
-cd src && install -p -m 0644 lua.h luaconf.h lualib.h lauxlib.h ../etc/lua.hpp /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/include
-cd src && install -p -m 0644 liblua.a liblua.so liblua.so.5.1 /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/lib
-cd doc && install -p -m 0644 lua.1 luac.1 /gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/share/man/man1
+cd src && mkdir -p /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/bin /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/include /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/lib /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/share/man/man1 /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/share/lua/5.1 /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/lib/lua/5.1
+cd src && install -p -m 0755 lua luac /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/bin
+cd src && install -p -m 0644 lua.h luaconf.h lualib.h lauxlib.h ../etc/lua.hpp /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/include
+cd src && install -p -m 0644 liblua.a liblua.so liblua.so.5.1 /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/lib
+cd doc && install -p -m 0644 lua.1 luac.1 /gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/share/man/man1
 phase `install' succeeded after 0.0 seconds
 starting phase `patch-shebangs'
 phase `patch-shebangs' succeeded after 0.0 seconds
 starting phase `strip'
-stripping binaries in "/gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/lib" with "strip" and flags ("--strip-debug")
-stripping binaries in "/gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/bin" with "strip" and flags ("--strip-debug")
-phase `strip' succeeded after 0.0 seconds
+stripping binaries in "/gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/lib" with "strip" and flags ("--strip-debug")
+stripping binaries in "/gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/bin" with "strip" and flags ("--strip-debug")
+phase `strip' succeeded after 0.1 seconds
 starting phase `validate-runpath'
-validating RUNPATH of 2 binaries in "/gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/lib"...
-validating RUNPATH of 2 binaries in "/gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/bin"...
+validating RUNPATH of 2 binaries in "/gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/lib"...
+validating RUNPATH of 2 binaries in "/gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/bin"...
 phase `validate-runpath' succeeded after 0.0 seconds
 starting phase `validate-documentation-location'
 phase `validate-documentation-location' succeeded after 0.0 seconds
 starting phase `delete-info-dir-file'
 phase `delete-info-dir-file' succeeded after 0.0 seconds
 starting phase `compress-documentation'
-compressing documentation in '/gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5/share/man' with "gzip" and flags ("--best" "--no-name")
+compressing documentation in '/gnu/store/v51lrdxh8h09bf5d3ml92g9nkxqmympr-lua-5.1.5/share/man' with "gzip" and flags ("--best" "--no-name")
 phase `compress-documentation' succeeded after 0.0 seconds
-@ build-succeeded /gnu/store/djjyfzi09p8ikjq03vv7rkscjmg9sxq8-lua-5.1.5.drv -
-/gnu/store/fl27mjm8kxp0rj989cd8mj67qjvl0jr3-lua-5.1.5