bug#25425: lua does not set search paths

bug#25425: lua does not set search paths

[Ricardo Wurmus]

The lua interpreter packages do not set native search paths, so they
cannot find lua modules.

  $ guix environment --ad-hoc lua lua-lpeg
  $ lua
  > require('lpeg')
  … ERROR …

However:

  $ guix environment --ad-hoc lua lua-lpeg
  $ export LUA_PATH="$GUIX_ENVIRONMENT/share/lua/5.3/?.lua;$GUIX_ENVIRONMENT/share/lua/5.3/?/?.lua"
  $ export LUA_CPATH="$GUIX_ENVIRONMENT/lib/lua/5.3/?.so;$GUIX_ENVIRONMENT/lib/lua/5.3/?/?.so"
  $ lua
  > require('lpeg')
  table: 0x1b80300


--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
http://elephly.net

bug#25425: cannot express search path

[Ricardo Wurmus]

It seems that the “search-path-specification” cannot be used to include
placeholders such as “?” as required by LUA_PATH and LUA_CPATH.  The
search paths in Lua are not directories and they are not actual file
names.

I suggest adding an optional “placeholder” field to
“search-path-specification”, which would hold a string like “?/?.lua” in
this case.

bug#25425: cannot express search path

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> It seems that the “search-path-specification” cannot be used to include
> placeholders such as “?” as required by LUA_PATH and LUA_CPATH.  The
> search paths in Lua are not directories and they are not actual file
> names.

Indeed.

> I suggest adding an optional “placeholder” field to
> “search-path-specification”, which would hold a string like “?/?.lua” in
> this case.

I think this should work:

       (search-path-specification
         (variable "LUA_PATH")
         (separator ";")
         (files '("share/lua/5.3"))
         (file-pattern "\\.lua$")
         (file-type 'regular))

See libxml2 for an example with similar requirements.

HTH!

Ludo’.

bug#25425: cannot express search path

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> I think this should work:
>
>        (search-path-specification
>          (variable "LUA_PATH")
>          (separator ";")
>          (files '("share/lua/5.3"))
>          (file-pattern "\\.lua$")
>          (file-type 'regular))

I tried this very same thing but it doesn’t work because Lua expects
placeholders (“?”) in the search paths.  The placeholders are replaced
with the actual package names.  If the actual file name does not exist
it will try the next pattern.  If the file *does* exist – which *will* be
the case for any of the files on LUA_PATH that have been generated by
the search-path-specification — Lua will try to load the package from
that path.

This will fail because a search for the “lpeg” module would be satisfied
by the file “re.lua”, because that’s the first valid file on the
LUA_PATH.  “re.lua” requires “lpeg” itself, so another lookup is
performed, which will again result in “re.lua” to be loaded…

AIUI we must generate a value for LUA_PATH that keeps the placeholders
intact.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
http://elephly.net

bug#25425: cannot express search path

[Ludovic Courtès]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> I think this should work:
>>
>>        (search-path-specification
>>          (variable "LUA_PATH")
>>          (separator ";")
>>          (files '("share/lua/5.3"))
>>          (file-pattern "\\.lua$")
>>          (file-type 'regular))
>
> I tried this very same thing but it doesn’t work because Lua expects
> placeholders (“?”) in the search paths.  The placeholders are replaced
> with the actual package names.  If the actual file name does not exist
> it will try the next pattern.  If the file *does* exist – which *will* be
> the case for any of the files on LUA_PATH that have been generated by
> the search-path-specification — Lua will try to load the package from
> that path.
>
> This will fail because a search for the “lpeg” module would be satisfied
> by the file “re.lua”, because that’s the first valid file on the
> LUA_PATH.  “re.lua” requires “lpeg” itself, so another lookup is
> performed, which will again result in “re.lua” to be loaded…
>
> AIUI we must generate a value for LUA_PATH that keeps the placeholders
> intact.

So are you saying that it’s important for the question marks to remain
intact?

This sounds terrible.  I’m not sure how to address it, and I don’t feel
like stretching the search path mechanism this much.

Thoughts?

Ludo’.

bug#25425: cannot express search path

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> I think this should work:
>>>
>>>        (search-path-specification
>>>          (variable "LUA_PATH")
>>>          (separator ";")
>>>          (files '("share/lua/5.3"))
>>>          (file-pattern "\\.lua$")
>>>          (file-type 'regular))
>>
>> I tried this very same thing but it doesn’t work because Lua expects
>> placeholders (“?”) in the search paths.  The placeholders are replaced
>> with the actual package names.  If the actual file name does not exist
>> it will try the next pattern.  If the file *does* exist – which *will* be
>> the case for any of the files on LUA_PATH that have been generated by
>> the search-path-specification — Lua will try to load the package from
>> that path.
>>
>> This will fail because a search for the “lpeg” module would be satisfied
>> by the file “re.lua”, because that’s the first valid file on the
>> LUA_PATH.  “re.lua” requires “lpeg” itself, so another lookup is
>> performed, which will again result in “re.lua” to be loaded…
>>
>> AIUI we must generate a value for LUA_PATH that keeps the placeholders
>> intact.
>
> So are you saying that it’s important for the question marks to remain
> intact?

Yes, that seems to be the case.

> This sounds terrible.  I’m not sure how to address it, and I don’t feel
> like stretching the search path mechanism this much.
>
> Thoughts?

I agree.  It’s a really odd special case.  On the other hand, not
extending the search path mechanism would mean that we have to find
other ways to fix “guix environment”, build phases, and profiles when
Lua packages are involved.

Build phases are easy to fix (by using a procedure that sets the
appropriate environment variables depending on the build inputs), but I
don’t know how to make profiles behave the way they should.  Maybe it
would be fine to set a single value for LUA_PATH (and LUA_CPATH) that
assumes a single prefix (the profile path) and contains the necessary
placeholders.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#25425: cannot express search path

[zimoun]

Hi,

This is an old bug about Lua and search path.

  <http://issues.guix.gnu.org/issue/25425>

It is still unsolved:

--8<---------------cut here---------------start------------->8---
$ guix environment -C --ad-hoc lua lua-lpeg -- lua
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
> require('lpeg')
stdin:1: module 'lpeg' not found:
	no field package.preload['lpeg']
	no file '/usr/local/share/lua/5.3/lpeg.lua'
	no file '/usr/local/share/lua/5.3/lpeg/init.lua'
	no file '/usr/local/lib/lua/5.3/lpeg.lua'
	no file '/usr/local/lib/lua/5.3/lpeg/init.lua'
	no file './lpeg.lua'
	no file './lpeg/init.lua'
	no file '/usr/local/lib/lua/5.3/lpeg.so'
	no file '/usr/local/lib/lua/5.3/loadall.so'
	no file './lpeg.so'
stack traceback:
	[C]: in function 'require'
	stdin:1: in main chunk
	[C]: in ?
>
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
$ guix environment -C --ad-hoc lua lua-lpeg
[env]$ export LUA_PATH="$GUIX_ENVIRONMENT/share/lua/5.3/?.lua;$GUIX_ENVIRONMENT/share/lua/5.3/?/?.lua"
[env]$ export LUA_CPATH="$GUIX_ENVIRONMENT/lib/lua/5.3/?.so;$GUIX_ENVIRONMENT/lib/lua/5.3/?/?.so"
[env]$ lua
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
> require('lpeg')
table: 0x23fd510
>
--8<---------------cut here---------------end--------------->8---


On Sun, 22 Jan 2017 at 23:34, Ricardo Wurmus <rekado@elephly.net> wrote:
> Ludovic Courtès <ludo@gnu.org> writes:
>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>> Ludovic Courtès <ludo@gnu.org> writes:
>>>
>>>> I think this should work:
>>>>
>>>>        (search-path-specification
>>>>          (variable "LUA_PATH")
>>>>          (separator ";")
>>>>          (files '("share/lua/5.3"))
>>>>          (file-pattern "\\.lua$")
>>>>          (file-type 'regular))
>>>
>>> I tried this very same thing but it doesn’t work because Lua expects
>>> placeholders (“?”) in the search paths.  The placeholders are replaced
>>> with the actual package names.  If the actual file name does not exist
>>> it will try the next pattern.  If the file *does* exist – which *will* be
>>> the case for any of the files on LUA_PATH that have been generated by
>>> the search-path-specification — Lua will try to load the package from
>>> that path.
>>>
>>> This will fail because a search for the “lpeg” module would be satisfied
>>> by the file “re.lua”, because that’s the first valid file on the
>>> LUA_PATH.  “re.lua” requires “lpeg” itself, so another lookup is
>>> performed, which will again result in “re.lua” to be loaded…
>>>
>>> AIUI we must generate a value for LUA_PATH that keeps the placeholders
>>> intact.
>>
>> So are you saying that it’s important for the question marks to remain
>> intact?
>
> Yes, that seems to be the case.
>
>> This sounds terrible.  I’m not sure how to address it, and I don’t feel
>> like stretching the search path mechanism this much.
>>
>> Thoughts?
>
> I agree.  It’s a really odd special case.  On the other hand, not
> extending the search path mechanism would mean that we have to find
> other ways to fix “guix environment”, build phases, and profiles when
> Lua packages are involved.
>
> Build phases are easy to fix (by using a procedure that sets the
> appropriate environment variables depending on the build inputs), but I
> don’t know how to make profiles behave the way they should.  Maybe it
> would be fine to set a single value for LUA_PATH (and LUA_CPATH) that
> assumes a single prefix (the profile path) and contains the necessary
> placeholders.

Is it fixable?


All the best,
simon