Suggestion for a guix shell feature.

Suggestion for a guix shell feature.

[Apoorv via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

I want to make a suggestion for a feature for guix shell.

I was recently writing guix.scm files for my projects and was having problem with the libraries I added, not being found inside the shell. After struggling for hours I finally learned that that the LD_LIBRARY_PATH  is not being updated automatically inside the  guix shell environment so the libraries can be found easily.

So my feature suggestion is that maybe we can have a flag or a option for guix shell, something like --add-ld-path or --update-ld-path for example, to add all libraries specified in the guix.scm or even as sub-command guix shell --add-ld-path mesa as an example, to automatically update the LD_LIBRARY_PATH environment variable to contain the correct paths to these libraries so they can be found easily inside the shell.

--
Sent with Tuta; enjoy secure & ad-free emails:
https://tuta.com

[-- Attachment #2: Type: text/html, Size: 1999 bytes --]

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv via "Development of GNU Guix and the GNU System distribution."
<guix-devel@gnu.org> writes:

> I want to make a suggestion for a feature for guix shell.
>
> I was recently writing guix.scm files for my projects and was having
> problem with the libraries I added, not being found inside the
> shell. After struggling for hours I finally learned that that the
> LD_LIBRARY_PATH  is not being updated automatically inside the  guix
> shell environment so the libraries can be found easily.
>
> So my feature suggestion is that maybe we can have a flag or a option
> for guix shell, something like --add-ld-path or --update-ld-path for
> example, to add all libraries specified in the guix.scm or even as
> sub-command guix shell --add-ld-path mesa as an example, to
> automatically update the LD_LIBRARY_PATH environment variable to
> contain the correct paths to these libraries so they can be found
> easily inside the shell.

Guix doesn't/shouldn't make use of LD_LIBRARY_PATH, except in rare cases
to wrap binaries.  It's better to patch the dlopen calls to use the
absolute shared library file name.

Perhaps you are missing the package configuring LIBRARY_PATH and other
useful environment variables for finding libraries?  That'd be
gcc-toolchain, if I recall correctly.

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Ekaitz Zarraga]

On 2023-12-29 04:58, Maxim Cournoyer wrote:
> Hi,
> 
> Apoorv via "Development of GNU Guix and the GNU System distribution."
> <guix-devel@gnu.org> writes:
> 
>> I want to make a suggestion for a feature for guix shell.
>>
>> I was recently writing guix.scm files for my projects and was having
>> problem with the libraries I added, not being found inside the
>> shell. After struggling for hours I finally learned that that the
>> LD_LIBRARY_PATH  is not being updated automatically inside the  guix
>> shell environment so the libraries can be found easily.
>>
>> So my feature suggestion is that maybe we can have a flag or a option
>> for guix shell, something like --add-ld-path or --update-ld-path for
>> example, to add all libraries specified in the guix.scm or even as
>> sub-command guix shell --add-ld-path mesa as an example, to
>> automatically update the LD_LIBRARY_PATH environment variable to
>> contain the correct paths to these libraries so they can be found
>> easily inside the shell.
> 
> Guix doesn't/shouldn't make use of LD_LIBRARY_PATH, except in rare cases
> to wrap binaries.  It's better to patch the dlopen calls to use the
> absolute shared library file name.
> 
> Perhaps you are missing the package configuring LIBRARY_PATH and other
> useful environment variables for finding libraries?  That'd be
> gcc-toolchain, if I recall correctly.
> 

Hi,

I have a similar issue. I have a project that links GLFW statically, but 
I still need to use its dependencies, like x11 and so on. If I make a 
shell like:

	guix shell -D glfw

I don't have access to those libraries, but they are in my GUIX_PROFILE.
The only way to access them is setting LD_LIBRARY_PATH. It doesn't 
matter if I have `gcc-toolchain` installed or not. Why is this?
The search path is hardcoded in the statically built GLFW?
How can we make this usecase simpler (mostly for development)?

Thanks,
Ekaitz

Re: Suggestion for a guix shell feature.

[Tobias Geerinckx-Rice]

Hiyall,

On 29 December 2023 03:58:27 UTC, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>Guix doesn't/shouldn't make use of LD_LIBRARY_PATH, except in rare cases
>to wrap binaries.  It's better to patch the dlopen calls to use the
>absolute shared library file name.

Fully agree.

Furthermore, '--ld-library-path' sounds like an inflexible alias of a nonexistent '--ad-hoc-search-path=FOO=/bar' option.

So why not instead write a cute little (name "LD_LIBRARY_PATH") fake package that does nothing but declare a search path for /lib?  It could live in any channel rather than further fatten 'guix shell'.

>Perhaps you are missing the package configuring LIBRARY_PATH and other
>useful environment variables for finding libraries?  That'd be
>gcc-toolchain, if I recall correctly.

I might be mistaken but I assumed this was more about running 'pre-existing' (cough cough nudge nudge) software not built from source.



Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Hiyall,
>
> On 29 December 2023 03:58:27 UTC, Maxim Cournoyer
> <maxim.cournoyer@gmail.com> wrote:
>>Guix doesn't/shouldn't make use of LD_LIBRARY_PATH, except in rare cases
>>to wrap binaries.  It's better to patch the dlopen calls to use the
>>absolute shared library file name.
>
> Fully agree.
>
> Furthermore, '--ld-library-path' sounds like an inflexible alias of a
> nonexistent '--ad-hoc-search-path=FOO=/bar' option.
>
> So why not instead write a cute little (name "LD_LIBRARY_PATH") fake
> package that does nothing but declare a search path for /lib?  It
> could live in any channel rather than further fatten 'guix shell'.
>
>>Perhaps you are missing the package configuring LIBRARY_PATH and other
>>useful environment variables for finding libraries?  That'd be
>>gcc-toolchain, if I recall correctly.
>
> I might be mistaken but I assumed this was more about running
> 'pre-existing' (cough cough nudge nudge) software not built from
> source.

Ah, if this is the case, the 'guix shell --container --emulate-fhs' may
help, by providing some FHS layout with libraries under e.g. /usr/lib/.

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv <apoorvs@tuta.io> writes:

> There are a lot of libs that need to be added to LD_LIBRARY_PATH for
> my project to compile successfully. I do have gcc-toolchain added in
> the guix.scm file. I had to do a lot of trickery for the shell
> environment to work for me properly, using direnv,

[...]

Did you try the 'guix shell --container --emulate-fhs' trick I suggested
earlier?  I guess it should just work.  What is it that you are
building/attempting to run?

Also, please keep guix-devel@gnu.org and participants in CC, so that
everyone can follow/participate (the easiest way to do so is to "wide
reply" from your email client).

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Apoorv via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]

Sorry, this is my first time using some mailing list.

I tried using `--container` and `--emulate-fhs` first it doesn't seem to use `guix.scm` automatically I have to specify it using `--file=guix.scm`. Second, the container is missing so much more stuff like cargo for example.

This is my own project in Rust lang, I basically define `guix.scm` for all my projects since I started using Guix.

-- 
 Sent with Tuta; enjoy secure & ad-free emails: 
 https://tuta.com


Dec 30, 2023, 03:10 by maxim.cournoyer@gmail.com:

> Hi,
>
> Apoorv <apoorvs@tuta.io> writes:
>
>> There are a lot of libs that need to be added to LD_LIBRARY_PATH for
>> my project to compile successfully. I do have gcc-toolchain added in
>> the guix.scm file. I had to do a lot of trickery for the shell
>> environment to work for me properly, using direnv,
>>
>
> [...]
>
> Did you try the 'guix shell --container --emulate-fhs' trick I suggested
> earlier?  I guess it should just work.  What is it that you are
> building/attempting to run?
>
> Also, please keep guix-devel@gnu.org and participants in CC, so that
> everyone can follow/participate (the easiest way to do so is to "wide
> reply" from your email client).
>
> -- 
> Thanks,
> Maxim
>


[-- Attachment #2: Type: text/html, Size: 1998 bytes --]

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv <apoorvs@tuta.io> writes:

> Sorry, this is my first time using some mailing list.
>
> I tried using `--container` and `--emulate-fhs` first it doesn't seem
> to use `guix.scm` automatically I have to specify it using
> `--file=guix.scm`. Second, the container is missing so much more stuff
> like cargo for example.

Yes, that's a somewhat surprising behavior designed to avoid bad
surprises, in that if you start using arguments with 'guix shell', the
'do what I mean' magic is cancelled and you need to explicit each
arguments.

About cargo missing, what prevents you from adding rust:cargo to your
guix.scm file?

> This is my own project in Rust lang, I basically define `guix.scm` for
> all my projects since I started using Guix.

I guess this being rust everything is statically compiled, and the usual
dynamic loader is not useful here (the RUNPATH baked in elf binaries is
not used).

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Apoorv via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 1259 bytes --]

I can add rust:cargo but the package definition in the guix.scm is already using `cargo-build-system` shouldn't it include rust and cargo etc automatically?

-- 
 Sent with Tuta; enjoy secure & ad-free emails: 
 https://tuta.com


Dec 31, 2023, 16:56 by maxim.cournoyer@gmail.com:

> Hi,
>
> Apoorv <apoorvs@tuta.io> writes:
>
>> Sorry, this is my first time using some mailing list.
>>
>> I tried using `--container` and `--emulate-fhs` first it doesn't seem
>> to use `guix.scm` automatically I have to specify it using
>> `--file=guix.scm`. Second, the container is missing so much more stuff
>> like cargo for example.
>>
>
> Yes, that's a somewhat surprising behavior designed to avoid bad
> surprises, in that if you start using arguments with 'guix shell', the
> 'do what I mean' magic is cancelled and you need to explicit each
> arguments.
>
> About cargo missing, what prevents you from adding rust:cargo to your
> guix.scm file?
>
>> This is my own project in Rust lang, I basically define `guix.scm` for
>> all my projects since I started using Guix.
>>
>
> I guess this being rust everything is statically compiled, and the usual
> dynamic loader is not useful here (the RUNPATH baked in elf binaries is
> not used).
>
> -- 
> Thanks,
> Maxim
>


[-- Attachment #2: Type: text/html, Size: 2023 bytes --]

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv <apoorvs@tuta.io> writes:

> I can add rust:cargo but the package definition in the guix.scm is already using `cargo-build-system` shouldn't it include rust and cargo etc automatically?

It should, but if it doesn't you may want to review how rust:cargo input
gets added to the cargo-build-system; perhaps it's done in a ad-hoc
fashion that causes that problem.

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Apoorv via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 978 bytes --]

OK, it was my bad I had an older copy of guix.scm file I was passing that, it didn't have cargo-build-system. But now I'm getting so many other errors related to cargo deps missing or version mismatch, I tried adding those deps spent like 1-2 hours but something dep was still missing or version mismatch, finally I gave up. I'm not sure how many deps I have to add like this.. this is getting way to complicated then it needs to be.

-- 
 Sent with Tuta; enjoy secure & ad-free emails: 
 https://tuta.com


Jan 2, 2024, 16:33 by maxim.cournoyer@gmail.com:

> Hi,
>
> Apoorv <apoorvs@tuta.io> writes:
>
>> I can add rust:cargo but the package definition in the guix.scm is already using `cargo-build-system` shouldn't it include rust and cargo etc automatically?
>>
>
> It should, but if it doesn't you may want to review how rust:cargo input
> gets added to the cargo-build-system; perhaps it's done in a ad-hoc
> fashion that causes that problem.
>
> -- 
> Thanks,
> Maxim
>


[-- Attachment #2: Type: text/html, Size: 1535 bytes --]

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv <apoorvs@tuta.io> writes:

> OK, it was my bad I had an older copy of guix.scm file I was passing
> that, it didn't have cargo-build-system. But now I'm getting so many
> other errors related to cargo deps missing or version mismatch, I
> tried adding those deps spent like 1-2 hours but something dep was
> still missing or version mismatch, finally I gave up. I'm not sure how
> many deps I have to add like this.. this is getting way to complicated
> then it needs to be.

cargo/rust is poorly integrated in Guix, as there's some impedance
mismatch between package handling by Guix and Cargo itself, which would
rather take care of it.

I'd expect most Rust developpers using Guix only do so to provide
Rust/Cargo, and let Cargo take care of the rest.  Otherwise you'd need
to replicate the pain the cargo-build-system goes through to make it
work.

A more future-proof approach if you'd like to improve Rust integration
into Guix would be to continue integrating the 'antioxydant' build
system effort started by Maxime Devos [0], which doesn't rely on Cargo
to build Rust crates (and supports dynamic linking of Rust libraries, I
think?).

[0]  https://notabug.org/maximed/cargoless-rust-experiments

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Apoorv via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 1936 bytes --]

Tried to use to antioxydant build system, I cannot seem to add it as a channel but was able to clone the repo and authenticate it manually..  then  guix build -L . -f antioxidant-packages.scm fails to run with error,
ice-9/eval.scm:223:20: In procedure proc:
error: rust-sequoia-ipc-0.26: unbound variable
hint: Did you forget a `use-modules' form?

I think I am going to stick to adding libs to LD_LIBRARY_PATH manually as it works even though I know you said its wrong.
-- 
 Sent with Tuta; enjoy secure & ad-free emails: 
 https://tuta.com


Jan 3, 2024, 17:57 by maxim.cournoyer@gmail.com:

> Hi,
>
> Apoorv <apoorvs@tuta.io> writes:
>
>> OK, it was my bad I had an older copy of guix.scm file I was passing
>> that, it didn't have cargo-build-system. But now I'm getting so many
>> other errors related to cargo deps missing or version mismatch, I
>> tried adding those deps spent like 1-2 hours but something dep was
>> still missing or version mismatch, finally I gave up. I'm not sure how
>> many deps I have to add like this.. this is getting way to complicated
>> then it needs to be.
>>
>
> cargo/rust is poorly integrated in Guix, as there's some impedance
> mismatch between package handling by Guix and Cargo itself, which would
> rather take care of it.
>
> I'd expect most Rust developpers using Guix only do so to provide
> Rust/Cargo, and let Cargo take care of the rest.  Otherwise you'd need
> to replicate the pain the cargo-build-system goes through to make it
> work.
>
> A more future-proof approach if you'd like to improve Rust integration
> into Guix would be to continue integrating the 'antioxydant' build
> system effort started by Maxime Devos [0], which doesn't rely on Cargo
> to build Rust crates (and supports dynamic linking of Rust libraries, I
> think?).
>
> [0]  https://notabug.org/maximed/cargoless-rust-experiments
>
> -- 
> Thanks,
> Maxim
>


[-- Attachment #2: Type: text/html, Size: 3152 bytes --]