re: Drafting a Guix blog post on the FHS container

re: Drafting a Guix blog post on the FHS container

[Jim Newsome]

Sorry for (presumably) breaking threading; I came across this online and 
don't see a way to set my in-reply-to-email header properly.

Anyways just thought I'd mention that I recently learned about this 
feature, and was able to use it to get a downloaded [Tor Browser Bundle] 
running with:


```
guix shell \
   --container \
   --network \
   --emulate-fhs \
   --preserve='^DISPLAY$'
   --share=/run/user/$(id -u)/gdm \
   openssl@1 \
   libevent \
   pciutils \
   dbus-glib \
   bash \
   libgccjit \
   libcxx \
   gtk+ \
   coreutils \
   grep \
   sed \
   file \
   alsa-lib \
   -- \
   ./start-tor-browser.desktop -v
```

`--preserve='^DISPLAY$'` and `--share=/run/user/$(id -u)/gdm` are to get 
access to the display. I'm not sure the second parameter is universally 
correct; I reverse-engineered it via roughly `ps aux | grep -- -auth`.

The `-v` parameter to the browser script keeps it from trying to 
background itself, which otherwise causes the container and browser to 
terminate.

It'd ultimately be nice to package the Tor Browser Bundle properly for 
guix, but it's nice to be able to use it this way in the meantime.

-Jim

[Tor Browser Bundle]: https://www.torproject.org/download/

Re: Drafting a Guix blog post on the FHS container

[Ludovic Courtès]

Hi Jim,

Jim Newsome <jim@sporksmith.net> skribis:

> Anyways just thought I'd mention that I recently learned about this
> feature, and was able to use it to get a downloaded [Tor Browser
> Bundle] running with:

That’s another good example, thanks for sharing!  It’s especially
interesting because the Tor project vets the binaries they provide.

Ludo’.

Re: Drafting a Guix blog post on the FHS container

[Csepp]

Jim Newsome <jim@sporksmith.net> writes:

> Sorry for (presumably) breaking threading; I came across this online
> and don't see a way to set my in-reply-to-email header properly.
>
> Anyways just thought I'd mention that I recently learned about this
> feature, and was able to use it to get a downloaded [Tor Browser
> Bundle] running with:
>
>
> ```
> guix shell \
>   --container \
>   --network \
>   --emulate-fhs \
>   --preserve='^DISPLAY$'
>   --share=/run/user/$(id -u)/gdm \
>   openssl@1 \
>   libevent \
>   pciutils \
>   dbus-glib \
>   bash \
>   libgccjit \
>   libcxx \
>   gtk+ \
>   coreutils \
>   grep \
>   sed \
>   file \
>   alsa-lib \
>   -- \
>   ./start-tor-browser.desktop -v
> ```
>
> `--preserve='^DISPLAY$'` and `--share=/run/user/$(id -u)/gdm` are to
> get access to the display. I'm not sure the second parameter is
> universally correct; I reverse-engineered it via roughly `ps aux |
> grep -- -auth`.
>
> The `-v` parameter to the browser script keeps it from trying to
> background itself, which otherwise causes the container and browser to
> terminate.
>
> It'd ultimately be nice to package the Tor Browser Bundle properly for
> guix, but it's nice to be able to use it this way in the meantime.
>
> -Jim
>
> [Tor Browser Bundle]: https://www.torproject.org/download/

Any idea how to use this for running appimages?  Or anything that
requires FUSE in general?

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi all,

On Fri, Dec 23, 2022 at 03:04 AM, Csepp wrote:

> Jim Newsome <jim@sporksmith.net> writes:
>
>> Sorry for (presumably) breaking threading; I came across this online
>> and don't see a way to set my in-reply-to-email header properly.
>>
>> Anyways just thought I'd mention that I recently learned about this
>> feature, and was able to use it to get a downloaded [Tor Browser
>> Bundle] running with:
>>
>>
>> ```
>> guix shell \
>>   --container \
>>   --network \
>>   --emulate-fhs \
>>   --preserve='^DISPLAY$'
>>   --share=/run/user/$(id -u)/gdm \
>>   openssl@1 \
>>   libevent \
>>   pciutils \
>>   dbus-glib \
>>   bash \
>>   libgccjit \
>>   libcxx \
>>   gtk+ \
>>   coreutils \
>>   grep \
>>   sed \
>>   file \
>>   alsa-lib \
>>   -- \
>>   ./start-tor-browser.desktop -v
>> ```
>>
>> `--preserve='^DISPLAY$'` and `--share=/run/user/$(id -u)/gdm` are to
>> get access to the display. I'm not sure the second parameter is
>> universally correct; I reverse-engineered it via roughly `ps aux |
>> grep -- -auth`.
>>

Thanks for the example! That's a slight variation of what I've seen/used for display access, as you can see in my previous examples and the current draft of the blog post: <https://issues.guix.gnu.org/60112>

>> The `-v` parameter to the browser script keeps it from trying to
>> background itself, which otherwise causes the container and browser to
>> terminate.
>>
>> It'd ultimately be nice to package the Tor Browser Bundle properly for
>> guix, but it's nice to be able to use it this way in the meantime.
>>

Yes, that's handy, and some extra isolation via the container too.

>> -Jim
>>
>> [Tor Browser Bundle]: <https://www.torproject.org/download/>
>
> Any idea how to use this for running appimages?  Or anything that
> requires FUSE in general?

Please see my previous emails and the current draft, linked above, for exactly that. In short, use '--appimage-extract-and-run' instead of letting the appimage try to mount itself via FUSE.

For FUSE, one can't run it directly in the container as it is setuid. You can use flatpak-spawn from flatpak-xdg-utils though, to work around that, sort of. To be merged as soon as I have a chance (sadly dealing with an unexpected crisis at home these past weeks so I haven't committed anything yet): <https://issues.guix.gnu.org/59825>

Unfortunately the container that does this, or actually any created before the mounting, will not see the mounted appimage. I think this has something to do with namespaces and how containers are created, but I'm not sure the details. You can see some discussion of this in the IRC logs, but I can provide more summary later if you are interested, roughly <https://logs.guix.gnu.org/guix/2022-12-08.log#211221> and <https://logs.guix.gnu.org/guix/2022-12-09.log#020202> has the discussion.

John

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi Jim,

On Fri, Dec 16, 2022 at 05:39 PM, Jim Newsome wrote:

> Sorry for (presumably) breaking threading; I came across this online and
> don't see a way to set my in-reply-to-email header properly.
>
> Anyways just thought I'd mention that I recently learned about this
> feature, and was able to use it to get a downloaded [Tor Browser Bundle]
> running with:
>
>
> ```
> guix shell \
>    --container \
>    --network \
>    --emulate-fhs \
>    --preserve='^DISPLAY$'
>    --share=/run/user/$(id -u)/gdm \
>    openssl@1 \
>    libevent \
>    pciutils \
>    dbus-glib \
>    bash \
>    libgccjit \
>    libcxx \
>    gtk+ \
>    coreutils \
>    grep \
>    sed \
>    file \
>    alsa-lib \
>    -- \
>    ./start-tor-browser.desktop -v
> ```
>
> `--preserve='^DISPLAY$'` and `--share=/run/user/$(id -u)/gdm` are to get
> access to the display. I'm not sure the second parameter is universally
> correct; I reverse-engineered it via roughly `ps aux | grep -- -auth`.
>
> The `-v` parameter to the browser script keeps it from trying to
> background itself, which otherwise causes the container and browser to
> terminate.
>
> It'd ultimately be nice to package the Tor Browser Bundle properly for
> guix, but it's nice to be able to use it this way in the meantime.

Thanks again for this! I slightly modified it for the blog post, which you can see in draft form at <https://issues.guix.gnu.org/60112>. I used 'gcc:lib' instead of 'libgccjit' as it is smaller, and changed the needed display options to be like the previous ones I had. Yours didn't work for me since it looks like it relies on sharing something from GDM, which I don't use. But do let me know if my version doesn't work for you.

Also gave you credit for this example; if you prefer not to be mentioned by name/link to the mailing list for any reason, just let me know.

Oh, and we do have some (older) patches for building the Tor Browser from source, but I don't know if they currently work: <https://issues.guix.gnu.org/42380> Your example was great though, something very useful!

John

Re: Drafting a Guix blog post on the FHS container

[Jim Newsome]

On Wed, Jan 4, 2023, at 5:47 PM, John Kehayias wrote:
> Hi Jim,
> 
> On Fri, Dec 16, 2022 at 05:39 PM, Jim Newsome wrote:
> 
> > Sorry for (presumably) breaking threading; I came across this online and
> > don't see a way to set my in-reply-to-email header properly.
> >
> > Anyways just thought I'd mention that I recently learned about this
> > feature, and was able to use it to get a downloaded [Tor Browser Bundle]
> > running with:
> >
> >
> > ```
> > guix shell \
> >    --container \
> >    --network \
> >    --emulate-fhs \
> >    --preserve='^DISPLAY$'
> >    --share=/run/user/$(id -u)/gdm \
> >    openssl@1 \
> >    libevent \
> >    pciutils \
> >    dbus-glib \
> >    bash \
> >    libgccjit \
> >    libcxx \
> >    gtk+ \
> >    coreutils \
> >    grep \
> >    sed \
> >    file \
> >    alsa-lib \
> >    -- \
> >    ./start-tor-browser.desktop -v
> > ```
> >
> > `--preserve='^DISPLAY$'` and `--share=/run/user/$(id -u)/gdm` are to get
> > access to the display. I'm not sure the second parameter is universally
> > correct; I reverse-engineered it via roughly `ps aux | grep -- -auth`.
> >
> > The `-v` parameter to the browser script keeps it from trying to
> > background itself, which otherwise causes the container and browser to
> > terminate.
> >
> > It'd ultimately be nice to package the Tor Browser Bundle properly for
> > guix, but it's nice to be able to use it this way in the meantime.
> 
> Thanks again for this! I slightly modified it for the blog post, which you can see in draft form at <https://issues.guix.gnu.org/60112>. I used 'gcc:lib' instead of 'libgccjit' as it is smaller, and changed the needed display options to be like the previous ones I had. Yours didn't work for me since it looks like it relies on sharing something from GDM, which I don't use. But do let me know if my version doesn't work for you.
> 
> Also gave you credit for this example; if you prefer not to be mentioned by name/link to the mailing list for any reason, just let me know.
> 
> Oh, and we do have some (older) patches for building the Tor Browser from source, but I don't know if they currently work: <https://issues.guix.gnu.org/42380> Your example was great though, something very useful!
> 
> John

Thanks, looks good, and the command in your patch also works for me.

I agree that passing and exposing XAUTHORITY seems better. Experimentally, sharing the directory read-only also works (using `--expose` instead of `--share`) also works, but I'm not familiar enough with this mechanism to be confident that'll work for everyone, or whether making it read-only is worth the fuss.

Btw it turns out that `libevent` and `openssl@1` can be dropped; they're already bundled. All together, here's my current "best" version:

```
guix shell --container --network --emulate-fhs \
    --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --expose=$XAUTHORITY \
    alsa-lib bash coreutils dbus-glib file gcc:lib grep gtk+ \
    libcxx pciutils sed \
    -- ./start-tor-browser.desktop -v
```

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi Jim,

On Wed, Jan 04, 2023 at 06:07 PM, Jim Newsome wrote:

> Thanks, looks good, and the command in your patch also works for me.
>

Great, thanks for testing!

> I agree that passing and exposing XAUTHORITY seems better. Experimentally, sharing the directory
> read-only also works (using `--expose` instead of `--share`) also works, but I'm not familiar enough with
> this mechanism to be confident that'll work for everyone, or whether making it read-only is worth the
> fuss.
>

Ah, you are right, that seems to be just fine for VSCodium and Tor, in my quick test. I think I'll change that.

> Btw it turns out that `libevent` and `openssl@1` can be dropped; they're already bundled. All together,
> here's my current "best" version:
>
> ```
> guix shell --container --network --emulate-fhs \
>     --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --expose=$XAUTHORITY \
>     alsa-lib bash coreutils dbus-glib file gcc:lib grep gtk+ \
>     libcxx pciutils sed \
>     -- ./start-tor-browser.desktop -v
> ```

Nice, thanks for that too! I tried eliminating a few random inputs, but they were needed. It is difficult sometimes to get a really minimal set, but this looks good to me.

John

Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi Guixers!

I've started working on a little blog post about our new --emulate-fhs option for Guix containers. In short, this sets up an FHS-like (Filesystem Hierarchy Standard) container which has things like /lib and /bin.

I would like to get some suggestions on good examples to include. More general feedback, questions, and other comments are also welcome! I've included a rough draft of the beginning of the post, leading up to showing some examples.

(I've sent this to the devel and help list as I think input from different types of users will be helpful given the feature being discussed. I'm not currently subscribed to the help list, so please cc the devel list or me directly.)

One question: what is appropriate or recommended for examples concerning things like pre-built binaries? As an example, I had tested the FHS container by running the Siril appimage, which has since been packaged for Guix (nice work!). There are ones that I don't see that happening for anytime soon, like an Electron-based app. Something like VSCodium is very popular, free (as in freedom and I believe the FSDG sense), but just not something you can package fully from source due to JavaScript as I understand it. It runs in the FHS container.

Examples I was thinking of including: using rustup (uses pre-build rust binaries) and building a package that depends on newer (nightly) rust, like eww <https://github.com/elkowar/eww> This builds and nicely is screenshot-able with pretty looking desktop widgets.

What would be useful examples? What is the right line to toe regarding binaries? I don't want to necessarily advocate for that, yet sometimes we may feel we have no other choice or want to be able to test something. I was thinking to keep it to what we do have packaged in Guix yet may want to run in a different way, or something that would fit if the language ecosystem wasn't so at odds with the Guix approach (and reproducibility more generally).

Appreciative of any and all thoughts!

John


Here is a current (rough!) draft. For the ease of plain text email I've exported from the org source to text with some light edits:


______________________________

FHS COMES TO GUIX CONTAINERS

        John Kehayias
______________________________


GNU Guix is different from most other GNU/Linux distributions and
perhaps nowhere is that more obvious than the organization of the
filesystem: Guix does not conform to the [File Hierarchy Standard]
(FHS). In practical terms, this means there is no global `/lib'
containing libraries, `/bin' containing binaries[1], and so on. This is
very much at the core of how Guix works and some of the convenient
features, like per-user installation of programs (different versions,
for instance) and a declarative system configuration where the system is
determined from a configuration file.

However, this also leads to a difference in how many pieces of software
expect their world to look like, relying on finding a library in `/lib'
or an external tool in `/bin'. When these are hard coded and not
overcome with appropriate build options, we patch code to refer to
absolute paths in the store, like
`/gnu/store/hrgqa7m498wfavq4awai3xz86ifkjxdr-grep-3.6/bin/grep', to keep
everything consistently contained within the store.

It all works great and is thanks to the hard work of everyone that has
contributed to Guix. But what if we need a more FHS-like environment for
developing, testing, or running a piece of software?

To that end, we've [recently added] a new option for Guix containers,
`--emulate-fhs' (or `-F'). This will set up an environment in the
container that follows FHS expectations, so that libraries are visible
in `/lib' in the container, as an example. Additionally, for the more
technically-minded, the [`glibc' used in this container] will read from
a global cache in `/etc/ld.so.cache' contrary to the behavior in [Guix
otherwise].

Here is a very simple example:
,----
 guix shell --container --emulate-fhs coreutils -- ls /bin
`----

[
b2sum
base32
base64
basename
basenc
cat
catchsegv
chcon
chgrp
chmod
...

Contrast that with `/bin' on a Guix system:
,----
 ls /bin -la
`----

lrwxrwxrwx  1  root  root   61  Dec  3  16:37  sh  ->  /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin/sh

There are several uses that spring to mind for such a container in Guix.
For developers, or those aspiring to hack on a project, this is a
helpful tool when needing to emulate a different (non-Guix) environment.
For example, one could use this to more easily follow build instructions
meant for a general distribution, say when a Guix package is not (yet)
available or easy to write immediately. Another usage is to be able to
use tools that don't really fit into Guix's model, like ones that use
pre-built binaries. There are many reasons why this is not ideal and
Guix strives to replace or supplement such tools, but practically
speaking they can be hard to avoid entirely. The FHS container helps
bridge this gap, providing an isolated and reproducible environment as
needed.

Users not interested in development will also find the FHS container
useful. For example, there may be software that is free and conforms to
the FSDG Guix follows, yet is not feasible to be [packaged] by our
standards. JavaScript and particularly Electron applications are not yet
packaged for Guix due to the [difficulties] of a properly source-based
and bootstrapable approach in this ecosystem.


[File Hierarchy Standard]
<https://refspecs.linuxfoundation.org/fhs.shtml>

[recently added]
<https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c7ba5f38b80433b040d3946b8fc0b1e8621ba30a>

[`glibc' used in this container]
<https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3d1d29e440910a99531b738f8f090de2cd4df9da>

[Guix otherwise]
<https://guix.gnu.org/blog/2021/taming-the-stat-storm-with-a-loader-cache/>

[packaged] <https://hpc.guix.info/blog/2021/09/whats-in-a-package/>

[difficulties]
<https://dustycloud.org/blog/javascript-packaging-dystopia/>



Footnotes
_________

[1] Other than a symlink for `sh' from the `bash' package, for
compatibility reasons.

Re: Drafting a Guix blog post on the FHS container

[Wojtek Kosior via]

[-- Attachment #1: Type: text/plain, Size: 7720 bytes --]

Hello,


> Hi Guixers!
> 
> I've started working on a little blog post about our new
> --emulate-fhs option for Guix containers. In short, this sets up an
> FHS-like (Filesystem Hierarchy Standard) container which has things
> like /lib and /bin.
> 
> I would like to get some suggestions on good examples to include.
> More general feedback, questions, and other comments are also
> welcome! I've included a rough draft of the beginning of the post,
> leading up to showing some examples.

Hi,

One recent real-life example of `--emulate-fhs` being useful is with
Python's virtualenv. I mentioned it in one help-guix thread which
you can see here[1] :)

Wojtek

[1] https://lists.gnu.org/archive/html/help-guix/2022-11/msg00305.html

-- (sig_start)
website: https://koszko.org/koszko.html
PGP: https://koszko.org/key.gpg
fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A

Meet Kraków saints!           #11: saint Jacek Odrowąż
Poznaj świętych krakowskich!  #11: święty Jacek Odrowąż
https://pl.wikipedia.org/wiki/Jacek_Odrowąż
-- (sig_end)


On Mon, 05 Dec 2022 02:32:40 +0000
John Kehayias <john.kehayias@protonmail.com> wrote:

> Hi Guixers!
> 
> I've started working on a little blog post about our new --emulate-fhs option for Guix containers. In short, this sets up an FHS-like (Filesystem Hierarchy Standard) container which has things like /lib and /bin.
> 
> I would like to get some suggestions on good examples to include. More general feedback, questions, and other comments are also welcome! I've included a rough draft of the beginning of the post, leading up to showing some examples.
> 
> (I've sent this to the devel and help list as I think input from different types of users will be helpful given the feature being discussed. I'm not currently subscribed to the help list, so please cc the devel list or me directly.)
> 
> One question: what is appropriate or recommended for examples concerning things like pre-built binaries? As an example, I had tested the FHS container by running the Siril appimage, which has since been packaged for Guix (nice work!). There are ones that I don't see that happening for anytime soon, like an Electron-based app. Something like VSCodium is very popular, free (as in freedom and I believe the FSDG sense), but just not something you can package fully from source due to JavaScript as I understand it. It runs in the FHS container.
> 
> Examples I was thinking of including: using rustup (uses pre-build rust binaries) and building a package that depends on newer (nightly) rust, like eww <https://github.com/elkowar/eww> This builds and nicely is screenshot-able with pretty looking desktop widgets.
> 
> What would be useful examples? What is the right line to toe regarding binaries? I don't want to necessarily advocate for that, yet sometimes we may feel we have no other choice or want to be able to test something. I was thinking to keep it to what we do have packaged in Guix yet may want to run in a different way, or something that would fit if the language ecosystem wasn't so at odds with the Guix approach (and reproducibility more generally).
> 
> Appreciative of any and all thoughts!
> 
> John
> 
> 
> Here is a current (rough!) draft. For the ease of plain text email I've exported from the org source to text with some light edits:
> 
> 
> ______________________________
> 
> FHS COMES TO GUIX CONTAINERS
> 
>         John Kehayias
> ______________________________
> 
> 
> GNU Guix is different from most other GNU/Linux distributions and
> perhaps nowhere is that more obvious than the organization of the
> filesystem: Guix does not conform to the [File Hierarchy Standard]
> (FHS). In practical terms, this means there is no global `/lib'
> containing libraries, `/bin' containing binaries[1], and so on. This is
> very much at the core of how Guix works and some of the convenient
> features, like per-user installation of programs (different versions,
> for instance) and a declarative system configuration where the system is
> determined from a configuration file.
> 
> However, this also leads to a difference in how many pieces of software
> expect their world to look like, relying on finding a library in `/lib'
> or an external tool in `/bin'. When these are hard coded and not
> overcome with appropriate build options, we patch code to refer to
> absolute paths in the store, like
> `/gnu/store/hrgqa7m498wfavq4awai3xz86ifkjxdr-grep-3.6/bin/grep', to keep
> everything consistently contained within the store.
> 
> It all works great and is thanks to the hard work of everyone that has
> contributed to Guix. But what if we need a more FHS-like environment for
> developing, testing, or running a piece of software?
> 
> To that end, we've [recently added] a new option for Guix containers,
> `--emulate-fhs' (or `-F'). This will set up an environment in the
> container that follows FHS expectations, so that libraries are visible
> in `/lib' in the container, as an example. Additionally, for the more
> technically-minded, the [`glibc' used in this container] will read from
> a global cache in `/etc/ld.so.cache' contrary to the behavior in [Guix
> otherwise].
> 
> Here is a very simple example:
> ,----
>  guix shell --container --emulate-fhs coreutils -- ls /bin
> `----
> 
> [
> b2sum
> base32
> base64
> basename
> basenc
> cat
> catchsegv
> chcon
> chgrp
> chmod
> ...
> 
> Contrast that with `/bin' on a Guix system:
> ,----
>  ls /bin -la
> `----
> 
> lrwxrwxrwx  1  root  root   61  Dec  3  16:37  sh  ->  /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin/sh
> 
> There are several uses that spring to mind for such a container in Guix.
> For developers, or those aspiring to hack on a project, this is a
> helpful tool when needing to emulate a different (non-Guix) environment.
> For example, one could use this to more easily follow build instructions
> meant for a general distribution, say when a Guix package is not (yet)
> available or easy to write immediately. Another usage is to be able to
> use tools that don't really fit into Guix's model, like ones that use
> pre-built binaries. There are many reasons why this is not ideal and
> Guix strives to replace or supplement such tools, but practically
> speaking they can be hard to avoid entirely. The FHS container helps
> bridge this gap, providing an isolated and reproducible environment as
> needed.
> 
> Users not interested in development will also find the FHS container
> useful. For example, there may be software that is free and conforms to
> the FSDG Guix follows, yet is not feasible to be [packaged] by our
> standards. JavaScript and particularly Electron applications are not yet
> packaged for Guix due to the [difficulties] of a properly source-based
> and bootstrapable approach in this ecosystem.
> 
> 
> [File Hierarchy Standard]
> <https://refspecs.linuxfoundation.org/fhs.shtml>
> 
> [recently added]
> <https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c7ba5f38b80433b040d3946b8fc0b1e8621ba30a>
> 
> [`glibc' used in this container]
> <https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3d1d29e440910a99531b738f8f090de2cd4df9da>
> 
> [Guix otherwise]
> <https://guix.gnu.org/blog/2021/taming-the-stat-storm-with-a-loader-cache/>
> 
> [packaged] <https://hpc.guix.info/blog/2021/09/whats-in-a-package/>
> 
> [difficulties]
> <https://dustycloud.org/blog/javascript-packaging-dystopia/>
> 
> 
> 
> Footnotes
> _________
> 
> [1] Other than a symlink for `sh' from the `bash' package, for
> compatibility reasons.
> 
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi Wojtek,

Thanks for your input!

On Mon, Dec 05, 2022 at 07:51 AM, Wojtek Kosior wrote:

> Hello,
>
>
>> Hi Guixers!
>>
>> I've started working on a little blog post about our new
>> --emulate-fhs option for Guix containers. In short, this sets up an
>> FHS-like (Filesystem Hierarchy Standard) container which has things
>> like /lib and /bin.
>>
>> I would like to get some suggestions on good examples to include.
>> More general feedback, questions, and other comments are also
>> welcome! I've included a rough draft of the beginning of the post,
>> leading up to showing some examples.
>
> Hi,
>
> One recent real-life example of `--emulate-fhs` being useful is with
> Python's virtualenv. I mentioned it in one help-guix thread which
> you can see here[1] :)
>
> Wojtek
>
> [1] <https://lists.gnu.org/archive/html/help-guix/2022-11/msg00305.html>
>

I took a look but wasn't sure the exact commands I would use to show this, I guess it is when invoking vitualenv and using Guix python packages? I'm not a virtualenv user, so apologies for the simple question!

Though I'm happy to generally mention this as an example use and point to the linked email thread.

Thanks again,
John

Re: Drafting a Guix blog post on the FHS container

[Ludovic Courtès]

Hello!

John Kehayias <john.kehayias@protonmail.com> skribis:

> One question: what is appropriate or recommended for examples concerning things like pre-built binaries? As an example, I had tested the FHS container by running the Siril appimage, which has since been packaged for Guix (nice work!). There are ones that I don't see that happening for anytime soon, like an Electron-based app. Something like VSCodium is very popular, free (as in freedom and I believe the FSDG sense), but just not something you can package fully from source due to JavaScript as I understand it. It runs in the FHS container.

A good example might be a free application not currently packaged in
Guix, for example due to being full of JavaScript, or nightly builds as
you wrote provided by an upstream project.

> Here is a current (rough!) draft. For the ease of plain text email I've exported from the org source to text with some light edits:

Note that the blog takes Markdown¹, but hopefully the Org-to-markdown
export works well.

¹ https://git.savannah.gnu.org/cgit/guix/guix-artwork.git/tree/website/posts

The post looks great to me!  I have minor suggestions below:

> GNU Guix is different from most other GNU/Linux distributions and
> perhaps nowhere is that more obvious than the organization of the
> filesystem: Guix does not conform to the [File Hierarchy Standard]
> (FHS). In practical terms, this means there is no global `/lib'

It’s “Filesystem Hierarchy Standard”.

> To that end, we've [recently added] a new option for Guix containers,
> `--emulate-fhs' (or `-F'). This will set up an environment in the

Perhaps s/Guix containers/`guix shell`/ and add a few words about what
‘guix shell --container’ does (you can link to the manual or blog post).

> container that follows FHS expectations, so that libraries are visible
> in `/lib' in the container, as an example. Additionally, for the more
> technically-minded, the [`glibc' used in this container] will read from
> a global cache in `/etc/ld.so.cache' contrary to the behavior in [Guix
> otherwise].

Since the ld.so.cache issue is more involved (compared to simply having
/lib and /bin), maybe you can move it after the “ls /bin” example?

> Contrast that with `/bin' on a Guix system:
> ,----
>  ls /bin -la
> `----
>
> lrwxrwxrwx  1  root  root   61  Dec  3  16:37  sh  ->  /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin/sh

You can show ‘ls /lib’ too.  :-)

Actually you can use or get inspiration from this animated GIF if you
like:

  https://web.fdn.fr/~lcourtes/tmp/guix-shell-fhs.gif

> useful. For example, there may be software that is free and conforms to
> the FSDG Guix follows, yet is not feasible to be [packaged] by our

s/FSDG/Free System Distribution Guidelines (FSDG)/

Thanks,
Ludo’.

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi Ludo’ and Guixers,

Before I get to some quick responses, I wanted to share some FHS container examples I've been testing. I hope others can try as I did get a response on IRC that one didn't work as expected. I think it might be when needing to expose some of the host for things like hardware access.

First, let's dive right into a big one: the popular VSCodium editor. This is a freely licensed build of VS Code: <https://github.com/VSCodium/vscodium#why-does-this-exist>

This comes in AppImage format. Downloading it and making it executable (with a 'chmod +x') I can run it in a container as

--8<---------------cut here---------------start------------->8---
guix shell -CNF -D ungoogled-chromium gcc:lib \
     --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --share=$XAUTHORITY \
     --preserve='^DBUS_' --expose=/var/run/dbus \
     --expose=/sys/dev --expose=/sys/devices --expose=/dev/dri \
     -- ./VSCodium-1.74.0.22342.glibc2.17-x86_64.AppImage --appimage-extract-and-run
--8<---------------cut here---------------end--------------->8---

where the first line is a cheat I like to get lots of libraries often needed for graphical applications (development inputs of ungoogled-chromium) though it is a bit overkill if the AppImage does actually bundle everything (they don't!). Next line is for display on the host's X server, the one after for DBus communication, and lastly exposing some of the host hardware for rendering (perhaps can be disabled in VSCodium somehow?). This is what may need some tweaking for others, but I'm not sure.

Note that we can't run an AppImage with out the 'appimage-extract-and-run' as it will want to use fuse to mount the image which we can't do from the container. I have some more details on this and actually did get this to mount, though it wasn't visible from the container, in the coming blog post draft.

Another example is to get the latest nightly builds of Rust, via <https://rustup.rs/>

--8<---------------cut here---------------start------------->8---
$ mkdir ~/temphome

$ guix shell -NCF bash coreutils curl grep nss-certs gcc:lib gcc-toolchain pkg-config glib cairo atk pango@1.48.10 gdk-pixbuf gtk+ git --share=$HOME/temphome=$HOME

~/temphome [env]$ curl --proto '=https' --tlsv1.2 -sSf <https://sh.rustup.rs> | sh
--8<---------------cut here---------------end--------------->8---

where I first created a '~/temphome' directory to use as $HOME in the container and included a bunch of libraries for the next example.

Finally, we can build a Rust project of desktop widgets, <https://github.com/elkowar/eww>, following their directions <https://elkowar.github.io/eww/> Ultimately this uses just 'cargo build --release' and this builds after downloading all the needed libraries. It needs similar stuff from the host as the VSCodium example to get things to run and display, which I'll detail in the blog post.

Happy to try other examples and to hear feedback on these!

Now, back to the rest of the email.

On Tue, Dec 06, 2022 at 11:41 AM, Ludovic Courtès wrote:

> Hello!
>
> John Kehayias <john.kehayias@protonmail.com> skribis:
>
>> One question: what is appropriate or recommended for examples concerning things like
>> pre-built binaries? As an example, I had tested the FHS container by running the Siril
>> appimage, which has since been packaged for Guix (nice work!). There are ones that I
>> don't see that happening for anytime soon, like an Electron-based app. Something like
>> VSCodium is very popular, free (as in freedom and I believe the FSDG sense), but just
>> not something you can package fully from source due to JavaScript as I understand it. It
>> runs in the FHS container.
>
> A good example might be a free application not currently packaged in
> Guix, for example due to being full of JavaScript, or nightly builds as
> you wrote provided by an upstream project.
>

I used VSCodium above, though honestly I've only seen that it opens and seems to work fine. Open to other suggestions but that one will probably get some attention :)

>> Here is a current (rough!) draft. For the ease of plain text email I've exported from
>> the org source to text with some light edits:
>
> Note that the blog takes Markdown¹, but hopefully the Org-to-markdown
> export works well.
>
> ¹ <https://git.savannah.gnu.org/cgit/guix/guix-artwork.git/tree/website/posts>
>

Thanks, and I saw simon's email about this as well. I may have to tweak the Markdown a little, but should work easily enough.

> The post looks great to me!  I have minor suggestions below:
>

Thank you!

>> GNU Guix is different from most other GNU/Linux distributions and
>> perhaps nowhere is that more obvious than the organization of the
>> filesystem: Guix does not conform to the [File Hierarchy Standard]
>> (FHS). In practical terms, this means there is no global `/lib'
>
> It’s “Filesystem Hierarchy Standard”.
>
>> To that end, we've [recently added] a new option for Guix containers,
>> `--emulate-fhs' (or `-F'). This will set up an environment in the
>
> Perhaps s/Guix containers/`guix shell`/ and add a few words about what
> ‘guix shell --container’ does (you can link to the manual or blog post).
>
>> container that follows FHS expectations, so that libraries are visible
>> in `/lib' in the container, as an example. Additionally, for the more
>> technically-minded, the [`glibc' used in this container] will read from
>> a global cache in `/etc/ld.so.cache' contrary to the behavior in [Guix
>> otherwise].
>
> Since the ld.so.cache issue is more involved (compared to simply having
> /lib and /bin), maybe you can move it after the “ls /bin” example?
>
>> Contrast that with `/bin' on a Guix system:
>> ,----
>>  ls /bin -la
>> `----
>>
>> lrwxrwxrwx 1 root root 61 Dec 3 16:37 sh ->
>> /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin/sh
>
> You can show ‘ls /lib’ too.  :-)
>

Thanks for all these corrections and tips! Sadly(?) no matter how many times I've written out FHS I still get it wrong sometimes.

> Actually you can use or get inspiration from this animated GIF if you
> like:
>
>   <https://web.fdn.fr/~lcourtes/tmp/guix-shell-fhs.gif>
>

Either I forgot to save this or wasn't able to access it before, and can't access it now.

>> useful. For example, there may be software that is free and conforms to
>> the FSDG Guix follows, yet is not feasible to be [packaged] by our
>
> s/FSDG/Free System Distribution Guidelines (FSDG)/
>

Thanks!
John

Re: Drafting a Guix blog post on the FHS container

[Ludovic Courtès]

Hello!

John Kehayias <john.kehayias@protonmail.com> skribis:

> First, let's dive right into a big one: the popular VSCodium editor. This is a freely licensed build of VS Code: <https://github.com/VSCodium/vscodium#why-does-this-exist>
>
> This comes in AppImage format. Downloading it and making it executable (with a 'chmod +x') I can run it in a container as
>
> guix shell -CNF -D ungoogled-chromium gcc:lib \
>      --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --share=$XAUTHORITY \
>      --preserve='^DBUS_' --expose=/var/run/dbus \
>      --expose=/sys/dev --expose=/sys/devices --expose=/dev/dri \
>      -- ./VSCodium-1.74.0.22342.glibc2.17-x86_64.AppImage --appimage-extract-and-run

The code in that AppImage is free software, right?

> Another example is to get the latest nightly builds of Rust, via <https://rustup.rs/>

That’s a nice one too!

> Happy to try other examples and to hear feedback on these!

I think these are two good examples, likely to correspond to the kind of
thing people may want to try.

>> Actually you can use or get inspiration from this animated GIF if you
>> like:
>>
>>   <https://web.fdn.fr/~lcourtes/tmp/guix-shell-fhs.gif>
>>
>
> Either I forgot to save this or wasn't able to access it before, and can't access it now.

Yeah, the TLS setup on that machine is broken, so you’d have to “Accept
the risk and continue”; I sent you a copy off-list.

If we release on Monday, it would be great to have it published…
tomorrow (Friday).  Otherwise next Friday maybe?

Thanks!

Ludo’.

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi again!

On Thu, Dec 15, 2022 at 03:53 PM, Ludovic Courtès wrote:

> Hello!
>
> John Kehayias <john.kehayias@protonmail.com> skribis:
>
>> First, let's dive right into a big one: the popular VSCodium editor. This is a freely
>> licensed build of VS Code: <https://github.com/VSCodium/vscodium#why-does-this-exist>
>>
>> This comes in AppImage format. Downloading it and making it executable (with a 'chmod
>> +x') I can run it in a container as
>>
>> guix shell -CNF -D ungoogled-chromium gcc:lib \
>>      --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --share=$XAUTHORITY \
>>      --preserve='^DBUS_' --expose=/var/run/dbus \
>>      --expose=/sys/dev --expose=/sys/devices --expose=/dev/dri \
>>      -- ./VSCodium-1.74.0.22342.glibc2.17-x86_64.AppImage --appimage-extract-and-run
>
> The code in that AppImage is free software, right?
>

The usual disclaimer of IANAL, but I linked to the part of their readme which describes the project licensing. So it seems all free to me, similar maybe to ungoogled-chromium: all the source without the telemetry and non-free branding, etc. But please do check.

>> Another example is to get the latest nightly builds of Rust, via <https://rustup.rs/>
>
> That’s a nice one too!
>
>> Happy to try other examples and to hear feedback on these!
>
> I think these are two good examples, likely to correspond to the kind of
> thing people may want to try.
>

Great, thanks!

>>> Actually you can use or get inspiration from this animated GIF if you
>>> like:
>>>
>>>   <https://web.fdn.fr/~lcourtes/tmp/guix-shell-fhs.gif>
>>>
>>
>> Either I forgot to save this or wasn't able to access it before, and can't access it
>> now.
>
> Yeah, the TLS setup on that machine is broken, so you’d have to “Accept
> the risk and continue”; I sent you a copy off-list.
>

Thank you. I included it but realize now I forgot to add credit, so I can do that later.

> If we release on Monday, it would be great to have it published…
> tomorrow (Friday).  Otherwise next Friday maybe?
>

It has gotten crazy here this week, hence my slow response (and not using my new found commit powers yet!). I've just sent the draft post which is the previous version with changes you suggested and a slightly expanded version of the examples from my previous message.

<https://issues.guix.gnu.org/60112>

I did some manual tweaking to the markdown export but I think it should look okay. Wasn't sure about the footnotes or if that should just be in-text. Oh, and now realized may want it with a fill column rather than long lines. (It always wraps nicely on my Emacs setup so I forget.)

I should be able to make quick edits tomorrow (it is very late here now), but we don't need to rush if there are other changes or checks anyone wants to make. I can also record some screen grabs if that is helpful.

> Thanks!
>
> Ludo’.

Thanks for the input, wish I had some more time this past week to have gotten this done earlier.

John

Re: Drafting a Guix blog post on the FHS container

[zimoun]

Hi,

On Mon, 05 Dec 2022 at 02:32, John Kehayias <john.kehayias@protonmail.com> wrote:

> Here is a current (rough!) draft. For the ease of plain text email
> I've exported from the org source to text with some light edits:

Nice!  If you can turn the draft into Markdown and format a patch for
guix-artwork [1] under website/drafts, it could be awesome! :-)

Then you could send the patch to guix-devel, guix-blog or guix-patches.

1: <https://git.savannah.gnu.org/cgit/guix/guix-artwork.git>


Cheers,
simon

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi simon,

On Fri, Dec 09, 2022 at 06:56 PM, zimoun wrote:

> Hi,
>
> On Mon, 05 Dec 2022 at 02:32, John Kehayias <john.kehayias@protonmail.com> wrote:
>
>> Here is a current (rough!) draft. For the ease of plain text email
>> I've exported from the org source to text with some light edits:
>
> Nice!  If you can turn the draft into Markdown and format a patch for
> guix-artwork [1] under website/drafts, it could be awesome! :-)
>
> Then you could send the patch to guix-devel, guix-blog or guix-patches.
>
> 1: <https://git.savannah.gnu.org/cgit/guix/guix-artwork.git>
>
>

Ah, thanks for this info, I'll take care of that. In the meantime I'll at least send out the current examples I've been using, as I want to make sure they work pretty generally. Of course they should since they will produce the same environment (until perhaps what is run in the environment), but a few of the commands may depend on what needs to be shared from the host. For example, to get graphical support.

I'll get the markdown formatted patch sent too and update everyone on that.

Thanks!
John