Running IceCat in a container

Running IceCat in a container

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 2308 bytes --]

Hello, everyone:

I'm running IceCat in a container, with the goal of isolating it form
the rest of my system as much as possible without running a full
VM.  Here's what I have so far:

#+BEGIN_SRC sh
guix environment \
     --container \
     --network \
     -r "$gc_root" \
     --share=/tmp/.X11-unix/ \
     --expose=/etc/machine-id \
     --share=$HOME/.mozilla/ \
     --share=$HOME/.cache/mozilla/ \
     --share=$HOME/.Xauthority \
     --share=$HOME/Downloads/icecat-container/=$HOME/Downloads/ \
     --ad-hoc icecat coreutils
     -- \
     env DISPLAY="$DISPLAY" icecat "$@"
#+END_SRC

The most difficult problem I'm having is dealing with
fonts.  Specifically, I want to share the system fonts
(/run/current-system/profile/share/fonts).  The problem is, I can't just
expose that directory, because it symlinks into the store, and those
derivations don't exist within the container.

  - I do not want to expose all of /gnu.
  - I can provide the fonts as inputs to the environment, but I do not
    want to have to run fc-cache every time I start the container,
    because that is very slow.  Exposing the cache directory doesn't
    help since the derivation used in the container ($GUIX_ENVIRONMENT)
    always appears to be different than the font derivation used on my
    system, and also by my user.
  - I don't want to expose my user's entire ~/.guix-profile/.

I'm making things difficult for myself because I want as little
shared/exposed with the container as possible.

To complicate things further, for privacy, I don't want my user exposed
to the container via the name of my home directory; Guix creates that
automatically.  I haven't yet looked at the code to see what exactly it
does.

Is there a reasonable solution here?  Should I create a separate user
entirely and then just share the entire home directory?  I'm not sure
how that might impact X11 socket sharing, though.  Can I maybe
pre-create an image, already having run fc-cache, and run that image as
a container (like one would with Docker?)?  But that wouldn't solve my
user privacy issue.

Thanks,

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Running IceCat in a container

[Ludovic Courtès]

Hi Mike,

Mike Gerwitz <mtg@gnu.org> skribis:

> I'm running IceCat in a container, with the goal of isolating it form
> the rest of my system as much as possible without running a full
> VM.  Here's what I have so far:
>
> #+BEGIN_SRC sh
> guix environment \
>      --container \
>      --network \
>      -r "$gc_root" \
>      --share=/tmp/.X11-unix/ \
>      --expose=/etc/machine-id \
>      --share=$HOME/.mozilla/ \
>      --share=$HOME/.cache/mozilla/ \
>      --share=$HOME/.Xauthority \
>      --share=$HOME/Downloads/icecat-container/=$HOME/Downloads/ \
>      --ad-hoc icecat coreutils
>      -- \
>      env DISPLAY="$DISPLAY" icecat "$@"
> #+END_SRC

I’ve been dreaming of having it baked in into the shell (like Plash did;
we could write a Bash or Guile-Bash extension) or something along these
lines…

> The most difficult problem I'm having is dealing with
> fonts.  Specifically, I want to share the system fonts
> (/run/current-system/profile/share/fonts).  The problem is, I can't just
> expose that directory, because it symlinks into the store, and those
> derivations don't exist within the container.
>
>   - I do not want to expose all of /gnu.
>   - I can provide the fonts as inputs to the environment, but I do not
>     want to have to run fc-cache every time I start the container,
>     because that is very slow.  Exposing the cache directory doesn't
>     help since the derivation used in the container ($GUIX_ENVIRONMENT)
>     always appears to be different than the font derivation used on my
>     system, and also by my user.
>   - I don't want to expose my user's entire ~/.guix-profile/.
>
> I'm making things difficult for myself because I want as little
> shared/exposed with the container as possible.
>
> To complicate things further, for privacy, I don't want my user exposed
> to the container via the name of my home directory; Guix creates that
> automatically.  I haven't yet looked at the code to see what exactly it
> does.

“guix environment -C” makes $PWD shared; if you do (cd /tmp; guix
environment -C …), then /tmp is shared but not $HOME.

> Is there a reasonable solution here?  Should I create a separate user
> entirely and then just share the entire home directory?  I'm not sure
> how that might impact X11 socket sharing, though.  Can I maybe
> pre-create an image, already having run fc-cache, and run that image as
> a container (like one would with Docker?)?  But that wouldn't solve my
> user privacy issue.

Perhaps you could define a package that simply runs “fc-cache” with the
fonts it has as inputs, and then pass that to ‘guix environment’.

But really, we should make a specific tool for this.

Thoughts?

Ludo’.

Re: Running IceCat in a container

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 2568 bytes --]

On Tue, Jan 16, 2018 at 17:30:42 +0100, Ludovic Courtès wrote:
> “guix environment -C” makes $PWD shared; if you do (cd /tmp; guix
> environment -C …), then /tmp is shared but not $HOME.

I am doing that (I made a ~/.empty so as not to expose /tmp contents),
but that still creates the home dir (as documented):

  $ pwd && guix environment -C coreutils -- ls /home
  /tmp
  mikegerwitz

>> Is there a reasonable solution here?  Should I create a separate user
>> entirely and then just share the entire home directory?  I'm not sure
>> how that might impact X11 socket sharing, though.  Can I maybe
>> pre-create an image, already having run fc-cache, and run that image as
>> a container (like one would with Docker?)?  But that wouldn't solve my
>> user privacy issue.
>
> Perhaps you could define a package that simply runs “fc-cache” with the
> fonts it has as inputs, and then pass that to ‘guix environment’.

Oh, interesting; I wouldn't have thought of that.  If there is a general
solution/script, I think this needs to be considered---automatically
including system fonts; any program that displays text needs a broad
range of UTF-8/multi-lingual font support.  If I were to containerize my
shell, I'd have the same problem.

> But really, we should make a specific tool for this.
>
> Thoughts?

Yes, though I'd be curious how you'd approach it---each package requires
certain paths be shared, and those paths would further depend on user
privacy preferences, so need to be able to be overridden.  Perhaps it'd
be useful for those paths to be part of a package definition---the paths
that a program creates/uses at runtime, and perhaps additional metadata
associated with them, such as whether the path is necessary for its
operation (will it break the program if it's ephemeral or
read-only?).  Something extensible for the future.

Those directory metadata may have other uses that may make it worth
adding, but I haven't given it much thought.  For example, if a user
wishes to purge a package from her system, she could opt to purge those
paths from her home directory.  Or maybe Guix could create a backup of
user preferences such that a restoration would involve only a list of
packages and a tarball of those directories.  Might be useful for
provisioning as well.

Just some thoughts.  I'm too new to Guix to provide much.

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Running IceCat in a container

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 1780 bytes --]

On Tue, Jan 16, 2018 at 21:25:19 -0500, Mike Gerwitz wrote:
> On Tue, Jan 16, 2018 at 17:30:42 +0100, Ludovic Courtès wrote:
>> Perhaps you could define a package that simply runs “fc-cache” with the
>> fonts it has as inputs, and then pass that to ‘guix environment’.
>
> Oh, interesting; I wouldn't have thought of that.

Actually, I could use a little bit of help.

After hours of fontconf research and related stuff (more than I ever
cared to know), I think I'll be able to get away with running fc-cache
as you suggested using a package.

My ultimate goal I think is to still use the user's fonts, but I still
don't know a way to do that, since the /gnu symlinks are unavailable
within the container.[0] It _does_ work if the links are identical
between the host and cointainer---e.g. copying the font files into
~/.local/share/fonts, but that's obviously undesirable.

Unless you happen to know a good way to selectively expose those to a
container.

With that said, I'm having trouble creating a package: it wants a
`source' field, but this is a metapackage of sorts, and I didn't intend
on having any source files; I can generate them using the builder and
trivial-build-system.  How can I work around this?

Thanks.


[0]: What seems to be the case---which is probably obvious to anyone who
knows about this stuff---is that X11 on the host (since we're sharing
the socket) needs access to the font in addition to the software running
in the container.  I don't think this is the case for traditional X11
fonts (not using fontconf), but I'm not dealing with those.

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Running IceCat in a container

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]

On Wed, Jan 17, 2018 at 02:05:28PM -0500, Mike Gerwitz wrote:
> With that said, I'm having trouble creating a package: it wants a
> `source' field, but this is a metapackage of sorts, and I didn't intend
> on having any source files; I can generate them using the builder and
> trivial-build-system.  How can I work around this?

You can use (source #f) with the trivial build system for metapackages.

Check out the package definitions of 'gnome' and
'gnome-default-applications' in gnu/packages/gnome.scm for some
examples.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Running IceCat in a container

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 619 bytes --]

On Wed, Jan 17, 2018 at 15:20:44 -0800, Leo Famulari wrote:
> You can use (source #f) with the trivial build system for metapackages.
>
> Check out the package definitions of 'gnome' and
> 'gnome-default-applications' in gnu/packages/gnome.scm for some
> examples.

Ah, great, thank you!  I tried looking for examples, but it looks like I
never expected #f and inadvertently filtered the line out of my grep
results.  Those will be very helpful.

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Running IceCat in a container

[Ludovic Courtès]

Mike Gerwitz <mtg@gnu.org> skribis:

> On Tue, Jan 16, 2018 at 17:30:42 +0100, Ludovic Courtès wrote:

[...]

>> But really, we should make a specific tool for this.
>>
>> Thoughts?
>
> Yes, though I'd be curious how you'd approach it---each package requires
> certain paths be shared, and those paths would further depend on user
> privacy preferences, so need to be able to be overridden.

For a start, we could simply share everything that’s in:

  guix gc -R $(guix build the-package)

plus /tmp/.X11-unix, maybe $PWD, and a few more.

After that we could add the option to let users define what needs to be
shared (like the ‘file-system-mapping’ API we already have).  We could
have a predefined settings for X11 and console apps to minimize
boilerplate.  And like you write, the settings could be added as package
metadata.

Plash (Mark Seaborn’s POLA shell which used to live at
<http://plash.beasts.org/>) would automatically determine what needs to
be mapped based on the command line.  For instance, if you typed:

  gcc -o foo.o ~/src/bar/foo.c

then it would map ~/src/bar/foo.c as read-only and $(dirname foo.o) as
read-write.

Food for thought!

Ludo’.

Re: Running IceCat in a container

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 882 bytes --]

ludo@gnu.org (Ludovic Courtès) skribis:

> Mike Gerwitz <mtg@gnu.org> skribis:
>
>> On Tue, Jan 16, 2018 at 17:30:42 +0100, Ludovic Courtès wrote:
>
> [...]
>
>>> But really, we should make a specific tool for this.
>>>
>>> Thoughts?
>>
>> Yes, though I'd be curious how you'd approach it---each package requires
>> certain paths be shared, and those paths would further depend on user
>> privacy preferences, so need to be able to be overridden.
>
> For a start, we could simply share everything that’s in:
>
>   guix gc -R $(guix build the-package)
>
> plus /tmp/.X11-unix, maybe $PWD, and a few more.

If you drop the attached file under guix/scripts/, you can then run:

  guix run icecat icecat

and similar.  This particular example doesn’t work well because of the
font issue you’re familiar with, but you get the idea.  :-)

Ludo’.


[-- Attachment #2: The 'guix run' command. --]
[-- Type: text/x-scheme, Size: 4288 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix scripts run)
  #:use-module (guix ui)
  #:use-module (guix scripts)
  #:use-module (guix store)
  #:use-module (guix packages)
  #:use-module (guix derivations)
  #:use-module ((guix build utils) #:select (which mkdir-p))
  #:use-module (gnu build linux-container)
  #:use-module (gnu system file-systems)
  #:use-module (gnu packages)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-11)
  #:use-module (srfi srfi-37)
  #:use-module (ice-9 match)
  #:export (guix-run))

(define %options
  (list (option '(#\h "help") #f #f
                (lambda args
                  (show-help)
                  (exit 0)))
        (option '(#\V "version") #f #f
                (lambda args
                  (show-version-and-exit "guix run")))))

(define (show-help)
  (display (G_ "Usage: guix run PACKAGE COMMAND...
Run COMMAND from PACKAGE in a container.\n"))
  (newline)
  (display (G_ "
  -h, --help             display this help and exit"))
  (display (G_ "
  -V, --version          display version information and exit"))
  (newline)
  (show-bug-report-information))

\f

(define (bind-mount-spec/ro item)
  (and (file-exists? item)
       (file-system
         (device item)
         (mount-point item)
         (title 'device)
         (type "none")
         (flags '(bind-mount read-only))
         (check? #f))))

(define (bind-mount-spec/rw item)
  (and (file-exists? item)
       (file-system
         (inherit (bind-mount-spec/ro item))
         (flags '(bind-mount)))))

(define %writable-things
  (list "/var/run/nscd/socket"
        (string-append (getenv "HOME") "/.Xauthority")
        "/tmp/.X11-unix"
        "/etc/machine-id"))

(define (guix-run . args)
  (with-error-handling
    (match (reverse (parse-command-line args %options '(())
                                        #:argument-handler cons))
      ((spec command args ...)
       (with-store store
         (let-values (((package output)
                       (specification->package+output spec)))
           (let* ((drv    (package-derivation store package))
                  (prefix (derivation->output-path drv output)))
             (show-what-to-build store (list drv))
             (build-derivations store (list drv))

             (let* ((items (requisites store (list prefix)))
                    (env   (environ))
                    (full  (search-path (list (string-append prefix "/bin")
                                              (string-append prefix "/sbin"))
                                        command)))
               (unless full
                 (leave (G_ "command '~a' not found in package '~a'~%")
                        command (package-name package)))

               (call-with-container
                   (append (filter-map bind-mount-spec/ro items)
                           (filter-map bind-mount-spec/rw %writable-things))
                 (lambda ()
                   (environ env)                  ;TODO: filter ENV
                   (mkdir-p (getenv "HOME"))

                   (newline)
                   (catch #t
                     (lambda ()
                       (apply execl full command args))
                     (lambda (key . args)
                       (print-exception (current-error-port) #f key args)
                       (exit 1))))
                 #:namespaces (delq 'net %namespaces))))))))))

Re: Running IceCat in a container

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 5053 bytes --]

On Thu, Jan 25, 2018 at 23:16:47 +0100, Ludovic Courtès wrote:
> If you drop the attached file under guix/scripts/, you can then run:
>
>   guix run icecat icecat
>
> and similar.  This particular example doesn’t work well because of the
> font issue you’re familiar with, but you get the idea.  :-)

Oh, this is interesting.  I won't get a chance to try this out until
tomorrow, but I think it's a good start.

I sent a few patches moments ago that I've been sitting on for a
bit.  My intent was originally to go further, but I ran out of
time.  But I didn't think `guix environment' was the appropriate place
to put such things---this script, though, is a good starting point for
them.

For example, if one of the dependencies of a program is X11, it can
automatically share the X paths (unless overridden by the user).  Same
with DBUS, sound devices, etc.  I mentioned previous ideas earlier in
the thread.

I'd also want to integrate changes I made to `guix environment'.  If
people here like the changes and they are merged, I'd want to refactor
it into a common place, not just copy the code.

I think this gives us a lot to move forward with, and some good
discussion to have.  A lot of subtle details will have to be worked out,
like what default behavior should be.


Anyway, here's what I have so far.  I still have to get sound working; I
took a pause on that, not having spent more than a few minutes on it;
I'll get back to it hopefully in the next few days.  If anyone else
knows exactly what needs to be done, please lmk.


#+BEGIN_SRC sh
~/guix/pre-inst-env guix environment \
     --container \
     --link-profile \
     --no-cwd \
     --user=user \
     --network \
     -r "$gc_root" \
     --expose=/etc/machine-id \
     --expose=/tmp/.X11-unix/ \
     --expose=$HOME/.Xauthority \
     --share=/dev/snd \
     --share=$HOME/.mozilla/ \
     --share=$HOME/Downloads/icecat-container/=$HOME/Downloads/ \
     --ad-hoc mtg-icecat-containerized  \
     -- \
     icecat --display=:0.0 "$@" \
#+END_SRC


#+BEGIN_SRC scheme
(define-module (mtg personal)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix packages)
  #:use-module (gnu packages)
  #:use-module (guix build-system trivial)
  #:use-module (gnu packages gnome)
  #:use-module (gnu packages gnuzilla)
  #:use-module (gnu packages fonts)
  #:use-module (gnu packages fontutils)
  #:use-module (gnu packages pulseaudio))


(define-public mtg-icecat-containerized
  (package
    (name "mtg-icecat-containerized")
    (version "1.0")
    (home-page "https://mikegerwitz.com/")
    (build-system trivial-build-system)
    (source #f)
    (native-inputs
     `(("fontconfig" ,fontconfig)))
    (build-system trivial-build-system)
    (arguments
     `(#:modules ((guix build utils))
       #:builder
       (begin
         (use-modules (guix build utils))
         (let* ((share-dir (string-append %output "/share"))
                (cache-dir (string-append %output "/var/cache"))
                (bin-dir   (string-append %output "/bin"))
                (fc-dir    (string-append share-dir "/fontconfig/conf.avail"))
                (fc-mtg    (string-append fc-dir "/52-mtg-container.conf"))
                (fc-cache-dir (string-append cache-dir "/fontconfig"))
                (fonts-dir (string-append share-dir "/fonts")))
           ;; container script to invoke IceCat
           (mkdir-p bin-dir)
           (call-with-output-file (string-append bin-dir "icecat-container")
             (lambda (port)
               (format port "#!/bin/bash")))

           ;; fontconfig configuration
           (mkdir-p fc-dir)
           (call-with-output-file fc-mtg
             (lambda (port)
               (format port (string-append "<?xml version=\"1.0\"?>
<!DOCTYPE fontconfig SYSTEM \"fonts.dtd\">
<fontconfig>
  <dir>" (string-append (assoc-ref %build-inputs "font-dejavu")
                        "/share/fonts") "</dir>
  <cachedir>" fc-cache-dir "</cachedir>
</fontconfig>\n"))))

           (setenv "PATH"
                   (string-append (assoc-ref %build-inputs "fontconfig")
                                  "/bin"))
           (setenv "FONTCONFIG_FILE" fc-mtg)
           (setenv "XDG_DATA_HOME" share-dir)

           (mkdir-p cache-dir)
           (invoke "fc-cache" "-fv")))))
    (propagated-inputs
     `(("icecat" ,icecat)
       ("zenity" ,zenity)
       ("font-dejavu" ,font-dejavu)
       ("pulseaudio" ,pulseaudio)
       ;;("font-adobe-source-han-sans", font-adobe-source-han-sans)
       ))
    (synopsis "GNU IceCat packaged for running within a container")
    (description
     "GNU IceCat packaged with various fonts (including multi-lingual).
Suitable for use within a container.")
    (license license:gpl3+)))
#+END_SRC

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Running IceCat in a container

[Ludovic Courtès]

Heya,

Mike Gerwitz <mtg@gnu.org> skribis:

> On Thu, Jan 25, 2018 at 23:16:47 +0100, Ludovic Courtès wrote:
>> If you drop the attached file under guix/scripts/, you can then run:
>>
>>   guix run icecat icecat
>>
>> and similar.  This particular example doesn’t work well because of the
>> font issue you’re familiar with, but you get the idea.  :-)

I also realized that we can support:

  guix run icecat

which looks up ‘icecat’ in $PATH, and “readlink -f” to get at the
underlying store item.  This would be faster (and probably more
convenient) than “building” icecat as the script currently does.

> I sent a few patches moments ago that I've been sitting on for a
> bit.  My intent was originally to go further, but I ran out of
> time.  But I didn't think `guix environment' was the appropriate place
> to put such things---this script, though, is a good starting point for
> them.

Agreed.  I like the ideas in the patches you sent, but I’m not sure
‘guix environment’ is the right place for them.  It goes beyond the
typical job of ‘guix environment’.

> For example, if one of the dependencies of a program is X11, it can
> automatically share the X paths (unless overridden by the user).  Same
> with DBUS, sound devices, etc.  I mentioned previous ideas earlier in
> the thread.

Indeed, that’s a good idea.  It may be good enough to pattern-match the
store file names.

The attached version follows this suggestion:

--8<---------------cut here---------------start------------->8---
ludo@ribbon ~/src/guix$ ./pre-inst-env guix run xdpyinfo |head

name of display:    :0.0
version number:    11.0
vendor string:    The X.Org Foundation
vendor release number:    11906000
X.Org version: 1.19.6
maximum request size:  16777212 bytes
motion buffer size:  256
bitmap unit, bit order, padding:    32, LSBFirst, 32
image byte order:    LSBFirst
ludo@ribbon ~/src/guix$ ./pre-inst-env guix run ls -la .

total 0
drwxr-xr-x 2 0 0 40 Jan 29 16:40 .
drwxr-xr-x 3 0 0 60 Jan 29 16:40 ..
--8<---------------cut here---------------end--------------->8---

For some reason IceCat and Evince crash with a GDK error, whereas GIMP
and Xpdf are fine.

> I'd also want to integrate changes I made to `guix environment'.  If
> people here like the changes and they are merged, I'd want to refactor
> it into a common place, not just copy the code.

Yes, it’d be nice to have a module of utilities for environment
management.

>            ;; container script to invoke IceCat
>            (mkdir-p bin-dir)
>            (call-with-output-file (string-append bin-dir "icecat-container")
>              (lambda (port)
>                (format port "#!/bin/bash")))
>
>            ;; fontconfig configuration
>            (mkdir-p fc-dir)
>            (call-with-output-file fc-mtg
>              (lambda (port)
>                (format port (string-append "<?xml version=\"1.0\"?>
> <!DOCTYPE fontconfig SYSTEM \"fonts.dtd\">
> <fontconfig>
>   <dir>" (string-append (assoc-ref %build-inputs "font-dejavu")
>                         "/share/fonts") "</dir>
>   <cachedir>" fc-cache-dir "</cachedir>
> </fontconfig>\n"))))
>
>            (setenv "PATH"
>                    (string-append (assoc-ref %build-inputs "fontconfig")
>                                   "/bin"))
>            (setenv "FONTCONFIG_FILE" fc-mtg)
>            (setenv "XDG_DATA_HOME" share-dir)
>
>            (mkdir-p cache-dir)
>            (invoke "fc-cache" "-fv")))))

Nice!

Actually there are really two approaches we could use.  One is to create
wrappers like this one that do the right thing, independently of what
the user’s profile contains (‘guix package’ could even generate wrappers
automatically in some cases.)

The second approach is a ‘guix run/environment’ kind of command that
generates the environment at run time.

There are pros and cons to both, I think.

Food for thought!

Thanks,
Ludo’.

Re: Running IceCat in a container

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Actually there are really two approaches we could use.  One is to create
> wrappers like this one that do the right thing, independently of what
> the user’s profile contains (‘guix package’ could even generate wrappers
> automatically in some cases.)
>
> The second approach is a ‘guix run/environment’ kind of command that
> generates the environment at run time.
>
> There are pros and cons to both, I think.

This is just a tangent:

I’ve been thinking that “guix run” (or an extension of “guix container”)
would be great not only for running applications in containers that are
*already* in the store, but also to run applications from tarballs that
were generated with "guix pack“.

    pack=$(guix pack $(readlink -f $HOME/.guix-profile) -S /bin=bin)
    guix run --image=${pack} /bin/icecat

Look, we’ve got our own container image format! :) This seems to cover
85% of all uses of Docker/Singularity in the field of bioinformatics.

~~~

The setup to create configuration files and set environment variables so
that the target application feels at home in the container — that all
looks an awful lot like profile hooks to me.  Maybe we can have a set of
common hooks that we can automatically derive from package inputs?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Running code from packs in containers

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Actually there are really two approaches we could use.  One is to create
>> wrappers like this one that do the right thing, independently of what
>> the user’s profile contains (‘guix package’ could even generate wrappers
>> automatically in some cases.)
>>
>> The second approach is a ‘guix run/environment’ kind of command that
>> generates the environment at run time.
>>
>> There are pros and cons to both, I think.
>
> This is just a tangent:
>
> I’ve been thinking that “guix run” (or an extension of “guix container”)
> would be great not only for running applications in containers that are
> *already* in the store, but also to run applications from tarballs that
> were generated with "guix pack“.
>
>     pack=$(guix pack $(readlink -f $HOME/.guix-profile) -S /bin=bin)
>     guix run --image=${pack} /bin/icecat
>
> Look, we’ve got our own container image format! :) This seems to cover
> 85% of all uses of Docker/Singularity in the field of bioinformatics.

Indeed, very good point!

> The setup to create configuration files and set environment variables so
> that the target application feels at home in the container — that all
> looks an awful lot like profile hooks to me.  Maybe we can have a set of
> common hooks that we can automatically derive from package inputs?

Sounds like search paths no?  Or maybe a bit of both?

Anyway, it wouldn’t cost us much so to speak, so it’s worth looking into
that.

Ludo’.

Re: Running IceCat in a container

[ng0]

Ludovic Courtès transcribed 4.0K bytes:
> Heya,
> 
> Mike Gerwitz <mtg@gnu.org> skribis:
> 
> > On Thu, Jan 25, 2018 at 23:16:47 +0100, Ludovic Courtès wrote:
> >> If you drop the attached file under guix/scripts/, you can then run:
> >>
> >>   guix run icecat icecat
> >>
> >> and similar.  This particular example doesn’t work well because of the
> >> font issue you’re familiar with, but you get the idea.  :-)
> 
> I also realized that we can support:
> 
>   guix run icecat
> 
> which looks up ‘icecat’ in $PATH, and “readlink -f” to get at the
> underlying store item.  This would be faster (and probably more
> convenient) than “building” icecat as the script currently does.
> 
> > I sent a few patches moments ago that I've been sitting on for a
> > bit.  My intent was originally to go further, but I ran out of
> > time.  But I didn't think `guix environment' was the appropriate place
> > to put such things---this script, though, is a good starting point for
> > them.
> 
> Agreed.  I like the ideas in the patches you sent, but I’m not sure
> ‘guix environment’ is the right place for them.  It goes beyond the
> typical job of ‘guix environment’.
> 
> > For example, if one of the dependencies of a program is X11, it can
> > automatically share the X paths (unless overridden by the user).  Same
> > with DBUS, sound devices, etc.  I mentioned previous ideas earlier in
> > the thread.
> 
> Indeed, that’s a good idea.  It may be good enough to pattern-match the
> store file names.
> 
> The attached version follows this suggestion:
> 
> --8<---------------cut here---------------start------------->8---
> ludo@ribbon ~/src/guix$ ./pre-inst-env guix run xdpyinfo |head
> 
> name of display:    :0.0
> version number:    11.0
> vendor string:    The X.Org Foundation
> vendor release number:    11906000
> X.Org version: 1.19.6
> maximum request size:  16777212 bytes
> motion buffer size:  256
> bitmap unit, bit order, padding:    32, LSBFirst, 32
> image byte order:    LSBFirst
> ludo@ribbon ~/src/guix$ ./pre-inst-env guix run ls -la .
> 
> total 0
> drwxr-xr-x 2 0 0 40 Jan 29 16:40 .
> drwxr-xr-x 3 0 0 60 Jan 29 16:40 ..
> --8<---------------cut here---------------end--------------->8---
> 
> For some reason IceCat and Evince crash with a GDK error, whereas GIMP
> and Xpdf are fine.
> 
> > I'd also want to integrate changes I made to `guix environment'.  If
> > people here like the changes and they are merged, I'd want to refactor
> > it into a common place, not just copy the code.
> 
> Yes, it’d be nice to have a module of utilities for environment
> management.
> 
> >            ;; container script to invoke IceCat
> >            (mkdir-p bin-dir)
> >            (call-with-output-file (string-append bin-dir "icecat-container")
> >              (lambda (port)
> >                (format port "#!/bin/bash")))
> >
> >            ;; fontconfig configuration
> >            (mkdir-p fc-dir)
> >            (call-with-output-file fc-mtg
> >              (lambda (port)
> >                (format port (string-append "<?xml version=\"1.0\"?>
> > <!DOCTYPE fontconfig SYSTEM \"fonts.dtd\">
> > <fontconfig>
> >   <dir>" (string-append (assoc-ref %build-inputs "font-dejavu")
> >                         "/share/fonts") "</dir>
> >   <cachedir>" fc-cache-dir "</cachedir>
> > </fontconfig>\n"))))
> >
> >            (setenv "PATH"
> >                    (string-append (assoc-ref %build-inputs "fontconfig")
> >                                   "/bin"))
> >            (setenv "FONTCONFIG_FILE" fc-mtg)
> >            (setenv "XDG_DATA_HOME" share-dir)
> >
> >            (mkdir-p cache-dir)
> >            (invoke "fc-cache" "-fv")))))
> 
> Nice!
> 
> Actually there are really two approaches we could use.  One is to create
> wrappers like this one that do the right thing, independently of what
> the user’s profile contains (‘guix package’ could even generate wrappers
> automatically in some cases.)
> 
> The second approach is a ‘guix run/environment’ kind of command that
> generates the environment at run time.
> 
> There are pros and cons to both, I think.
>
> Food for thought!

Interesting ideas in this thread, I just tried your last version of guix run!

Couldn't we use both? For example, we can not predict where people will store their
mail and which applications they define in the config of the mailing application
to send it and receive it (case for environment) but we can predict roughly what
will be needed for w3m to be sandboxed.

> Thanks,
> Ludo’.
> 
> 

-- 
A88C8ADD129828D7EAC02E52E22F9BBFEE348588
https://n0.is

Re: Running IceCat in a container

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 37 bytes --]

And the attachment…

Ludo’.


[-- Attachment #2: the 'guix run' command --]
[-- Type: text/plain, Size: 5709 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix scripts run)
  #:use-module (guix ui)
  #:use-module (guix utils)
  #:use-module (guix scripts)
  #:use-module (guix store)
  #:use-module (guix packages)
  #:use-module (guix derivations)
  #:use-module ((guix build utils) #:select (which mkdir-p))
  #:use-module (gnu build linux-container)
  #:use-module (gnu system file-systems)
  #:use-module (gnu packages)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-11)
  #:use-module (srfi srfi-26)
  #:use-module (srfi srfi-37)
  #:use-module (ice-9 match)
  #:export (guix-run))

(define %options
  (list (option '(#\h "help") #f #f
                (lambda args
                  (show-help)
                  (exit 0)))
        (option '(#\V "version") #f #f
                (lambda args
                  (show-version-and-exit "guix run")))))

(define (show-help)
  (display (G_ "Usage: guix run PACKAGE COMMAND...
Run COMMAND from PACKAGE in a container.\n"))
  (newline)
  (display (G_ "
  -h, --help             display this help and exit"))
  (display (G_ "
  -V, --version          display version information and exit"))
  (newline)
  (show-bug-report-information))

\f

(define (bind-mount-spec/ro item)
  (and (file-exists? item)
       (file-system
         (device item)
         (mount-point item)
         (title 'device)
         (type "none")
         (flags '(bind-mount read-only))
         (check? #f))))

(define (bind-mount-spec/rw item)
  (and (file-exists? item)
       (file-system
         (inherit (bind-mount-spec/ro item))
         (flags '(bind-mount)))))

(define (application-file-system-mappings items)
  "Return the list of <file-system> objects corresponding to bind mounts
required by the applications whose dependencies are listed in ITEMS."
  (define packages
    (map (compose (cut package-name->name+version <> #\-)
                  store-path-package-name)
         items))

  (define x11? (member "libx11" packages))
  (define dbus? (member "dbus" packages))
  (define alsa? (member "alsa-lib" packages))
  (define pulseaudio? (member "pulseaudio" packages))

  (let-syntax ((if (syntax-rules ()
                     ((_ condition body)
                      (if condition (list body) '()))))
               (ro (identifier-syntax bind-mount-spec/ro))
               (rw (identifier-syntax bind-mount-spec/rw)))
    `(,(rw "/var/run/nscd/socket")
      ,@(if x11? (rw (string-append (getenv "HOME") "/.Xauthority")))
      ,@(if x11? (rw "/tmp/.X11-unix"))
      ,@(if dbus? (ro "/etc/machine-id"))
      ,@(if alsa? (rw "/dev/snd"))
      ,@(if pulseaudio? (rw (string-append (getenv "HOME") "/.pulse"))))))

(define %not-colon
  (char-set-complement (char-set #\:)))

(define (guix-run . args)
  (define (parse-options)
    ;; Return the alist of option values.  With this hack, the first
    ;; non-option argument is considered to be the beginning of the command.
    (let-values (((args command) (span (cut string-prefix? "-" <>) args)))
      (args-fold* args %options
                  (lambda (opt name arg result)
                    (leave (G_ "~A: unrecognized option~%") name))
                  (lambda (arg result)
                    (pk 'arg arg)
                    (alist-cons 'argument arg result))
                  '())
      command))

  (with-error-handling
    (match (parse-options)
      ((command args ...)
       (with-store store
         (let* ((full     (search-path (string-tokenize (getenv "PATH") %not-colon)
                                       command))
                (resolved (and=> full readlink*))
                (prefix   (and=> resolved (lambda (file)
                                            (and (store-path? file)
                                                 (direct-store-path file))))))
           (unless full
             (leave (G_ "command '~a' not found~%") command))
           (unless prefix
             (leave (G_ "command '~a' is not in '~a'~%")
                    command (%store-prefix)))

           (let ((items (requisites store (list prefix)))
                 (env   (environ)))

             (call-with-container
                 (append (map bind-mount-spec/ro items)
                         (application-file-system-mappings items))
               (lambda ()
                 (environ env)                    ;TODO: filter ENV
                 (mkdir-p (getenv "HOME"))
                 (chdir (getenv "HOME"))

                 (newline)
                 (catch #t
                   (lambda ()
                     (apply execl resolved command args))
                   (lambda (key . args)
                     (print-exception (current-error-port) #f key args)
                     (exit 1))))
               #:namespaces (delq 'net %namespaces)))))))))