bug#47106: Bubblewrap hates Guix containers 😞

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Hi Guix,

both Epiphany and Eolie (post fixing #47097; will submit patch shortly)
fail inside Guix containers with the suggested incantation.  After
getting the environment to no longer complain about $DISPLAY by adding
`--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly outputs
lines like
bwrap: Can't find source path /sys/class: No such file or directory
before closing the process altogether.

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Ludovic Courtès]

Hi!

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

> both Epiphany and Eolie (post fixing #47097; will submit patch shortly)
> fail inside Guix containers with the suggested incantation.  After
> getting the environment to no longer complain about $DISPLAY by adding
> `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly outputs
> lines like
> bwrap: Can't find source path /sys/class: No such file or directory
> before closing the process altogether.

What is ‘bwrap’ looking for?  /sys is mounted inside ‘guix environment -C’,
but perhaps it needs something special?

I suggest running these things (or ‘bwrap’ directly) in ‘strace -f -o
log’ inside the container to see.

Thanks,
Ludo’.

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Hi!
Am Samstag, den 13.03.2021, 11:48 +0100 schrieb Ludovic Courtès:
> Hi!
> 
> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> 
> > both Epiphany and Eolie (post fixing #47097; will submit patch
> > shortly)
> > fail inside Guix containers with the suggested incantation.  After
> > getting the environment to no longer complain about $DISPLAY by
> > adding
> > `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly
> > outputs
> > lines like
> > bwrap: Can't find source path /sys/class: No such file or directory
> > before closing the process altogether.
> 
> What is ‘bwrap’ looking for?  /sys is mounted inside ‘guix
> environment -C’,
> but perhaps it needs something special?
> 
> I suggest running these things (or ‘bwrap’ directly) in ‘strace -f -o
> log’ inside the container to see.
It seems to be 
> openat(AT_FDCWD, "/sys/class/dmi/id/chassis_type", O_RDONLY) = -1
> ENOENT (No such file or directory)
> openat(AT_FDCWD, "/sys/firmware/acpi/pm_profile", O_RDONLY) = -1
> ENOENT (No such file or directory)

I haven't repeated that for all warnings of similar kind, but if I add
`--expose=/sys/block --expose=/sys/class --expose=/sys/bus --
expose=/sys/dev --expose=/sys/devices` to the invocation, I instead get
a warning, that the WebKitWebProcess can't open $DISPLAY.  I'm not sure
how to resolve that one, given that I already had to sneak DISPLAY and
XAUTHORITY into the container, but it's a start.

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Bengt Richter]

Hi,

On +2021-03-13 12:07:51 +0100, Leo Prikler wrote:
> Hi!
> Am Samstag, den 13.03.2021, 11:48 +0100 schrieb Ludovic Courtès:
> > Hi!
> > 
> > Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> > 
> > > both Epiphany and Eolie (post fixing #47097; will submit patch
> > > shortly)
> > > fail inside Guix containers with the suggested incantation.  After
> > > getting the environment to no longer complain about $DISPLAY by
> > > adding
> > > `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly
> > > outputsn
> > > lines like
> > > bwrap: Can't find source path /sys/class: No such file or directory
> > > before closing the process altogether.
> > 
> > What is ‘bwrap’ looking for?  /sys is mounted inside ‘guix
> > environment -C’,
> > but perhaps it needs something special?
> > 
> > I suggest running these things (or ‘bwrap’ directly) in ‘strace -f -o
> > log’ inside the container to see.
> It seems to be 
> > openat(AT_FDCWD, "/sys/class/dmi/id/chassis_type", O_RDONLY) = -1
> > ENOENT (No such file or directory)
> > openat(AT_FDCWD, "/sys/firmware/acpi/pm_profile", O_RDONLY) = -1
> > ENOENT (No such file or directory)
> 
> I haven't repeated that for all warnings of similar kind, but if I add
> `--expose=/sys/block --expose=/sys/class --expose=/sys/bus --
> expose=/sys/dev --expose=/sys/devices` to the invocation, I instead get
> a warning, that the WebKitWebProcess can't open $DISPLAY.  I'm not sure
> how to resolve that one, given that I already had to sneak DISPLAY and
> XAUTHORITY into the container, but it's a start.
> 
> Regards,
> Leo
> 
Does $DISPLAY mean ":0" and does the Wayland server answer that with
its XWayland X-interface?

I am wonderering how that is resolved inside a container.

HTH with the detective work ;)

-- 
Regards,
Bengt Richter

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Am Samstag, den 13.03.2021, 13:27 +0100 schrieb Bengt Richter:
> Hi,
> 
> On +2021-03-13 12:07:51 +0100, Leo Prikler wrote:
> > Hi!
> > Am Samstag, den 13.03.2021, 11:48 +0100 schrieb Ludovic Courtès:
> > > Hi!
> > > 
> > > Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> > > 
> > > > both Epiphany and Eolie (post fixing #47097; will submit patch
> > > > shortly)
> > > > fail inside Guix containers with the suggested
> > > > incantation.  After
> > > > getting the environment to no longer complain about $DISPLAY by
> > > > adding
> > > > `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly
> > > > outputsn
> > > > lines like
> > > > bwrap: Can't find source path /sys/class: No such file or
> > > > directory
> > > > before closing the process altogether.
> > > 
> > > What is ‘bwrap’ looking for?  /sys is mounted inside ‘guix
> > > environment -C’,
> > > but perhaps it needs something special?
> > > 
> > > I suggest running these things (or ‘bwrap’ directly) in ‘strace
> > > -f -o
> > > log’ inside the container to see.
> > It seems to be 
> > > openat(AT_FDCWD, "/sys/class/dmi/id/chassis_type", O_RDONLY) = -1
> > > ENOENT (No such file or directory)
> > > openat(AT_FDCWD, "/sys/firmware/acpi/pm_profile", O_RDONLY) = -1
> > > ENOENT (No such file or directory)
> > 
> > I haven't repeated that for all warnings of similar kind, but if I
> > add
> > `--expose=/sys/block --expose=/sys/class --expose=/sys/bus --
> > expose=/sys/dev --expose=/sys/devices` to the invocation, I instead
> > get
> > a warning, that the WebKitWebProcess can't open $DISPLAY.  I'm not
> > sure
> > how to resolve that one, given that I already had to sneak DISPLAY
> > and
> > XAUTHORITY into the container, but it's a start.
> > 
> > Regards,
> > Leo
> > 
> Does $DISPLAY mean ":0" and does the Wayland server answer that with
> its XWayland X-interface?
In my setup $DISPLAY=:1, but obviously the exact value depends on other
circumstances (i.e. if there's already an open session belonging to
another user it'd be :2, :3, ...).  I'm not sure how X vs. Wayland
plays out here, but I'm still using Gnome on X, so that should
hopefully not be an issue here.

> I am wonderering how that is resolved inside a container.
Well, for X you'd usually preserve DISPLAY and XAUTHORITY and also
expose $XAUTHORITY or something along those lines.  Not sure how you
Wayland folk do that.

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Bengt Richter]

Hi again,

On +2021-03-13 15:43:15 +0100, Leo Prikler wrote:
> Am Samstag, den 13.03.2021, 13:27 +0100 schrieb Bengt Richter:
> > Hi,
> > 
> > On +2021-03-13 12:07:51 +0100, Leo Prikler wrote:
> > > Hi!
> > > Am Samstag, den 13.03.2021, 11:48 +0100 schrieb Ludovic Courtès:
> > > > Hi!
> > > > 
> > > > Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> > > > 
> > > > > both Epiphany and Eolie (post fixing #47097; will submit patch
> > > > > shortly)
> > > > > fail inside Guix containers with the suggested
> > > > > incantation.  After
> > > > > getting the environment to no longer complain about $DISPLAY by
> > > > > adding
> > > > > `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly
> > > > > outputsn
> > > > > lines like
> > > > > bwrap: Can't find source path /sys/class: No such file or
> > > > > directory
> > > > > before closing the process altogether.
> > > > 
> > > > What is ‘bwrap’ looking for?  /sys is mounted inside ‘guix
> > > > environment -C’,
> > > > but perhaps it needs something special?
> > > > 
> > > > I suggest running these things (or ‘bwrap’ directly) in ‘strace
> > > > -f -o
> > > > log’ inside the container to see.
> > > It seems to be 
> > > > openat(AT_FDCWD, "/sys/class/dmi/id/chassis_type", O_RDONLY) = -1
> > > > ENOENT (No such file or directory)
> > > > openat(AT_FDCWD, "/sys/firmware/acpi/pm_profile", O_RDONLY) = -1
> > > > ENOENT (No such file or directory)
> > > 
> > > I haven't repeated that for all warnings of similar kind, but if I
> > > add
> > > `--expose=/sys/block --expose=/sys/class --expose=/sys/bus --
> > > expose=/sys/dev --expose=/sys/devices` to the invocation, I instead
> > > get
> > > a warning, that the WebKitWebProcess can't open $DISPLAY.  I'm not
> > > sure
> > > how to resolve that one, given that I already had to sneak DISPLAY
> > > and
> > > XAUTHORITY into the container, but it's a start.
> > > 
> > > Regards,
> > > Leo
> > > 
> > Does $DISPLAY mean ":0" and does the Wayland server answer that with
> > its XWayland X-interface?
> In my setup $DISPLAY=:1, but obviously the exact value depends on other
> circumstances (i.e. if there's already an open session belonging to
> another user it'd be :2, :3, ...).  I'm not sure how X vs. Wayland
> plays out here, but I'm still using Gnome on X, so that should
> hopefully not be an issue here.
> 
> > I am wonderering how that is resolved inside a container.
> Well, for X you'd usually preserve DISPLAY and XAUTHORITY and also
> expose $XAUTHORITY or something along those lines.  Not sure how you
> Wayland folk do that.
>

I am not a Wayland developer, if that's what you mean by "Wayland folk" :)
But I have been experimenting with writing my own text and graphics widget,
poking 32-bit pixels into buffers for display by the Wayland compositor,
so I've learned a little :) I am using the linux kernel's sun12x22 bitmap
font to do text at a low level.

Idk much about containers yet, but I imagine using lowlevel stuff to
make images for a trivial web server running in a container could be interesting.

> Regards,
> Leo
> 

I am curious what the commands below would show inside your container.
"pidparents" [1] is a little script I find handy, which would have to be
accessible in your container of course. Idk how you put local bash scripts
in your container. I assume it's possible :)

I did these commands in a debian gnome terminal window, where pidparents[1]
showed (timetagged later, since I just went back to do that) this context:
--8<---------------cut here---------------start------------->8---
[17:29 ~/bs]$ pidparents
pidparents      pts/1     5800 S+   /usr/bin/bash /home/bokr/bin/pidparents
bash            pts/1     5711 Ss   /bin/bash
tilix           ?         2007 Sl   /usr/bin/tilix --gapplication-service
systemd         ?         1308 Ss   /lib/systemd/systemd --user
systemd         ?            1 Ss   /sbin/init splash
--8<---------------cut here---------------end--------------->8---

;;;; First I just look for processes with X11 or way in their names, 
;;;; then I use pidparents to see how they are started.

[17:18 ~/bs]$ ps af|egrep -i 'x11|way'
 5741 pts/1    S+     0:00  \_ grep -E -i x11|way
 1329 tty2     Ssl+   0:00 /usr/lib/gdm3/gdm-wayland-session /usr/bin/gnome-session
 1433 tty2     Sl+    0:13      |   \_ /usr/bin/Xwayland :0 -rootless -terminate -accessx -core -listen 4 -listen 5 -displayfd 6
 1468 tty2     Sl     0:00 /usr/lib/ibus/ibus-x11 --kill-daemon

[17:21 ~/bs]$ pidparents 1329
gdm-wayland-ses tty2      1329 Ssl+ /usr/lib/gdm3/gdm-wayland-session /usr/bin/gnome-session
gdm-session-wor ?         1304 Sl   gdm-session-worker [pam/gdm-password]
gdm3            ?          711 Ssl  /usr/sbin/gdm3
systemd         ?            1 Ss   /sbin/init splash

;;;; this one might be the most interesting in your container
;;;; can you SSH into it to do these things?
[17:22 ~/bs]$ pidparents 1433
Xwayland        tty2      1433 Sl+  /usr/bin/Xwayland :0 -rootless -terminate -accessx -core -listen 4 -listen 5 -displayfd 6
gnome-shell     tty2      1408 Rl+  /usr/bin/gnome-shell
gnome-session-b tty2      1333 Sl+  /usr/lib/gnome-session/gnome-session-binary
gdm-wayland-ses tty2      1329 Ssl+ /usr/lib/gdm3/gdm-wayland-session /usr/bin/gnome-session
gdm-session-wor ?         1304 Sl   gdm-session-worker [pam/gdm-password]
gdm3            ?          711 Ssl  /usr/sbin/gdm3
systemd         ?            1 Ss   /sbin/init splash

;;;; not really sure what this one does
[17:22 ~/bs]$ pidparents 1468
ibus-x11        tty2      1468 Sl   /usr/lib/ibus/ibus-x11 --kill-daemon
systemd         ?            1 Ss   /sbin/init splash

;;;; [1] here is pidparents -- pretty short, so you could manually
;;;; enter it if necessary :)

[17:22 ~/bs]$ cat $(which pidparents)|gxsnip 
--8<---------------cut here---------------start------------->8---
#!/usr/bin/bash
# ~/bin/pidparents

pid=${1:-$$}	#this process if no pid specified as $1

while [ $(($pid)) -gt 0 ]; do    
      ps h -p $pid -o comm,tt,pid,stat,args
      pid=$(ps -q $pid -o ppid=)
done

--8<---------------cut here---------------end--------------->8---
[17:24 ~/bs]$ 

Hopefully this would reveal a little more about what $DISPLAY means in your container.

WDYT?

-- 
Regards,
Bengt Richter

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Am Samstag, den 13.03.2021, 18:07 +0100 schrieb Bengt Richter:
> I am not a Wayland developer, if that's what you mean by "Wayland
> folk" :)
I meant it as "folk using Wayland in their display manager".

> I am curious what the commands below would show inside your
> container.
> "pidparents" [1] is a little script I find handy, which would have to
> be
> accessible in your container of course. Idk how you put local bash
> scripts
> in your container. I assume it's possible :)
Far from getting a script into my container, I can't even really get
into my container through means like `guix container exec`, so I simply
bloated it with screen and pstree.  The result:

--8<---------------cut here---------------start------------->8---
sh-+-dbus-daemon
   |-dbus-launch
   `-screen---screen-+-sh---.epiphany-real-+-WebKitNetworkPr---
11*[{WebKitNetworkPr}]
                     |                     |-bwrap---bwrap---
WebKitWebProces
                     |                     `-18*[{.epiphany-real}]
                     `-sh---pstree
--8<---------------cut here---------------end--------------->8---

I think these processes are created and die too quickly for me to
reliably extract PIDs.

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Bengt Richter]

Hi again^2,

On +2021-03-13 19:01:29 +0100, Leo Prikler wrote:
> Am Samstag, den 13.03.2021, 18:07 +0100 schrieb Bengt Richter:
> > I am not a Wayland developer, if that's what you mean by "Wayland
> > folk" :)
> I meant it as "folk using Wayland in their display manager".
> 
> > I am curious what the commands below would show inside your
> > container.
> > "pidparents" [1] is a little script I find handy, which would have to
> > be
> > accessible in your container of course. Idk how you put local bash
> > scripts
> > in your container. I assume it's possible :)
> Far from getting a script into my container, I can't even really get
> into my container through means like `guix container exec`, so I simply
> bloated it with screen and pstree.  The result:
> 
> --8<---------------cut here---------------start------------->8---
> sh-+-dbus-daemon
>    |-dbus-launch
>    `-screen---screen-+-sh---.epiphany-real-+-WebKitNetworkPr---
> 11*[{WebKitNetworkPr}]
>                      |                     |-bwrap---bwrap---
> WebKitWebProces
>                      |                     `-18*[{.epiphany-real}]
>                      `-sh---pstree
> --8<---------------cut here---------------end--------------->8---
> 
> I think these processes are created and die too quickly for me to
> reliably extract PIDs.
> 
> Regards,
> Leo
> 

Maybe
    pstree -at
would show a little more?
Also,
    ls -lr /sys/class/drm
if that's accessible -- I'm wondering if the version of screen
in the container is built with libdrm and is bypassing X or ??

Do you have a makefile or a guix something.scm defining
what's built/packed into your container? 

Sorry if my curiosity is making work for you, but I'd like to
try containers down the road -- tho right now I'm taking a break
from events IRL, so I may disappear for a while...

-- 
Regards,
Bengt Richter

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Hi again³

Am Sonntag, den 14.03.2021, 18:45 +0100 schrieb Bengt Richter:
> Hi again^2,
> 
> Maybe
>     pstree -at
> would show a little more?
sh
  |-dbus-daemon --syslog-only --fork --print-pid 5 --print-address 7
--sess
  |-dbus-launch --autolaunch=fa7a4d52637958ddd37547bb5d8bd9d2--binary-
synt
  `-screen
      `-screen
          |-sh
          |   `-.epiphany-real
          |       |-WebKitNetworkPr 3 21
          |       |   |-{BMScavenger}
          |       |   |-{ReceiveQueue}
          |       |   |-{StorageTask}
          |       |   |-{Storage}
          |       |   |-{WebStorage}
          |       |   |-{background}
          |       |   |-{dconf worker}
          |       |   |-{erialBackground}
          |       |   |-{gdbus}
          |       |   `-{gmain}
          |       |-bwrap --args 37 --
/gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
          |       |   `-bwrap --args 37 --
/gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
          |       |       `-WebKitWebProces 1277 28
          |       |-{.epiphany-real}
          |       |-{BMScavenger}
          |       |-{HashSaltStorage}
          |       |-{IconDatabase}
          |       |-{PressureMonitor}
          |       |-2*[{ReceiveQueue}]
          |       |-{dconf worker}
          |       |-{e Compile Queue}
          |       |-{ebsiteDataStore}
          |       |-{gdbus}
          |       |-{gmain}
          |       |-{re Remove Queue}
          |       `-{tore Read Queue}
          `-sh
              `-pstree -at
> Also,
>     ls -lr /sys/class/drm
total 0
-r--r--r-- 1 65534 overflow 4096 Mar 14 17:59 version
lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 ttm ->
../../devices/virtual/drm/ttm
lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 renderD128 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/renderD128
lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 card0-VGA-1 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-VGA-
1
lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 card0-HDMI-A-1 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
HDMI-A-1
lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 card0-DVI-D-1 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-DVI-
D-1
lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 card0 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0
> if that's accessible -- I'm wondering if the version of screen
> in the container is built with libdrm and is bypassing X or ??
I doubt it is being built differently than screen normally is.

> Do you have a makefile or a guix something.scm defining
> what's built/packed into your container? 
Nah, it's a rather ad-hoc definition grown from what should be an Eolie
container from the cookbook (also refer to #47097).

    guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
     --preserve=TERM \
     --expose=$XAUTHORITY \
     --expose=/etc/machine-id \
     --expose=/etc/ssl/certs/ \
     --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
     --expose=/sys/dev --expose=/sys/devices \
     --ad-hoc epiphany nss-certs dbus procps coreutils psmisc screen

Given that I expose most of /sys explicitly, you should take the above
with a grain of salt.

> Sorry if my curiosity is making work for you, but I'd like to
> try containers down the road -- tho right now I'm taking a break
> from events IRL, so I may disappear for a while...
I'm not personally impacted by this bug or anything, it's much rather a
follow-up to my attempted fix of #47097.  I think there might be some
flaw in trying to run a sandbox inside a sandbox (like bubblewrap
inside `guix container`), that doesn't actually improve security in any
meaningful way.

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Ludovic Courtès]

Hi Leo,

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

> Nah, it's a rather ad-hoc definition grown from what should be an Eolie
> container from the cookbook (also refer to #47097).
>
>     guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
>      --preserve=TERM \
>      --expose=$XAUTHORITY \
>      --expose=/etc/machine-id \
>      --expose=/etc/ssl/certs/ \
>      --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
>      --expose=/sys/dev --expose=/sys/devices \
>      --ad-hoc epiphany nss-certs dbus procps coreutils psmisc screen

I’m not sure I follow; does it work when you do this?

/sys is already mounted inside ‘guix environment -C’ containers so I
don’t see what difference it would make.

But wait, the example above lacks ‘-C’; a mistake?

Thanks,
Ludo’.

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Am Sonntag, den 14.03.2021, 21:32 +0100 schrieb Ludovic Courtès:
> Hi Leo,
> 
> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> 
> > Nah, it's a rather ad-hoc definition grown from what should be an
> > Eolie
> > container from the cookbook (also refer to #47097).
> > 
> >     guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
> >      --preserve=TERM \
> >      --expose=$XAUTHORITY \
> >      --expose=/etc/machine-id \
> >      --expose=/etc/ssl/certs/ \
> >      --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
> >      --expose=/sys/dev --expose=/sys/devices \
> >      --ad-hoc epiphany nss-certs dbus procps coreutils psmisc
> > screen
> 
> I’m not sure I follow; does it work when you do this?
It does work insofar as I don't get any warnings about resources
missing from /sys, but the bubblewrapped WebKit processes don't have
access to $DISPLAY even though epiphany itself has.  While they don't
crash the browser itself and just infinitely respawn, that's still far
from usable.

> /sys is already mounted inside ‘guix environment -C’ containers so I
> don’t see what difference it would make.
I think I've been told this several times, but I don't believe it.  Not
adding all these expose=/sys lines triggers the "warnings" in the
original post.  (Okay, perhaps one of /sys/dev and /sys/devices is
superfluous, I would need to check.)

> But wait, the example above lacks ‘-C’; a mistake?
Indeed, -CN should also be given, but I hastily edited the command line
inside the email to make it appear more beautiful than it actually is,
thereby deleting it.  I'm sorry.  The preserves and exposes should be
the same list as I'm actually using however.

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Ludovic Courtès]

Hi Leo,

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

>> /sys is already mounted inside ‘guix environment -C’ containers so I
>> don’t see what difference it would make.
> I think I've been told this several times, but I don't believe it.  Not
> adding all these expose=/sys lines triggers the "warnings" in the
> original post.  (Okay, perhaps one of /sys/dev and /sys/devices is
> superfluous, I would need to check.)

It would be great if you could pinpoint which of these --expose=/sys/xyz
makes a difference.  From there we could compare the output of ‘find
/sys/xyz’ inside and outside the container, without --expose.

Thanks,
Ludo’.

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Am Montag, den 15.03.2021, 10:52 +0100 schrieb Ludovic Courtès:
> Hi Leo,
> 
> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> 
> > > /sys is already mounted inside ‘guix environment -C’ containers
> > > so I
> > > don’t see what difference it would make.
> > I think I've been told this several times, but I don't believe
> > it.  Not
> > adding all these expose=/sys lines triggers the "warnings" in the
> > original post.  (Okay, perhaps one of /sys/dev and /sys/devices is
> > superfluous, I would need to check.)
> 
> It would be great if you could pinpoint which of these --
> expose=/sys/xyz
> makes a difference.  From there we could compare the output of ‘find
> /sys/xyz’ inside and outside the container, without --expose.
Okay, so here's my basic workflow: Starting with an empty set of --
expose:
bwrap: Can't find source path /sys/block: No such file or directory
repeated a few times along with warnings, that the web process crashed
until finally Epiphany itself crashes. 
I add /sys/block, and Epiphany miraculously doesn't crash, but bwrap
still complains and the web processes still crash, so we march on. 
I'll abbreviate it a little and only show the error messages.
bwrap: Can't find source path /sys/bus: No such file or directory
bwrap: Can't find source path /sys/class: No such file or directory 
bwrap: Can't find source path /sys/dev: No such file or directory 
bwrap: Can't find source path /sys/devices: No such file or directory

After exposing all of the above, I get
Unable to init server: Could not connect: Connection refused

(WebKitWebProcess:2): Gtk-WARNING **: 10:09:01.497: cannot open
display: :1
Using --share instead of --expose for the final set does not seem to
change anything.

For /sys/block, the find inside the container is empty before exposing
it and non-empty on the host.  I assume the same holds for the others.

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Ludovic Courtès]

Hi,

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

> Okay, so here's my basic workflow: Starting with an empty set of --
> expose:
> bwrap: Can't find source path /sys/block: No such file or directory
> repeated a few times along with warnings, that the web process crashed
> until finally Epiphany itself crashes. 
> I add /sys/block, and Epiphany miraculously doesn't crash, but bwrap
> still complains and the web processes still crash, so we march on. 
> I'll abbreviate it a little and only show the error messages.
> bwrap: Can't find source path /sys/bus: No such file or directory
> bwrap: Can't find source path /sys/class: No such file or directory 
> bwrap: Can't find source path /sys/dev: No such file or directory 
> bwrap: Can't find source path /sys/devices: No such file or directory

OK.  Additional data points from my Guix System laptop:

--8<---------------cut here---------------start------------->8---
$ guix environment -C --ad-hoc findutils -- find /sys |wc -l
37575
$ find /sys | wc -l
find: ‘/sys/kernel/debug’: Mankas permeso
38026
$ guix environment -C --ad-hoc findutils -- find /sys/block |wc -l
27
$ find /sys/block | wc -l
27
--8<---------------cut here---------------end--------------->8---

The offending Bubblewrap code is:

--8<---------------cut here---------------start------------->8---
static void
resolve_symlinks_in_ops (void)
{
  SetupOp *op;

  for (op = ops; op != NULL; op = op->next)
    {
      const char *old_source;

      switch (op->type)
        {
        case SETUP_RO_BIND_MOUNT:
        case SETUP_DEV_BIND_MOUNT:
        case SETUP_BIND_MOUNT:
          old_source = op->source;
          op->source = realpath (old_source, NULL);
          if (op->source == NULL)
            {
              if (op->flags & ALLOW_NOTEXIST && errno == ENOENT)
                op->source = old_source;
              else
                die_with_error("Can't find source path %s", old_source);
            }
          break;
        default:
          break;
        }
    }
}
--8<---------------cut here---------------end--------------->8---

I wonder how ‘realpath’ can fail here.  In fact, it Works For Me:

--8<---------------cut here---------------start------------->8---
$ cat realpath.scm
(use-modules (system foreign))

(define realpath
  (let ((proc (pointer->procedure '* (dynamic-func "realpath" (dynamic-link))
                                  '(* *))))
    (lambda (path)
      (let ((result (proc (string->pointer path) %null-pointer)))
        (and (not (null-pointer? result))
             (pointer->string result))))))

(pk 'realpath-> (realpath "/sys/block"))
$ guix environment -C --ad-hoc guile -- guile -s realpath.scm
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/ludo/src/guix-debugging/realpath.scm
;;; compiled /home/ludo/.cache/guile/ccache/3.0-LE-8-4.4/home/ludo/src/guix-debugging/realpath.scm.go

;;; (realpath-> "/sys/block")
--8<---------------cut here---------------end--------------->8---

So I presume bwrap does something else before it reaches that warning.
We should really strace it.

Thanks,
Ludo’.

bug#47106: Bubblewrap hates Guix containers 😞

[Bengt Richter]

Hi Leo,
One more favor? ;)

On +2021-03-14 19:05:24 +0100, Leo Prikler wrote:
> Hi again³
> 
> Am Sonntag, den 14.03.2021, 18:45 +0100 schrieb Bengt Richter:
> > Hi again^2,
> > 
> > Maybe
> >     pstree -at
> > would show a little more?
> sh
>   |-dbus-daemon --syslog-only --fork --print-pid 5 --print-address 7
> --sess
>   |-dbus-launch --autolaunch=fa7a4d52637958ddd37547bb5d8bd9d2--binary-
> synt
>   `-screen
>       `-screen
>           |-sh
>           |   `-.epiphany-real
>           |       |-WebKitNetworkPr 3 21
>           |       |   |-{BMScavenger}
>           |       |   |-{ReceiveQueue}
>           |       |   |-{StorageTask}
>           |       |   |-{Storage}
>           |       |   |-{WebStorage}
>           |       |   |-{background}
>           |       |   |-{dconf worker}
>           |       |   |-{erialBackground}
>           |       |   |-{gdbus}
>           |       |   `-{gmain}
>           |       |-bwrap --args 37 --
> /gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
>           |       |   `-bwrap --args 37 --
> /gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
>           |       |       `-WebKitWebProces 1277 28
>           |       |-{.epiphany-real}
>           |       |-{BMScavenger}
>           |       |-{HashSaltStorage}
>           |       |-{IconDatabase}
>           |       |-{PressureMonitor}
>           |       |-2*[{ReceiveQueue}]
>           |       |-{dconf worker}
>           |       |-{e Compile Queue}
>           |       |-{ebsiteDataStore}
>           |       |-{gdbus}
>           |       |-{gmain}
>           |       |-{re Remove Queue}
>           |       `-{tore Read Queue}
>           `-sh
>               `-pstree -at
> > Also,
> >     ls -lr /sys/class/drm
> total 0
> -r--r--r-- 1 65534 overflow 4096 Mar 14 17:59 version
> lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 ttm ->
> ../../devices/virtual/drm/ttm
> lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 renderD128 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/renderD128
> lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 card0-VGA-1 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-VGA-
> 1
> lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 card0-HDMI-A-1 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
> HDMI-A-1
> lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 card0-DVI-D-1 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-DVI-
> D-1
> lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 card0 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0
> > if that's accessible -- I'm wondering if the version of screen
> > in the container is built with libdrm and is bypassing X or ??
> I doubt it is being built differently than screen normally is.
> 
> > Do you have a makefile or a guix something.scm defining
> > what's built/packed into your container? 
> Nah, it's a rather ad-hoc definition grown from what should be an Eolie
> container from the cookbook (also refer to #47097).
> 
>     guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
>      --preserve=TERM \
>      --expose=$XAUTHORITY \
>      --expose=/etc/machine-id \
>      --expose=/etc/ssl/certs/ \
>      --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
>      --expose=/sys/dev --expose=/sys/devices \
>      --ad-hoc epiphany nss-certs dbus procps coreutils psmisc screen
> 
> Given that I expose most of /sys explicitly, you should take the above
> with a grain of salt.
> 
> > Sorry if my curiosity is making work for you, but I'd like to
> > try containers down the road -- tho right now I'm taking a break
> > from events IRL, so I may disappear for a while...
> I'm not personally impacted by this bug or anything, it's much rather a
> follow-up to my attempted fix of #47097.  I think there might be some
> flaw in trying to run a sandbox inside a sandbox (like bubblewrap
> inside `guix container`), that doesn't actually improve security in any
> meaningful way.
> 
> Regards,
> Leo
> 

If you can run this inside your container, I think it will be interesting:
    lsof -U|grep -i wayland

The above ought to show quickly if wayland is running.

lsof -U shows the open sockets.

If the above shows nothing, try
    lsof -U|grep -i x11
or    
    lsof -U|grep X

finally, it is interesting to see
    lsof -U|less

but on my laptop I just got
    lsof -U|wc
        403    3760   34643

so its a lot to look at.
Hopefully less in a container ;)

-- 
Regards,
Bengt Richter

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Hi,
Am Dienstag, den 16.03.2021, 11:54 +0100 schrieb Bengt Richter:
> Hi Leo,
> One more favor? ;)
> 
> On +2021-03-14 19:05:24 +0100, Leo Prikler wrote:
> > Hi again³
> > 
> > Am Sonntag, den 14.03.2021, 18:45 +0100 schrieb Bengt Richter:
> > > Hi again^2,
> > > 
> > > Maybe
> > >     pstree -at
> > > would show a little more?
> > sh
> >   |-dbus-daemon --syslog-only --fork --print-pid 5 --print-address
> > 7
> > --sess
> >   |-dbus-launch --autolaunch=fa7a4d52637958ddd37547bb5d8bd9d2
> > --binary-
> > synt
> >   `-screen
> >       `-screen
> >           |-sh
> >           |   `-.epiphany-real
> >           |       |-WebKitNetworkPr 3 21
> >           |       |   |-{BMScavenger}
> >           |       |   |-{ReceiveQueue}
> >           |       |   |-{StorageTask}
> >           |       |   |-{Storage}
> >           |       |   |-{WebStorage}
> >           |       |   |-{background}
> >           |       |   |-{dconf worker}
> >           |       |   |-{erialBackground}
> >           |       |   |-{gdbus}
> >           |       |   `-{gmain}
> >           |       |-bwrap --args 37 --
> > /gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
> >           |       |   `-bwrap --args 37 --
> > /gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
> >           |       |       `-WebKitWebProces 1277 28
> >           |       |-{.epiphany-real}
> >           |       |-{BMScavenger}
> >           |       |-{HashSaltStorage}
> >           |       |-{IconDatabase}
> >           |       |-{PressureMonitor}
> >           |       |-2*[{ReceiveQueue}]
> >           |       |-{dconf worker}
> >           |       |-{e Compile Queue}
> >           |       |-{ebsiteDataStore}
> >           |       |-{gdbus}
> >           |       |-{gmain}
> >           |       |-{re Remove Queue}
> >           |       `-{tore Read Queue}
> >           `-sh
> >               `-pstree -at
> > > Also,
> > >     ls -lr /sys/class/drm
> > total 0
> > -r--r--r-- 1 65534 overflow 4096 Mar 14 17:59 version
> > lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 ttm ->
> > ../../devices/virtual/drm/ttm
> > lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 renderD128 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/renderD128
> > lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 card0-VGA-1 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
> > VGA-
> > 1
> > lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:59 card0-HDMI-A-1 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
> > HDMI-A-1
> > lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 card0-DVI-D-1 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
> > DVI-
> > D-1
> > lrwxrwxrwx 1 65534 overflow    0 Mar 14 17:58 card0 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0
> > > if that's accessible -- I'm wondering if the version of screen
> > > in the container is built with libdrm and is bypassing X or ??
> > I doubt it is being built differently than screen normally is.
> > 
> > > Do you have a makefile or a guix something.scm defining
> > > what's built/packed into your container? 
> > Nah, it's a rather ad-hoc definition grown from what should be an
> > Eolie
> > container from the cookbook (also refer to #47097).
> > 
> >     guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
> >      --preserve=TERM \
> >      --expose=$XAUTHORITY \
> >      --expose=/etc/machine-id \
> >      --expose=/etc/ssl/certs/ \
> >      --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
> >      --expose=/sys/dev --expose=/sys/devices \
> >      --ad-hoc epiphany nss-certs dbus procps coreutils psmisc
> > screen
> > 
> > Given that I expose most of /sys explicitly, you should take the
> > above
> > with a grain of salt.
> > 
> > > Sorry if my curiosity is making work for you, but I'd like to
> > > try containers down the road -- tho right now I'm taking a break
> > > from events IRL, so I may disappear for a while...
> > I'm not personally impacted by this bug or anything, it's much
> > rather a
> > follow-up to my attempted fix of #47097.  I think there might be
> > some
> > flaw in trying to run a sandbox inside a sandbox (like bubblewrap
> > inside `guix container`), that doesn't actually improve security in
> > any
> > meaningful way.
> > 
> > Regards,
> > Leo
> > 
> 
> If you can run this inside your container, I think it will be
> interesting:
>     lsof -U|grep -i wayland
> 
> The above ought to show quickly if wayland is running.
> 
> lsof -U shows the open sockets.
> 
> If the above shows nothing, try
>     lsof -U|grep -i x11
> or    
>     lsof -U|grep X
Nothing showed up for either, but this got me thinking.  Exposing
/tmp/.X11-unix/X1 did do away with the warning, now it's unexposed
dbus, missing icons, etc. etc.  Exposing all of /tmp instead yields 

** (epiphany:2): ERROR **: 11:11:28.855: Failed to start embed shell D-
Bus server on unix:dir=(null): Error binding to address: No such file
or directory

I still think that exposing all of that is perhaps not the wisest idea,
but eh…

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Famulari]

This bug is registered as "blocking" the upcoming release.

Is there any chance it will be closed in the next 3 days? Or should I
"unblock"?

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Prikler]

Am Mittwoch, den 14.04.2021, 16:07 -0400 schrieb Leo Famulari:
> This bug is registered as "blocking" the upcoming release.
> 
> Is there any chance it will be closed in the next 3 days? Or should I
> "unblock"?
There is a slim chance, that someone might fix it "by accident", but
I'm personally not diving that deep into bubblewrap, so I don't even
have an estimate of how much we would need to change.  I think
unblocking is the wiser choice here; we can update the manual post
string freeze.

Regards,
Leo

bug#47106: Bubblewrap hates Guix containers 😞

[Leo Famulari]

On Wed, Apr 14, 2021 at 11:23:28PM +0200, Leo Prikler wrote:
> Am Mittwoch, den 14.04.2021, 16:07 -0400 schrieb Leo Famulari:
> > This bug is registered as "blocking" the upcoming release.
> > 
> > Is there any chance it will be closed in the next 3 days? Or should I
> > "unblock"?
> There is a slim chance, that someone might fix it "by accident", but
> I'm personally not diving that deep into bubblewrap, so I don't even
> have an estimate of how much we would need to change.  I think
> unblocking is the wiser choice here; we can update the manual post
> string freeze.

Alright, thanks for explaining. I've done the "unblocking".