guixsd in lxd container

guixsd in lxd container

[Eddy Pronk]

[-- Attachment #1: Type: text/plain, Size: 4847 bytes --]

Hello guix!

I'm trying to run guixsd in an lxd container.
My lxd containers run on an Ubuntu server 16.04.

I took the usb-installer image and imported in as an lxd image.

When a container start it runs /sbin/init.
In guixsd /proc/1 is shepherd, but a lot of stuff happens before shepherd
is started.

I've set a few things in the environment matching values in /proc/1/environ.
The argument of --load in grub.cfg is a guile program.

I found some details about the kernel loading guile here:
https://lists.gnu.org/archive/html/guix-devel/2016-12/msg00704.html

To be able to get some logging during startup of guixsd I'm trying to run
it from a shell script as root.

====
cat /sbin/start
export HOME=/
export TERM=linux
export
BOOT_IMAGE="/gnu/store/fqc2kg4lq1lz1ymk41080jzb5q90icg0-linux-libre-4.11/bzImage
--root=gnu-disk-image
--system=/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system
--load=/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system/boot"
export PATH=/gnu/store/crvb68g89b479n4h44r8l42hy39axhg2-shadow-4.4/sbin/
cd $HOME
/gnu/store/sa7zrdfqglnb5rvvr11qdj0rspbs292v-profile/bin/ln -s
/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system /run/current-system
/gnu/store/zk41gmzbibvpx9dpsm5gs8p0liz8shy0-guile-2.0.14/bin/guile
--no-auto-compile /gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system/boot
===

When I run the start script I get the following output.

$ lxc exec guixsd --
/gnu/store/sa7zrdfqglnb5rvvr11qdj0rspbs292v-profile/bin/bash -c
"/sbin/start 2>&1"
/gnu/store/sa7zrdfqglnb5rvvr11qdj0rspbs292v-profile/bin/ln: failed to
create symbolic link
'/run/current-system/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system': File exists
making '#f' the current system...
Backtrace:
In ice-9/boot-9.scm:
 160: 13 [catch #t #<catch-closure 938020> ...]
In unknown file:
   ?: 12 [apply-smob/1 #<catch-closure 938020>]
In ice-9/boot-9.scm:
  66: 11 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 10 [eval # #]
In ice-9/boot-9.scm:
2412: 9 [save-module-excursion #<procedure 95c900 at
ice-9/boot-9.scm:4084:3 ()>]
4089: 8 [#<procedure 95c900 at ice-9/boot-9.scm:4084:3 ()>]
1734: 7 [%start-stack load-stack #<procedure 9663a0 at
ice-9/boot-9.scm:4080:10 ()>]
1739: 6 [#<procedure 96ebd0 ()>]
In unknown file:
   ?: 5 [primitive-load
"/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system/boot"]
In ice-9/eval.scm:
 432: 4 [eval # ()]
In unknown file:
   ?: 3 [primitive-load
"/gnu/store/9j944zjslsihhsgipa7gz7x046fkcjm7-activate"]
In ice-9/eval.scm:
 432: 2 [eval # ()]
In ./gnu/build/activation.scm:
 456: 1 [activate-current-system #f]
In unknown file:
   ?: 0 [symlink #f "/run/current-system.new"]

ERROR: In procedure symlink:
ERROR: Wrong type (expecting string): #f
===

Ignoring the errors above I'll now try to start shepherd, to see how far I
get.

jenkins@ubuntu16041:~/guixsd$ lxc exec guixsd --
/gnu/store/sa7zrdfqglnb5rvvr11qdj0rspbs292v-profile/bin/bash -c
"/gnu/store/q49si29djfcrpzibqg6cg8k6xixxvd2f-shepherd-0.3.2/bin/shepherd
--config /gnu/store/df56ad2rw1ayjyhs1kqadskf5zsmsc5l-shepherd.conf 2>&1"
Service root has been started.
starting services...
Service root-file-system has been started.
Service user-file-systems has been started.
Service file-system-/tmp has been started.
failed to start service 'file-systems'  <<== first problem.
failed to start service 'file-system-/dev/pts'
Service file-system-/dev/shm has been started.
failed to start service 'file-system-/gnu/store'
failed to start service 'user-processes'
Service host-name has been started.
failed to start service 'user-homes'
failed to start service 'nscd'
failed to start service 'ssh-daemon'
waiting for udevd...
waiting for udevd...
waiting for udevd...
waiting for udevd...
Service udev has been started.
Service gpm could not be started.
failed to start service 'console-font-tty1'
failed to start service 'console-font-tty2'
failed to start service 'console-font-tty3'
failed to start service 'console-font-tty4'
failed to start service 'console-font-tty5'
failed to start service 'console-font-tty6'
failed to start service 'guix-daemon'
failed to start service 'syslogd'
failed to start service 'term-tty6'
failed to start service 'term-tty5'
failed to start service 'term-tty4'
failed to start service 'term-tty3'
failed to start service 'term-tty2'
failed to start service 'term-tty1'


  C-c C-c^CExiting shepherd...
unmounting '/dev'...
failed to unmount '/dev': Device or resource busy
unmounting '/dev/null'...
failed to unmount '/dev/null': Device or resource busy
Service user-file-systems has been stopped.
Service host-name has been stopped.
Service file-system-/dev/shm has been stopped.
Service file-system-/tmp has been stopped.
Service udev has been stopped.
closing log
===

See also:
https://lists.gnu.org/archive/html/guix-devel/2016-12/msg00733.html

I would like to get some help to solve this puzzle.



Cheers,
Eddy

[-- Attachment #2: Type: text/html, Size: 6038 bytes --]

Re: guixsd in lxd container

[Ludovic Courtès]

Hi Eddy,

Eddy Pronk <epronk@muftor.com> skribis:

> I'm trying to run guixsd in an lxd container.
> My lxd containers run on an Ubuntu server 16.04.
>
> I took the usb-installer image and imported in as an lxd image.
>
> When a container start it runs /sbin/init.

You mean LXD expects to run /sbin/init, right?

> In guixsd /proc/1 is shepherd, but a lot of stuff happens before shepherd
> is started.
>
> I've set a few things in the environment matching values in /proc/1/environ.
> The argument of --load in grub.cfg is a guile program.
>
> I found some details about the kernel loading guile here:
> https://lists.gnu.org/archive/html/guix-devel/2016-12/msg00704.html
>
> To be able to get some logging during startup of guixsd I'm trying to run
> it from a shell script as root.
>
> ====
> cat /sbin/start
> export HOME=/
> export TERM=linux
> export
> BOOT_IMAGE="/gnu/store/fqc2kg4lq1lz1ymk41080jzb5q90icg0-linux-libre-4.11/bzImage
> --root=gnu-disk-image
> --system=/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system
> --load=/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system/boot"
> export PATH=/gnu/store/crvb68g89b479n4h44r8l42hy39axhg2-shadow-4.4/sbin/
> cd $HOME
> /gnu/store/sa7zrdfqglnb5rvvr11qdj0rspbs292v-profile/bin/ln -s
> /gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system /run/current-system
> /gnu/store/zk41gmzbibvpx9dpsm5gs8p0liz8shy0-guile-2.0.14/bin/guile
> --no-auto-compile /gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system/boot
> ===
>
> When I run the start script I get the following output.
>
> $ lxc exec guixsd --
> /gnu/store/sa7zrdfqglnb5rvvr11qdj0rspbs292v-profile/bin/bash -c
> "/sbin/start 2>&1"
> /gnu/store/sa7zrdfqglnb5rvvr11qdj0rspbs292v-profile/bin/ln: failed to
> create symbolic link
> '/run/current-system/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system': File exists
> making '#f' the current system...

[...]

> In ./gnu/build/activation.scm:
>  456: 1 [activate-current-system #f]
> In unknown file:
>    ?: 0 [symlink #f "/run/current-system.new"]
>
> ERROR: In procedure symlink:
> ERROR: Wrong type (expecting string): #f

The line that’s printed here comes from (gnu build activation):

  https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/build/activation.scm#n442

As you can see, it takes the value of ‘system’ either from the kernel’s
‘--system’ command-line argument (/proc/cmdline), or from the
‘GUIX_NEW_SYSTEM’ environment variable.

So you’d have to set ‘GUIX_NEW_SYSTEM’ in your case to fix this.

> Service file-system-/tmp has been started.
> failed to start service 'file-systems'  <<== first problem.
> failed to start service 'file-system-/dev/pts'

What ‘guix system container’ does to work around this is to try to mount
only file systems that can really be mounted inside a container, with
the right options:

  https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system/linux-container.scm#n37
  https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system/file-systems.scm#n325

HTH!

BTW, did you consider using ‘guix system container’ directly instead of
LXC?  It’s not perfect but probably worth a try:

  https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-system.html

Ludo’.

Re: guixsd in lxd container

[Eddy Pronk]

On Sat, Jun 10, 2017 at 7:54 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>> When a container start it runs /sbin/init.
>
> You mean LXD expects to run /sbin/init, right?

Yes, and the LXD container can be configured to run any other program instead.

> The line that’s printed here comes from (gnu build activation):
>
>   https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/build/activation.scm#n442
>
> As you can see, it takes the value of ‘system’ either from the kernel’s
> ‘--system’ command-line argument (/proc/cmdline), or from the
> ‘GUIX_NEW_SYSTEM’ environment variable.
>
> So you’d have to set ‘GUIX_NEW_SYSTEM’ in your case to fix this.

Added this to my start script:
export GUIX_NEW_SYSTEM=/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system

Great. That works. This got me 2 steps further.

I had to comment out some snippets from activation.scm:

;; activate-modprobe
;; activate-firmware
;; activate-ptrace-attach

(I actually edited the 1 line snippets that get generated somehow.)

Now the boot script starts shepherd.

>> Service file-system-/tmp has been started.
>> failed to start service 'file-systems'  <<== first problem.
>> failed to start service 'file-system-/dev/pts'
>
> What ‘guix system container’ does to work around this is to try to mount
> only file systems that can really be mounted inside a container, with
> the right options:
>
>   https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system/linux-container.scm#n37
>   https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system/file-systems.scm#n325

I'd like to get some logging out of the "failed to start service
'file-systems'".
When /sbin/init starts the usb-install image is already mounted on '/'.

What I see as pstree is:

---shepherd-+-udevd
                       `-{shepherd}

failed to start service 'term-tty1' is the last thing I see.
Can someone post a full log of the shepherd startup?

> BTW, did you consider using ‘guix system container’ directly instead of
> LXC?  It’s not perfect but probably worth a try:

I'll give that a try. Maybe just to learn how it does it.

For Ubuntu users (or others distros with LXD) it would be a nice
managed way of trying out GuixSD if I get this to work.


Cheers,
Eddy

Re: guixsd in lxd container

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 1052 bytes --]

Eddy Pronk writes:

> On Sat, Jun 10, 2017 at 7:54 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> When a container start it runs /sbin/init.
>> As you can see, it takes the value of ‘system’ either from the kernel’s
>> ‘--system’ command-line argument (/proc/cmdline), or from the
>> ‘GUIX_NEW_SYSTEM’ environment variable.
>>
>> So you’d have to set ‘GUIX_NEW_SYSTEM’ in your case to fix this.

Nice to know, kind of obvious when you do...  I saw

>>> making '#f' the current system...

which even gave me a smile...that won't work ;-)

> Added this to my start script:
> export GUIX_NEW_SYSTEM=/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system
>
> Great. That works. This got me 2 steps further.

Yay!

> failed to start service 'term-tty1' is the last thing I see.
> Can someone post a full log of the shepherd startup?

PFA

> For Ubuntu users (or others distros with LXD) it would be a nice
> managed way of trying out GuixSD if I get this to work.

Great work, thanks!

Greetings, janneke


[-- Attachment #2: shepherd.log --]
[-- Type: application/octet-stream, Size: 3269 bytes --]

2016-08-31 07:00:05 Service root has been started.
2016-08-31 07:00:05 starting services...
2016-08-31 07:00:06 Service root-file-system has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/hugetlb has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/perf_event has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/blkio has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/freezer has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/devices has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/memory has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/cpuacct has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/cpu has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/cpuset has been started.
2016-08-31 07:00:06 Service file-system-/sys/fs/cgroup/elogind has been started.
2016-08-31 07:00:06 Service file-system-/run/user has been started.
2016-08-31 07:00:06 Service file-system-/run/systemd has been started.
2016-08-31 07:00:06 Service file-system-/gnu/store has been started.
2016-08-31 07:00:06 Service file-system-/dev/shm has been started.
2016-08-31 07:00:06 Service file-system-/dev/pts has been started.
2016-08-31 07:00:06 Service user-file-systems has been started.
2016-08-31 07:00:06 Service user-processes has been started.
2016-08-31 07:00:06 Service host-name has been started.
2016-08-31 07:00:06 waiting for udevd...
2016-08-31 07:00:09 Service udev has been started.
2016-08-31 07:00:10 Service nscd has been started.
2016-08-31 07:00:10 Service guix-daemon has been started.
2016-08-31 07:00:10 Service urandom-seed has been started.
2016-08-31 07:00:10 Service syslogd has been started.
2016-08-31 07:00:10 Service loopback has been started.
2016-08-31 07:00:10 Service term-tty6 has been started.
2016-08-31 07:00:10 Service term-tty5 has been started.
2016-08-31 07:00:10 Service term-tty4 has been started.
2016-08-31 07:00:10 Service term-tty3 has been started.
2016-08-31 07:00:10 Service term-tty2 has been started.
2016-08-31 07:00:10 Service term-tty1 has been started.
2016-08-31 07:00:10 Service console-font-tty6 has been started.
2016-08-31 07:00:10 Service console-font-tty5 has been started.
2016-08-31 07:00:10 Service console-font-tty4 has been started.
2016-08-31 07:00:10 Service console-font-tty3 has been started.
2016-08-31 07:00:10 Service console-font-tty2 has been started.
2016-08-31 07:00:10 Service console-font-tty1 has been started.
2016-08-31 07:00:11 Service dbus-system has been started.
2016-08-31 07:00:11 Service networking has been started.
2016-08-31 07:00:11 Service ntpd has been started.
2016-08-31 07:00:11 Service upower-daemon has been started.
2016-08-31 07:00:11 Service avahi-daemon has been started.
2016-08-31 07:00:11 Service xorg-server has been started.
2016-08-31 07:00:11 Service mcron has been started.
2016-08-31 07:00:11 Service postgres has been started.
2016-08-31 07:00:11 Service ssh-daemon has been started.
2016-08-31 07:00:11 Service console-keymap has been started.
2016-08-31 07:00:11 Service bluetooth has been started.

[-- Attachment #3: Type: text/plain, Size: 152 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: guixsd in lxd container

[Ludovic Courtès]

Hi,

Eddy Pronk <epronk@muftor.com> skribis:

> On Sat, Jun 10, 2017 at 7:54 AM, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

>> So you’d have to set ‘GUIX_NEW_SYSTEM’ in your case to fix this.
>
> Added this to my start script:
> export GUIX_NEW_SYSTEM=/gnu/store/kq71yhydfgc0nksvmmn66cbvbj5a3mvf-system
>
> Great. That works. This got me 2 steps further.
>
> I had to comment out some snippets from activation.scm:
>
> ;; activate-modprobe
> ;; activate-firmware
> ;; activate-ptrace-attach

Yeah, ‘guix system container’ does that too:

  http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system.scm#n418
  http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services.scm#n466

I think we should look for ways that would allow you to reuse what ‘guix
system container’ does.

HTH,
Ludo’.

Re: guixsd in lxd container

[Eddy Pronk]

On Mon, Jun 12, 2017 at 6:26 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>> I had to comment out some snippets from activation.scm:
>>
>> ;; activate-modprobe
>> ;; activate-firmware
>> ;; activate-ptrace-attach
>
> Yeah, ‘guix system container’ does that too:
>
>   http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system.scm#n418
>   http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services.scm#n466
>
> I think we should look for ways that would allow you to reuse what ‘guix
> system container’ does.

Shall I open a bug for this so it can be tracked?

Eddy

Re: guixsd in lxd container

[Ludovic Courtès]

Hi Eddy,

Sorry for the delay.

Eddy Pronk <epronk@muftor.com> skribis:

> On Mon, Jun 12, 2017 at 6:26 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> I had to comment out some snippets from activation.scm:
>>>
>>> ;; activate-modprobe
>>> ;; activate-firmware
>>> ;; activate-ptrace-attach
>>
>> Yeah, ‘guix system container’ does that too:
>>
>>   http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system.scm#n418
>>   http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services.scm#n466
>>
>> I think we should look for ways that would allow you to reuse what ‘guix
>> system container’ does.
>
> Shall I open a bug for this so it can be tracked?

Before, I’d like to make sure we have a good understanding of what we
need.

My first question would be, do we really need to have a mechanism other
than ‘guix system container’?  I guess that if the host system runs LXD,
it’s better to use it than to spawn the script that ‘guix system
container’ generates.

In that case, what about adding an LXD backend to ‘guix system
container’?  AIUI LXD has a REST API¹; by doing a POST on /1.0/images,
we should be able to register our container image, though it’s not clear
to me what image format is expected.  If we can figure out these
details, it might not be that hard to implement.

Dave, there’s also some overlap with your work on ‘guix deloy’ I think.
Thoughts?

Ludo’.

¹ https://github.com/lxc/lxd/blob/master/doc/rest-api.md

Re: guixsd in lxd container

[Eddy Pronk]

On Mon, Jun 19, 2017 at 9:41 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> Shall I open a bug for this so it can be tracked?
>
> Before, I’d like to make sure we have a good understanding of what we
> need.
>
> My first question would be, do we really need to have a mechanism other
> than ‘guix system container’?  I guess that if the host system runs LXD,
> it’s better to use it than to spawn the script that ‘guix system
> container’ generates.
>
> In that case, what about adding an LXD backend to ‘guix system
> container’?  AIUI LXD has a REST API¹; by doing a POST on /1.0/images,
> we should be able to register our container image, though it’s not clear
> to me what image format is expected.  If we can figure out these
> details, it might not be that hard to implement.

Lxd expects a root file system and a traditional 'init' process.
For this experiment I'm using a bash script to play the role of /sbin/init.

I'll need to spend some time to see what 'guix system container' generates.
I'm very new to guix, so that will be my homework for my spare time
this week. :-)

This weekend I set up a VM on Google cloud with lxd.

Below the log of all the steps I did for this experiment.

(I can give anyone who wants to experiment in this environment access.
Just send me you ssh public key.)


My recipe so far:

$ wget https://alpha.gnu.org/gnu/guix/guixsd-vm-image-0.13.0.x86_64-linux.xz

$ xz -d guixsd-vm-image-0.13.0.x86_64-linux.xz

$ qemu-img convert guixsd-vm-image-0.13.0.x86_64-linux image.raw

We need the sector size and start sector for the right offset:
$ fdisk image.raw

Sector size (logical/physical): 512 bytes / 512 bytes

Device     Boot   Start     End Sectors Size Id Type
image.raw1 *       2048 4093952 4091905   2G 83 Linux
image.raw2      4093953 4175873   81921  40M ef EFI (FAT-12/16/32)

Create a loopback device with an offset poiting to the Linux partition:

$ sudo losetup /dev/loop0 image.raw -o $((2048 * 512))

Mount it. Now we have to content of the vm image on /mnt.
$ sudo mount /dev/loop0 /mnt

I hope this preserves links, timestamps in the right way.
$ sudo tar cpf ./rootfs.tar -C /mnt/ .
tar: ./dev/log: socket ignored

=== metadata.yaml ===
architecture: "x86_64"
creation_date: 1424284563
properties:
  description: "GuixSD Intel 64bit"
  os: "guixsd"
  release: "0.0"
===

lxc imports an image from 2 tarballs:
$ tar cf metadata.tar metadata.yaml
$ lxc image import metadata.tar rootfs.tar --alias guixsd-vm

epronk@instance-1:~$ lxc image list
+-----------+--------------+--------+--------------------+--------+----------+------------------------------+
|   ALIAS   | FINGERPRINT  | PUBLIC |    DESCRIPTION     |  ARCH  |
SIZE   |         UPLOAD DATE          |
+-----------+--------------+--------+--------------------+--------+----------+------------------------------+
| guixsd-vm | c9eeb3dfcee7 | no     | GuixSD Intel 64bit | x86_64 |
883.92MB | Jun 17, 2017 at 5:43am (UTC) |
+-----------+--------------+--------+--------------------+--------+----------+------------------------------+

Create container called guixsd from guixsd-vm image:

$ lxc launch guixsd-vm guixsd

barf... no /sbin/init yet.

epronk@instance-1:~$ lxc list
+--------+---------+------+------+------------+-----------+
|  NAME  |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+------+------+------------+-----------+
| guixsd | STOPPED |      |      | PERSISTENT | 0         |
+--------+---------+------+------+------------+-----------+

epronk@instance-1:~$ sudo ls /var/lib/lxd/containers/guixsd/rootfs
bin  boot  dev         etc  gnu  home    lost+found  mnt  root  run  tmp  var

epronk@instance-1:~$ sudo find /var/lib/lxd/containers/guixsd/rootfs -name sleep
/var/lib/lxd/containers/guixsd/rootfs/gnu/store/xniak294s1x03zssfj2xzvfkcny1gn0x-profile/bin/sleep
(other entries omitted)

I don't know how to see the output of /sbin/init. For now all
/sbin/init does is sleep.
We start shepherd manually in later steps.

=== /sbin/init ===
#!/gnu/store/xniak294s1x03zssfj2xzvfkcny1gn0x-profile/bin/bash
/gnu/store/xniak294s1x03zssfj2xzvfkcny1gn0x-profile/bin/sleep 99999
===

$ sudo mkdir /var/lib/lxd/containers/guixsd/rootfs/sbin
$ sudo cp init /var/lib/lxd/containers/guixsd/rootfs/sbin/init

$ lxc start guixsd
$ lxc list
+--------+---------+------+------+------------+-----------+
|  NAME  |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+--------+---------+------+------+------------+-----------+
| guixsd | RUNNING |      |      | PERSISTENT | 0         |
+--------+---------+------+------+------------+-----------+

Now that the container is in a running state I can attach bash as a
process to it:
epronk@instance-1:~$ lxc exec guixsd
/gnu/store/xniak294s1x03zssfj2xzvfkcny1gn0x-profile/bin/bash
bash-4.4#

'start' is blueprint for /sbin/init.

=== /sbin/start/ ===
export HOME=/
export TERM=linux

export PATH=/gnu/store/crvb68g89b479n4h44r8l42hy39axhg2-shadow-4.4/sbin/
export GUIX_NEW_SYSTEM=/gnu/store/4pr317614r1ff1bi6vd1q0jjdca5h78s-system
cd $HOME
/gnu/store/zk41gmzbibvpx9dpsm5gs8p0liz8shy0-guile-2.0.14/bin/guile
--no-auto-compile $GUIX_NEW_SYSTEM/boot
===

Run start script via bash, so we can see stderr and stdout from host OS.

$ lxc exec guixsd --
/gnu/store/xniak294s1x03zssfj2xzvfkcny1gn0x-profile/bin/bash -c
"/sbin/start 2>&1"

Error #1:

   ?: 2 [primitive-load
"/gnu/store/ysvjgjb9ph1vg0m4y67lfrj06wc5gdx4-activate-service"]
   In ice-9/boot-9.scm:
    893: 1 [call-with-output-file
"/sys/module/firmware_class/parameters/path" ...]
    In unknown file:
       ?: 0 [open-file "/sys/module/firmware_class/parameters/path"
"w" #:encoding #f]

$ sudo chmod +w
/var/lib/lxd/containers/guixsd/rootfs/gnu/store/ysvjgjb9ph1vg0m4y67lfrj06wc5gdx4-activate-service

Comment out with ;;
$ sudo emacs /var/lib/lxd/containers/guixsd/rootfs/gnu/store/ysvjgjb9ph1vg0m4y67lfrj06wc5gdx4-activate-service

Error #2:

?: 3 [primitive-load
"/gnu/store/nz2wixyg218l9j56vb21w0whnvdrnmh5-activate-service"]
In ice-9/eval.scm:
 432: 2 [eval # ()]
 In ice-9/boot-9.scm:
  893: 1 [call-with-output-file "/proc/sys/kernel/modprobe" ...]
  In unknown file:
     ?: 0 [open-file "/proc/sys/kernel/modprobe" "w" #:encoding #f]


Commented out expression in
/gnu/store/nz2wixyg218l9j56vb21w0whnvdrnmh5-activate-service

After fixing these #1 #2, shepherd starts, but reports services that
didn't start.

The first service that reports an issue is file-systems.

Next we tried to start some services manually:

epronk@instance-1:~$ lxc exec guixsd
/gnu/store/xniak294s1x03zssfj2xzvfkcny1gn0x-profile/bin/bash
bash-4.4#

root@gnu ~# herd start guix-daemon
herd start guix-daemon herd: exception caught while executing 'start'
on service 'file-system-/gnu/store': ERROR:
In procedure mount: mount "/gnu/store" on "///gnu/store": Permission denied

This is how far I got.

I hope this give some idea of what the image looks like and what I
tried to start it.


Cheers,
Eddy