`dockerd` service not starting

`dockerd` service not starting

[Tanguy LE CARROUR]

Hi Guix,

My `dockerd` service has stopped working weeks (months!?) ago, but I've
had no use for it in the meantime, so it was no big deal, until today…

My configuration looks like this:

```scm
;…
(use-service-modules desktop docker)

(operating-system
  ;…
  (services
    (append
      (list
        (service docker-service-type)
       ;…
       ))))
```

This used to be enough. But now, it doesn't auto-start any more and,
when I `sudo herd start dockerd`, it hangs and displays:

```console
$ sudo herd start dockerd
Password:
Service dockerd could not be started.
herd: failed to start service dockerd
```

The log contains the following:

```log:/var/log/docker.log
time="2022-03-31T11:31:32.027833616+02:00" level=info msg="Starting up"
time="2022-03-31T11:31:32.031576010+02:00" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2022-03-31T11:31:32.031661605+02:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2022-03-31T11:31:32.031760157+02:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
time="2022-03-31T11:31:32.031816856+02:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
failed to start daemon: failed to dial "/run/containerd/containerd.sock": unknown service containerd.services.namespaces.v1.Namespaces: not implemented
```

`containerd` seems to be running fine, even though I sometimes see the
following in the log:

```log:/var/log/docker.log
failed to start containerd: exec: "containerd": executable file not found in $PATH
```

Actually, it looks like a problem reported in another thread [1][] a year ago,
that hasn't (yet) found a solution!? :-(

[1]: https://lists.gnu.org/archive/html/help-guix/2021-06/msg00019.html

Any help welcome!

Regards,

-- 
Tanguy

Re: `dockerd` service not starting

[Pierre Langlois]

[-- Attachment #1.1: Type: text/plain, Size: 0 bytes --]



[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

[-- Attachment #2.1: Type: text/plain, Size: 2300 bytes --]

Hi!

Tanguy LE CARROUR <tanguy@bioneland.org> writes:

> Hi Guix,
>
> My `dockerd` service has stopped working weeks (months!?) ago, but I've
> had no use for it in the meantime, so it was no big deal, until today…
>
> My configuration looks like this:
>
> ```scm
> ;…
> (use-service-modules desktop docker)
>
> (operating-system
>   ;…
>   (services
>     (append
>       (list
>         (service docker-service-type)
>        ;…
>        ))))
> ```
>
> This used to be enough. But now, it doesn't auto-start any more and,
> when I `sudo herd start dockerd`, it hangs and displays:
>
> ```console
> $ sudo herd start dockerd
> Password:
> Service dockerd could not be started.
> herd: failed to start service dockerd
> ```
>
> The log contains the following:
>
> ```log:/var/log/docker.log
> time="2022-03-31T11:31:32.027833616+02:00" level=info msg="Starting up"
> time="2022-03-31T11:31:32.031576010+02:00" level=info msg="parsed scheme: \"unix\"" module=grpc
> time="2022-03-31T11:31:32.031661605+02:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
> time="2022-03-31T11:31:32.031760157+02:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
> time="2022-03-31T11:31:32.031816856+02:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
> failed to start daemon: failed to dial "/run/containerd/containerd.sock": unknown service containerd.services.namespaces.v1.Namespaces: not implemented
> ```
>
> `containerd` seems to be running fine, even though I sometimes see the
> following in the log:
>
> ```log:/var/log/docker.log
> failed to start containerd: exec: "containerd": executable file not found in $PATH
> ```

That's strange, I can also see this in my logs from back in April 2021,
but not since then.

As a shot in the dark, could you try the attached patch?  It's something
I added while looking at updating docker (I need to rebase this series
and get it committed). It's a bugfix, the patch-paths phase does not run
withouth it so it could be related to your issues. But for some strange
reason it's been working for me.

https://issues.guix.gnu.org/52790#1


[-- Attachment #2.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-gnu-containerd-Fix-patch-paths-build-phase.patch --]
[-- Type: text/x-patch, Size: 1151 bytes --]

From b7ca964bd265a06ffeb9b2f1d6a36c592876fd52 Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Sat, 25 Dec 2021 02:04:35 +0000
Subject: [PATCH] gnu: containerd: Fix patch-paths build phase.

* gnu/packages/docker.scm (containerd)[arguments]: Add 'patch-paths
phases after 'unpack because 'chdir doesn't exist.  Substitute runc
binary for "pkg/cri/config/config_unix.go".
---
 gnu/packages/docker.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 3f52f4f8db..9b15405a72 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -190,7 +190,7 @@ (define-public containerd
        `(#:import-path "github.com/containerd/containerd"
          #:phases
          (modify-phases %standard-phases
-           (add-after 'chdir 'patch-paths
+           (add-after 'unpack 'patch-paths
              (lambda* (#:key inputs import-path outputs #:allow-other-keys)
                (with-directory-excursion (string-append "src/" import-path)
                  (substitute* "runtime/v1/linux/runtime.go"
--
2.34.0


[-- Attachment #4: Type: text/plain, Size: 16 bytes --]


Thanks,
Pierre

Re: `dockerd` service not starting

[tanguy]

Hi Pierre,

On 2022-03-31 19:30, Pierre Langlois wrote:
> Tanguy LE CARROUR <tanguy@bioneland.org> writes:
>> My `dockerd` service has stopped working weeks (months!?) ago, but 
>> I've
>> had no use for it in the meantime, so it was no big deal, until today…
>> […]
> 
> That's strange, I can also see this in my logs from back in April 2021,
> but not since then.
> 
> As a shot in the dark, could you try the attached patch?  It's 
> something
> I added while looking at updating docker (I need to rebase this series
> and get it committed). It's a bugfix, the patch-paths phase does not 
> run
> withouth it so it could be related to your issues. But for some strange
> reason it's been working for me.
> 
> https://issues.guix.gnu.org/52790#1

Thanks for the patch! I applied it and successfully built containerd and 
docker, but…
I have no clue how to `guix system reconfigure` from my guix repo 
checkout!? ^_^'

Usually, as my regular user, I do `./pre-inst-env guix …`, but in order 
to run
`guix system reconfigure` I use sudo.
But `sudo ./pre-inst-env guix system reconfigure config.scm` fails with 
an ugly:

```console
Backtrace:
In ice-9/boot-9.scm:
   3253:13 19 (_)
In ice-9/threads.scm:
     390:8 18 (_ _)
In ice-9/boot-9.scm:
   3544:20 17 (_)
    2836:4 16 (save-module-excursion _)
   3564:26 15 (_)
In unknown file:
           14 (primitive-load-path "guix/store" #<procedure 7f3bd28bf…>)
In ice-9/boot-9.scm:
   3923:23 13 (_)
    3411:4 12 (define-module* _ #:filename _ #:pure _ #:version _ # _ …)
   3424:24 11 (_)
    222:29 10 (map1 (((guix utils)) ((guix config)) ((guix #)) ((…)) …))
    222:29  9 (map1 (((guix config)) ((guix deprecation)) ((guix …)) …))
    222:29  8 (map1 (((guix deprecation)) ((guix memoization)) ((…)) …))
    222:29  7 (map1 (((guix memoization)) ((guix serialization)) (#) …))
    222:29  6 (map1 (((guix serialization)) ((guix monads)) ((# #)) …))
    222:29  5 (map1 (((guix monads)) ((guix records)) ((guix #)) (#) …))
    222:29  4 (map1 (((guix records)) ((guix base16)) ((guix #)) (#) …))
    222:29  3 (map1 (((guix base16)) ((guix base32)) ((gcrypt #)) # …))
    222:29  2 (map1 (((guix base32)) ((gcrypt hash)) ((guix #)) (#) …))
    222:17  1 (map1 (((gcrypt hash)) ((guix profiling)) ((rnrs #)) # …))
    3330:6  0 (resolve-interface (gcrypt hash) #:select _ #:hide _ # _ …)

ice-9/boot-9.scm:3330:6: In procedure resolve-interface:
no code for module (gcrypt hash)
```

How am I supposed to do it?!

Best regards,

-- 
Tanguy

Re: `dockerd` service not starting

[Pierre Langlois]

[-- Attachment #1: Type: text/plain, Size: 1529 bytes --]

Hi,

tanguy@bioneland.org writes:

> Hi Pierre,
>
> On 2022-03-31 19:30, Pierre Langlois wrote:
>> Tanguy LE CARROUR <tanguy@bioneland.org> writes:
>>> My `dockerd` service has stopped working weeks (months!?) ago, but I've
>>> had no use for it in the meantime, so it was no big deal, until today…
>>> […]
>> That's strange, I can also see this in my logs from back in April 2021,
>> but not since then.
>> As a shot in the dark, could you try the attached patch?  It's 
>> something
>> I added while looking at updating docker (I need to rebase this series
>> and get it committed). It's a bugfix, the patch-paths phase does not run
>> withouth it so it could be related to your issues. But for some strange
>> reason it's been working for me.
>> https://issues.guix.gnu.org/52790#1
>
> Thanks for the patch! I applied it and successfully built containerd and docker,
> but…
> I have no clue how to `guix system reconfigure` from my guix repo checkout!?
> ^_^'
>
> Usually, as my regular user, I do `./pre-inst-env guix …`, but in order to run
> `guix system reconfigure` I use sudo.
> But `sudo ./pre-inst-env guix system reconfigure config.scm` fails with an ugly:

Oh yeah, for this to work you need to use the -E option for sudo:

    sudo -E ./pre-inst-env guix system reconfigure config.scm

I believe it's so that the current $GUIX_ENVIRONMENT can be passed to
./pre-inst-env.  I see this hack isn't in the manual's contributing
section, we should add it!

Thanks,
Pierre

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

Re: `dockerd` service not starting

[Tanguy LE CARROUR]

Hi Pierre,


Quoting Pierre Langlois (2022-04-01 21:05:08)
> tanguy@bioneland.org writes:
> > On 2022-03-31 19:30, Pierre Langlois wrote:
> >> Tanguy LE CARROUR <tanguy@bioneland.org> writes:
> >>> My `dockerd` service has stopped working weeks (months!?) ago, but I've
> >>> had no use for it in the meantime, so it was no big deal, until today…
> >>> […]
> >> That's strange, I can also see this in my logs from back in April 2021,
> >> but not since then.
> >> As a shot in the dark, could you try the attached patch?  It's 
> >> something
> >> I added while looking at updating docker (I need to rebase this series
> >> and get it committed). It's a bugfix, the patch-paths phase does not run
> >> withouth it so it could be related to your issues. But for some strange
> >> reason it's been working for me.
> >> https://issues.guix.gnu.org/52790#1
> >
> > Thanks for the patch! I applied it and successfully built containerd and docker,
> > but…
> > I have no clue how to `guix system reconfigure` from my guix repo checkout!?
> > ^_^'
> >
> > Usually, as my regular user, I do `./pre-inst-env guix …`, but in order to run
> > `guix system reconfigure` I use sudo.
> > But `sudo ./pre-inst-env guix system reconfigure config.scm` fails with an ugly:
> 
> Oh yeah, for this to work you need to use the -E option for sudo:
> 
>     sudo -E ./pre-inst-env guix system reconfigure config.scm
> 
> I believe it's so that the current $GUIX_ENVIRONMENT can be passed to
> ./pre-inst-env.  I see this hack isn't in the manual's contributing
> section, we should add it!

`-E`! Of course! I should have thought about it. Sorry!
It worked. It built. But still, it does not start! :-(

I still have the same error message:

```
failed to start daemon: failed to dial "/run/containerd/containerd.sock": unknown service containerd.services.namespaces.v1.Namespaces: not implemented
```

Next version of `docker` might fix this… or not! ^_^'

Regards,

-- 
Tanguy

Re: `dockerd` service not starting

[Pierre Langlois]

[-- Attachment #1: Type: text/plain, Size: 2571 bytes --]

Hi,

Tanguy LE CARROUR <tanguy@bioneland.org> writes:

> Hi Pierre,
>
>
> Quoting Pierre Langlois (2022-04-01 21:05:08)
>> tanguy@bioneland.org writes:
>> > On 2022-03-31 19:30, Pierre Langlois wrote:
>> >> Tanguy LE CARROUR <tanguy@bioneland.org> writes:
>> >>> My `dockerd` service has stopped working weeks (months!?) ago, but I've
>> >>> had no use for it in the meantime, so it was no big deal, until today…
>> >>> […]
>> >> That's strange, I can also see this in my logs from back in April 2021,
>> >> but not since then.
>> >> As a shot in the dark, could you try the attached patch?  It's 
>> >> something
>> >> I added while looking at updating docker (I need to rebase this series
>> >> and get it committed). It's a bugfix, the patch-paths phase does not run
>> >> withouth it so it could be related to your issues. But for some strange
>> >> reason it's been working for me.
>> >> https://issues.guix.gnu.org/52790#1
>> >
>> > Thanks for the patch! I applied it and successfully built containerd and docker,
>> > but…
>> > I have no clue how to `guix system reconfigure` from my guix repo checkout!?
>> > ^_^'
>> >
>> > Usually, as my regular user, I do `./pre-inst-env guix …`, but in order to run
>> > `guix system reconfigure` I use sudo.
>> > But `sudo ./pre-inst-env guix system reconfigure config.scm` fails with an ugly:
>> 
>> Oh yeah, for this to work you need to use the -E option for sudo:
>> 
>>     sudo -E ./pre-inst-env guix system reconfigure config.scm
>> 
>> I believe it's so that the current $GUIX_ENVIRONMENT can be passed to
>> ./pre-inst-env.  I see this hack isn't in the manual's contributing
>> section, we should add it!
>
> `-E`! Of course! I should have thought about it. Sorry!
> It worked. It built. But still, it does not start! :-(
>
> I still have the same error message:
>
> ```
> failed to start daemon: failed to dial "/run/containerd/containerd.sock": unknown service containerd.services.namespaces.v1.Namespaces: not implemented
> ```
>
> Next version of `docker` might fix this… or not! ^_^'

Yeah that might be worth trying, I rebased the docker update series if
you want to test it: https://issues.guix.gnu.org/52790#7

I wonder if it could be related to the kernel, maybe it could missing
some support for namespaces?  Are you running the default Guix
linux-libre kernel?

You could also try and run docker's system test, to check if it works in
a fresh VM with the default kernel:

  make check-system TESTS="docker"

Hopefully this helps!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

Re: `dockerd` service not starting

[Tanguy LE CARROUR]

Hi Pierre,


Quoting Pierre Langlois (2022-04-02 12:43:35)
> Tanguy LE CARROUR <tanguy@bioneland.org> writes:
> > […]
> > I still have the same error message:
> >
> > ```
> > failed to start daemon: failed to dial "/run/containerd/containerd.sock": unknown service containerd.services.namespaces.v1.Namespaces: not implemented
> > ```
> >
> > Next version of `docker` might fix this… or not! ^_^'
> 
> Yeah that might be worth trying, I rebased the docker update series if
> you want to test it: https://issues.guix.gnu.org/52790#7

Thanks!
Unfortunately, it didn't help! :-(


> I wonder if it could be related to the kernel, maybe it could missing
> some support for namespaces?  Are you running the default Guix
> linux-libre kernel?

I don't even know how to run another kernel, so yes! ^_^'


> You could also try and run docker's system test, to check if it works in
> a fresh VM with the default kernel:
> 
>   make check-system TESTS="docker"
> 
> Hopefully this helps!

I'll give it a try! Thanks!


-- 
Tanguy