guix docker on gitlab-ci

guix docker on gitlab-ci

[Graham Addis]

Dear people,

I tried to create a docker image to use in a gitlab-ci instance but it
failed because I couldn't use --entry-point="bin/sh -l -c" or
equivalent, basically the gitlab-runner complains that it can't run
binaries.

I've managed to get it working by making some changes to guix/scripts/pack.scm

Adding a fn in docker-image, just before the call to
build-docker-image, to create a list from the string passed in from
--entry-point="bin/sh -l -c"

            (define (make-docker-exec-form prefix value)
              (cond
               ((equal? value '())
                '())
               ((equal? prefix '())
                (string-split value #\space))
               (else
                (let ((values (string-split value #\space)))
                  (cons
                   (string-append prefix "/" (car values))
                   (cdr values))))))

And replacing the setting of entry-point in the build-docker-image call to:

                                #:entry-point (make-docker-exec-form
#$profile #$entry-point)

The call to build-docker-image takes a list for entry-point, and it
all works fine as far as I can tell.

Before I send in a patch, some questions:

Am I missing something?

Am I on the right track?

Should I be splitting this out before the call to docker-image?

Thanks,

Graham

Re: guix docker on gitlab-ci

[wolf]

[-- Attachment #1: Type: text/plain, Size: 2548 bytes --]

On 2023-05-24 18:04:47 +0100, Graham Addis wrote:
> Dear people,
> 
> I tried to create a docker image to use in a gitlab-ci instance but it
> failed because I couldn't use --entry-point="bin/sh -l -c" or
> equivalent, basically the gitlab-runner complains that it can't run
> binaries.

Would this be better using just bin/sh for the entry point and passing the -l
and -c as an arguments?

> 
> I've managed to get it working by making some changes to guix/scripts/pack.scm
> 
> Adding a fn in docker-image, just before the call to
> build-docker-image, to create a list from the string passed in from
> --entry-point="bin/sh -l -c"
> 
>             (define (make-docker-exec-form prefix value)
>               (cond
>                ((equal? value '())
>                 '())
>                ((equal? prefix '())
>                 (string-split value #\space))
>                (else
>                 (let ((values (string-split value #\space)))
>                   (cons
>                    (string-append prefix "/" (car values))
>                    (cdr values))))))

If I read this right (sorry, still somewhat new to guile), you basically split
the --entry-point argument on spaces and use those parts as separate values to
invoke, is that correct?  If so, how would you pass a binary that has space in
the name (joke example: `/bin/ba sh') into the entry-point?

> 
> And replacing the setting of entry-point in the build-docker-image call to:
> 
>                                 #:entry-point (make-docker-exec-form
> #$profile #$entry-point)
> 
> The call to build-docker-image takes a list for entry-point, and it
> all works fine as far as I can tell.
> 
> Before I send in a patch, some questions:
> 
> Am I missing something?
> 
> Am I on the right track?

In my opinion (which you are free to disagree with :) ), I think it would be
better to either have /bin/sh as an entry-point (and pass -l -c as arguments
when starting the container, if required) or create a wrapper script /bin/shlc
that would exec /bin/sh with correct arguments.

Few random ideas: Maybe the same format Containerfiles use for cmd and
entrypoint directives could be used?  Maybe the --entry-point could also (in
addition to a string) accept a list of strings (LISP list)?

> 
> Should I be splitting this out before the call to docker-image?
> 
> Thanks,
> 
> Graham
> 

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: guix docker on gitlab-ci

[Graham Addis]

Hi Worf,

Thanks for the response, see below.

On Mon, 29 May 2023 at 20:41, wolf <wolf@wolfsden.cz> wrote:
>
> On 2023-05-24 18:04:47 +0100, Graham Addis wrote:
> > Dear people,
> >
> > I tried to create a docker image to use in a gitlab-ci instance but it
> > failed because I couldn't use --entry-point="bin/sh -l -c" or
> > equivalent, basically the gitlab-runner complains that it can't run
> > binaries.
>
> Would this be better using just bin/sh for the entry point and passing the -l
> and -c as an arguments?

Probably, but I don't think that's an option in gitlab ci and anyway
it would be nice to support the docker options.

> > I've managed to get it working by making some changes to guix/scripts/pack.scm
> >
> > Adding a fn in docker-image, just before the call to
> > build-docker-image, to create a list from the string passed in from
> > --entry-point="bin/sh -l -c"
> >
> >             (define (make-docker-exec-form prefix value)
> >               (cond
> >                ((equal? value '())
> >                 '())
> >                ((equal? prefix '())
> >                 (string-split value #\space))
> >                (else
> >                 (let ((values (string-split value #\space)))
> >                   (cons
> >                    (string-append prefix "/" (car values))
> >                    (cdr values))))))
>
> If I read this right (sorry, still somewhat new to guile), you basically split
> the --entry-point argument on spaces and use those parts as separate values to
> invoke, is that correct?  If so, how would you pass a binary that has space in
> the name (joke example: `/bin/ba sh') into the entry-point?

Basically, yes, and you are right about the problem.

I looked through all the guix documentation I could find and the only
other place I saw that a list was passed in an option was for URLs and
they were separated by spaces.

> > And replacing the setting of entry-point in the build-docker-image call to:
> >
> >                                 #:entry-point (make-docker-exec-form
> > #$profile #$entry-point)
> >
> > The call to build-docker-image takes a list for entry-point, and it
> > all works fine as far as I can tell.
> >
> > Before I send in a patch, some questions:
> >
> > Am I missing something?
> >
> > Am I on the right track?
>
> In my opinion (which you are free to disagree with :) ), I think it would be
> better to either have /bin/sh as an entry-point (and pass -l -c as arguments
> when starting the container, if required) or create a wrapper script /bin/shlc
> that would exec /bin/sh with correct arguments.

Yep, lots of possible workarounds, but it seems to me that it would be
better spending the time adjusting the pack command to fit the spec.

> Few random ideas: Maybe the same format Containerfiles use for cmd and
> entrypoint directives could be used?  Maybe the --entry-point could also (in
> addition to a string) accept a list of strings (LISP list)?

Sounds good to me. Do you have a reference for the json for this? (Not
a big deal as I think I've worked it out from the code, but it's
always nice to have the specs...)

From the Dockerfile reference for ENTRYPOINT
https://docs.docker.com/engine/reference/builder/#entrypoint there are
two fomrs:

ENTRYPOINT ["executable", "param1", "param2"] # The exec form, which
is the preferred form:

ENTRYPOINT command param1 param2 # The shell form:

To implement the shell form I'd need to update build-docker-image in
guix/docker.scm
https://git.savannah.gnu.org/cgit/guix.git/tree/guix/docker.scm#n139
to take a string instead of/ as well as the list it currently takes.
Then update docker-image in guix/scripts/pack.scm
https://git.savannah.gnu.org/cgit/guix.git/tree/guix/scripts/pack.scm#n592

Invocation would then simply be --entry-point="command param1 param2"

To implement the exec form (preferred according to docker) I wouldn't
need to touch guix/docker.scm, but I would probably need to change the
parsing for --entry-point as well as updating docker-iimge.

I prefer the second option, for which all I need is some guidance on
the option syntax

.e.g. --entry-point=["command", "param1", "param2"]

Suggestions please. :)

I could implement both and test for a string or a list and choose
between the shell and exec forms from there, but to be consistent with
the existing implementation.

Once I'm clear about the best approach for this, I could add the CMD
too, if that would be useful.
https://docs.docker.com/engine/reference/builder/#cmd

One strange thing, I couldn't see the need for prefixing the profile
to the ENTRYPOINT command:
https://git.savannah.gnu.org/cgit/guix.git/tree/guix/scripts/pack.scm#n670
I took it out and everything seems to work, so I'm not sure what
problem it is fixing. Anybody any idea?

Thanks,

Graham

Re: guix docker on gitlab-ci

[Graham Addis]

Hi Wolf,

On Tue, 30 May 2023 at 07:52, Graham Addis <grahamjamesaddis@gmail.com> wrote:
>
> ENTRYPOINT ["executable", "param1", "param2"] # The exec form, which
> is the preferred form:

I realised that pack takes multiple symlink(s) using

--symlink /bin=bin --symlink /opt=opt

I could use the equivalent format for --entry-point

--entry-point executable --entry-point param1 --entry-point param2

This seems the most consistent approach with the current implementation so far.

Thoughts?

Re: guix docker on gitlab-ci

[wolf]

[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]

On 2023-05-31 18:47:03 +0100, Graham Addis wrote:
> Hi Wolf,
> 
> On Tue, 30 May 2023 at 07:52, Graham Addis <grahamjamesaddis@gmail.com> wrote:
> >
> > ENTRYPOINT ["executable", "param1", "param2"] # The exec form, which
> > is the preferred form:
> 
> I realised that pack takes multiple symlink(s) using
> 
> --symlink /bin=bin --symlink /opt=opt
> 
> I could use the equivalent format for --entry-point
> 
> --entry-point executable --entry-point param1 --entry-point param2
> 
> This seems the most consistent approach with the current implementation so far.
> 
> Thoughts?
> 

I think that is a reasonable idea.  Only downside is that it would not be
backwards compatible (since currently last --entry-point wins), however I am not
sure if someone actually relies on that behavior.

Backwards compatible way would be keeping --entry-point as it is and introducing
new argument (--entry-point-arg) that could be used to build the argument list,
but I might be overthinking it :).

W.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: guix docker on gitlab-ci

[Graham Addis]

Hi Wolf,

On Thu, 1 Jun 2023 at 22:55, wolf <wolf@wolfsden.cz> wrote:
>
> On 2023-05-31 18:47:03 +0100, Graham Addis wrote:
> > I could use the equivalent format for --entry-point
> >
> > --entry-point executable --entry-point param1 --entry-point param2
> >
> I think that is a reasonable idea.  Only downside is that it would not be
> backwards compatible (since currently last --entry-point wins), however I am not
> sure if someone actually relies on that behavior.
>
> Backwards compatible way would be keeping --entry-point as it is and introducing
> new argument (--entry-point-arg) that could be used to build the argument list,
> but I might be overthinking it :).

I'll go with extending --entry-point to accept multiple values and use
all of them for --format=docker and simply use the last one provided
for the other formats.

I'll try to put a patch together at the weekend.

Thanks for all your input, it really helps.

Graham

Re: guix docker on gitlab-ci

[Graham Addis]

Hi Wolf,

Not a particularly successful weekend...

As --entry-point was used by other pack methods I thought the best
option would be to add a --docker-entry-point just for the docker
format and pass it into the build as an extra-option like for deb and
rpm formats.

However I couldn't work out how to pass in a list via the
extra-options, so I'm a bit stuck.

If there is anyone who knows their way around the pack scripts and can
point me in the right direction, that would help, but other than that
I'll probably get some more time next weekend.

Graham

On Fri, 2 Jun 2023 at 09:13, Graham Addis <grahamjamesaddis@gmail.com> wrote:
>
> Hi Wolf,
>
> On Thu, 1 Jun 2023 at 22:55, wolf <wolf@wolfsden.cz> wrote:
> >
> > On 2023-05-31 18:47:03 +0100, Graham Addis wrote:
> > > I could use the equivalent format for --entry-point
> > >
> > > --entry-point executable --entry-point param1 --entry-point param2
> > >
> > I think that is a reasonable idea.  Only downside is that it would not be
> > backwards compatible (since currently last --entry-point wins), however I am not
> > sure if someone actually relies on that behavior.
> >
> > Backwards compatible way would be keeping --entry-point as it is and introducing
> > new argument (--entry-point-arg) that could be used to build the argument list,
> > but I might be overthinking it :).
>
> I'll go with extending --entry-point to accept multiple values and use
> all of them for --format=docker and simply use the last one provided
> for the other formats.
>
> I'll try to put a patch together at the weekend.
>
> Thanks for all your input, it really helps.
>
> Graham

Re: guix docker on gitlab-ci

[wolf]

[-- Attachment #1: Type: text/plain, Size: 4021 bytes --]

On 2023-06-05 16:37:50 +0100, Graham Addis wrote:
> Hi Wolf,
> 
> Not a particularly successful weekend...
> 
> As --entry-point was used by other pack methods I thought the best
> option would be to add a --docker-entry-point just for the docker
> format and pass it into the build as an extra-option like for deb and
> rpm formats.
> 
> However I couldn't work out how to pass in a list via the
> extra-options, so I'm a bit stuck.
> 
> If there is anyone who knows their way around the pack scripts and can
> point me in the right direction, that would help, but other than that
> I'll probably get some more time next weekend.

I did not try to implement this, so my guess might be completely off, but
looking at how -S is implemented, I would suggest trying something like:

1. Introducing new %docker-format-options and friends (similar to already
   existing %deb-format-options and friends), providing the --entry-point-arg
   options, that would be specific to a docker format (although I am not sure if
   it needs to be specific, maybe some other formats support arguments as
   well?).
2. Writing entry-point-arg-spec-option-parser, while taking inspiration from
   symlink-spec-option-parser.  Yours would be much simpler, since you would be
   just accumulating values into a list.

As for the extra-options, I guess modifying current code (by adding the 'docker
option) into something like:

    (extra-options (match pack-format
                     ('deb
                      (list #:control-file
                            (process-file-arg opts 'control-file)
                            #:postinst-file
                            (process-file-arg opts 'postinst-file)
                            #:triggers-file
                            (process-file-arg opts 'triggers-file)))
                     ('docker
                      (list #:entry-point-args
                            (process-file-arg opts 'entry-point-arg)))
                     ('rpm
                      (list #:relocatable? relocatable?
                            #:prein-file
                            (process-file-arg opts 'prein-file)
                            #:postin-file
                            (process-file-arg opts 'postin-file)
                            #:preun-file
                            (process-file-arg opts 'preun-file)
                            #:postun-file
                            (process-file-arg opts 'postun-file)))
                     (_ '())))

could work?  build-docker-image already accepts a list as an #:entry-point, so
it should be just a matter of properly composing the list.

Not sure if this is helpful.

W.

> 
> Graham
> 
> On Fri, 2 Jun 2023 at 09:13, Graham Addis <grahamjamesaddis@gmail.com> wrote:
> >
> > Hi Wolf,
> >
> > On Thu, 1 Jun 2023 at 22:55, wolf <wolf@wolfsden.cz> wrote:
> > >
> > > On 2023-05-31 18:47:03 +0100, Graham Addis wrote:
> > > > I could use the equivalent format for --entry-point
> > > >
> > > > --entry-point executable --entry-point param1 --entry-point param2
> > > >
> > > I think that is a reasonable idea.  Only downside is that it would not be
> > > backwards compatible (since currently last --entry-point wins), however I am not
> > > sure if someone actually relies on that behavior.
> > >
> > > Backwards compatible way would be keeping --entry-point as it is and introducing
> > > new argument (--entry-point-arg) that could be used to build the argument list,
> > > but I might be overthinking it :).
> >
> > I'll go with extending --entry-point to accept multiple values and use
> > all of them for --format=docker and simply use the last one provided
> > for the other formats.
> >
> > I'll try to put a patch together at the weekend.
> >
> > Thanks for all your input, it really helps.
> >
> > Graham
> 

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: guix docker on gitlab-ci

[Graham Addis]

Hi Wolf,

Similar to what I was trying, which didn't work. I'll try to have a go
one evening and try and work out where I went wrong.

Thanks again. Graham

On Mon, 5 Jun 2023 at 18:35, wolf <wolf@wolfsden.cz> wrote:
>
> On 2023-06-05 16:37:50 +0100, Graham Addis wrote:
> > Hi Wolf,
> >
> > Not a particularly successful weekend...
> >
> > As --entry-point was used by other pack methods I thought the best
> > option would be to add a --docker-entry-point just for the docker
> > format and pass it into the build as an extra-option like for deb and
> > rpm formats.
> >
> > However I couldn't work out how to pass in a list via the
> > extra-options, so I'm a bit stuck.
> >
> > If there is anyone who knows their way around the pack scripts and can
> > point me in the right direction, that would help, but other than that
> > I'll probably get some more time next weekend.
>
> I did not try to implement this, so my guess might be completely off, but
> looking at how -S is implemented, I would suggest trying something like:
>
> 1. Introducing new %docker-format-options and friends (similar to already
>    existing %deb-format-options and friends), providing the --entry-point-arg
>    options, that would be specific to a docker format (although I am not sure if
>    it needs to be specific, maybe some other formats support arguments as
>    well?).
> 2. Writing entry-point-arg-spec-option-parser, while taking inspiration from
>    symlink-spec-option-parser.  Yours would be much simpler, since you would be
>    just accumulating values into a list.
>
> As for the extra-options, I guess modifying current code (by adding the 'docker
> option) into something like:
>
>     (extra-options (match pack-format
>                      ('deb
>                       (list #:control-file
>                             (process-file-arg opts 'control-file)
>                             #:postinst-file
>                             (process-file-arg opts 'postinst-file)
>                             #:triggers-file
>                             (process-file-arg opts 'triggers-file)))
>                      ('docker
>                       (list #:entry-point-args
>                             (process-file-arg opts 'entry-point-arg)))
>                      ('rpm
>                       (list #:relocatable? relocatable?
>                             #:prein-file
>                             (process-file-arg opts 'prein-file)
>                             #:postin-file
>                             (process-file-arg opts 'postin-file)
>                             #:preun-file
>                             (process-file-arg opts 'preun-file)
>                             #:postun-file
>                             (process-file-arg opts 'postun-file)))
>                      (_ '())))
>
> could work?  build-docker-image already accepts a list as an #:entry-point, so
> it should be just a matter of properly composing the list.
>
> Not sure if this is helpful.
>
> W.
>
> >
> > Graham
> >
> > On Fri, 2 Jun 2023 at 09:13, Graham Addis <grahamjamesaddis@gmail.com> wrote:
> > >
> > > Hi Wolf,
> > >
> > > On Thu, 1 Jun 2023 at 22:55, wolf <wolf@wolfsden.cz> wrote:
> > > >
> > > > On 2023-05-31 18:47:03 +0100, Graham Addis wrote:
> > > > > I could use the equivalent format for --entry-point
> > > > >
> > > > > --entry-point executable --entry-point param1 --entry-point param2
> > > > >
> > > > I think that is a reasonable idea.  Only downside is that it would not be
> > > > backwards compatible (since currently last --entry-point wins), however I am not
> > > > sure if someone actually relies on that behavior.
> > > >
> > > > Backwards compatible way would be keeping --entry-point as it is and introducing
> > > > new argument (--entry-point-arg) that could be used to build the argument list,
> > > > but I might be overthinking it :).
> > >
> > > I'll go with extending --entry-point to accept multiple values and use
> > > all of them for --format=docker and simply use the last one provided
> > > for the other formats.
> > >
> > > I'll try to put a patch together at the weekend.
> > >
> > > Thanks for all your input, it really helps.
> > >
> > > Graham
> >
>
> --
> There are only two hard things in Computer Science:
> cache invalidation, naming things and off-by-one errors.

Re: guix docker on gitlab-ci

[Graham Addis]

Hi Wolf,

Well I managed to sort it out last weekend.

I have added a --docker-entry-point option which can be invoked
multiple times to provide the docker EntryPoint value in exec form.

I have left the --entry-point behaviour alone except the
--docker-entry-point takes precedence.

I still have the docs to update, and to test the some of the different
cases, but it all seems to work.

Thanks again,

Graham

On Mon, 5 Jun 2023 at 22:38, Graham Addis <grahamjamesaddis@gmail.com> wrote:
>
> Hi Wolf,
>
> Similar to what I was trying, which didn't work. I'll try to have a go
> one evening and try and work out where I went wrong.
>
> Thanks again. Graham
>
> On Mon, 5 Jun 2023 at 18:35, wolf <wolf@wolfsden.cz> wrote:
> >
> > On 2023-06-05 16:37:50 +0100, Graham Addis wrote:
> > > Hi Wolf,
> > >
> > > Not a particularly successful weekend...
> > >
> > > As --entry-point was used by other pack methods I thought the best
> > > option would be to add a --docker-entry-point just for the docker
> > > format and pass it into the build as an extra-option like for deb and
> > > rpm formats.
> > >
> > > However I couldn't work out how to pass in a list via the
> > > extra-options, so I'm a bit stuck.
> > >
> > > If there is anyone who knows their way around the pack scripts and can
> > > point me in the right direction, that would help, but other than that
> > > I'll probably get some more time next weekend.
> >
> > I did not try to implement this, so my guess might be completely off, but
> > looking at how -S is implemented, I would suggest trying something like:
> >
> > 1. Introducing new %docker-format-options and friends (similar to already
> >    existing %deb-format-options and friends), providing the --entry-point-arg
> >    options, that would be specific to a docker format (although I am not sure if
> >    it needs to be specific, maybe some other formats support arguments as
> >    well?).
> > 2. Writing entry-point-arg-spec-option-parser, while taking inspiration from
> >    symlink-spec-option-parser.  Yours would be much simpler, since you would be
> >    just accumulating values into a list.
> >
> > As for the extra-options, I guess modifying current code (by adding the 'docker
> > option) into something like:
> >
> >     (extra-options (match pack-format
> >                      ('deb
> >                       (list #:control-file
> >                             (process-file-arg opts 'control-file)
> >                             #:postinst-file
> >                             (process-file-arg opts 'postinst-file)
> >                             #:triggers-file
> >                             (process-file-arg opts 'triggers-file)))
> >                      ('docker
> >                       (list #:entry-point-args
> >                             (process-file-arg opts 'entry-point-arg)))
> >                      ('rpm
> >                       (list #:relocatable? relocatable?
> >                             #:prein-file
> >                             (process-file-arg opts 'prein-file)
> >                             #:postin-file
> >                             (process-file-arg opts 'postin-file)
> >                             #:preun-file
> >                             (process-file-arg opts 'preun-file)
> >                             #:postun-file
> >                             (process-file-arg opts 'postun-file)))
> >                      (_ '())))
> >
> > could work?  build-docker-image already accepts a list as an #:entry-point, so
> > it should be just a matter of properly composing the list.
> >
> > Not sure if this is helpful.
> >
> > W.
> >
> > >
> > > Graham
> > >
> > > On Fri, 2 Jun 2023 at 09:13, Graham Addis <grahamjamesaddis@gmail.com> wrote:
> > > >
> > > > Hi Wolf,
> > > >
> > > > On Thu, 1 Jun 2023 at 22:55, wolf <wolf@wolfsden.cz> wrote:
> > > > >
> > > > > On 2023-05-31 18:47:03 +0100, Graham Addis wrote:
> > > > > > I could use the equivalent format for --entry-point
> > > > > >
> > > > > > --entry-point executable --entry-point param1 --entry-point param2
> > > > > >
> > > > > I think that is a reasonable idea.  Only downside is that it would not be
> > > > > backwards compatible (since currently last --entry-point wins), however I am not
> > > > > sure if someone actually relies on that behavior.
> > > > >
> > > > > Backwards compatible way would be keeping --entry-point as it is and introducing
> > > > > new argument (--entry-point-arg) that could be used to build the argument list,
> > > > > but I might be overthinking it :).
> > > >
> > > > I'll go with extending --entry-point to accept multiple values and use
> > > > all of them for --format=docker and simply use the last one provided
> > > > for the other formats.
> > > >
> > > > I'll try to put a patch together at the weekend.
> > > >
> > > > Thanks for all your input, it really helps.
> > > >
> > > > Graham
> > >
> >
> > --
> > There are only two hard things in Computer Science:
> > cache invalidation, naming things and off-by-one errors.

Re: guix docker on gitlab-ci

[Graham Addis]

Hi Wolf,

Patch is now submitted, see following for details.
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=64171
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=64173

Now we find out what I forgot. ;-)

Thanks again for your help,

Graham

On Tue, 13 Jun 2023 at 17:56, Graham Addis <grahamjamesaddis@gmail.com> wrote:
>
> Hi Wolf,
>
> Well I managed to sort it out last weekend.
>
> I have added a --docker-entry-point option which can be invoked
> multiple times to provide the docker EntryPoint value in exec form.
>
> I have left the --entry-point behaviour alone except the
> --docker-entry-point takes precedence.
>
> I still have the docs to update, and to test the some of the different
> cases, but it all seems to work.
>
> Thanks again,
>
> Graham
>
> On Mon, 5 Jun 2023 at 22:38, Graham Addis <grahamjamesaddis@gmail.com> wrote:
> >
> > Hi Wolf,
> >
> > Similar to what I was trying, which didn't work. I'll try to have a go
> > one evening and try and work out where I went wrong.
> >
> > Thanks again. Graham
> >
> > On Mon, 5 Jun 2023 at 18:35, wolf <wolf@wolfsden.cz> wrote:
> > >
> > > On 2023-06-05 16:37:50 +0100, Graham Addis wrote:
> > > > Hi Wolf,
> > > >
> > > > Not a particularly successful weekend...
> > > >
> > > > As --entry-point was used by other pack methods I thought the best
> > > > option would be to add a --docker-entry-point just for the docker
> > > > format and pass it into the build as an extra-option like for deb and
> > > > rpm formats.
> > > >
> > > > However I couldn't work out how to pass in a list via the
> > > > extra-options, so I'm a bit stuck.
> > > >
> > > > If there is anyone who knows their way around the pack scripts and can
> > > > point me in the right direction, that would help, but other than that
> > > > I'll probably get some more time next weekend.
> > >
> > > I did not try to implement this, so my guess might be completely off, but
> > > looking at how -S is implemented, I would suggest trying something like:
> > >
> > > 1. Introducing new %docker-format-options and friends (similar to already
> > >    existing %deb-format-options and friends), providing the --entry-point-arg
> > >    options, that would be specific to a docker format (although I am not sure if
> > >    it needs to be specific, maybe some other formats support arguments as
> > >    well?).
> > > 2. Writing entry-point-arg-spec-option-parser, while taking inspiration from
> > >    symlink-spec-option-parser.  Yours would be much simpler, since you would be
> > >    just accumulating values into a list.
> > >
> > > As for the extra-options, I guess modifying current code (by adding the 'docker
> > > option) into something like:
> > >
> > >     (extra-options (match pack-format
> > >                      ('deb
> > >                       (list #:control-file
> > >                             (process-file-arg opts 'control-file)
> > >                             #:postinst-file
> > >                             (process-file-arg opts 'postinst-file)
> > >                             #:triggers-file
> > >                             (process-file-arg opts 'triggers-file)))
> > >                      ('docker
> > >                       (list #:entry-point-args
> > >                             (process-file-arg opts 'entry-point-arg)))
> > >                      ('rpm
> > >                       (list #:relocatable? relocatable?
> > >                             #:prein-file
> > >                             (process-file-arg opts 'prein-file)
> > >                             #:postin-file
> > >                             (process-file-arg opts 'postin-file)
> > >                             #:preun-file
> > >                             (process-file-arg opts 'preun-file)
> > >                             #:postun-file
> > >                             (process-file-arg opts 'postun-file)))
> > >                      (_ '())))
> > >
> > > could work?  build-docker-image already accepts a list as an #:entry-point, so
> > > it should be just a matter of properly composing the list.
> > >
> > > Not sure if this is helpful.
> > >
> > > W.
> > >
> > > >
> > > > Graham
> > > >
> > > > On Fri, 2 Jun 2023 at 09:13, Graham Addis <grahamjamesaddis@gmail.com> wrote:
> > > > >
> > > > > Hi Wolf,
> > > > >
> > > > > On Thu, 1 Jun 2023 at 22:55, wolf <wolf@wolfsden.cz> wrote:
> > > > > >
> > > > > > On 2023-05-31 18:47:03 +0100, Graham Addis wrote:
> > > > > > > I could use the equivalent format for --entry-point
> > > > > > >
> > > > > > > --entry-point executable --entry-point param1 --entry-point param2
> > > > > > >
> > > > > > I think that is a reasonable idea.  Only downside is that it would not be
> > > > > > backwards compatible (since currently last --entry-point wins), however I am not
> > > > > > sure if someone actually relies on that behavior.
> > > > > >
> > > > > > Backwards compatible way would be keeping --entry-point as it is and introducing
> > > > > > new argument (--entry-point-arg) that could be used to build the argument list,
> > > > > > but I might be overthinking it :).
> > > > >
> > > > > I'll go with extending --entry-point to accept multiple values and use
> > > > > all of them for --format=docker and simply use the last one provided
> > > > > for the other formats.
> > > > >
> > > > > I'll try to put a patch together at the weekend.
> > > > >
> > > > > Thanks for all your input, it really helps.
> > > > >
> > > > > Graham
> > > >
> > >
> > > --
> > > There are only two hard things in Computer Science:
> > > cache invalidation, naming things and off-by-one errors.

Re: guix docker on gitlab-ci

[wolf]

[-- Attachment #1: Type: text/plain, Size: 5741 bytes --]

On 2023-05-30 07:52:57 +0100, Graham Addis wrote:
> Hi Worf,
> 
> Thanks for the response, see below.
> 
> On Mon, 29 May 2023 at 20:41, wolf <wolf@wolfsden.cz> wrote:
> >
> > On 2023-05-24 18:04:47 +0100, Graham Addis wrote:
> > > Dear people,
> > >
> > > I tried to create a docker image to use in a gitlab-ci instance but it
> > > failed because I couldn't use --entry-point="bin/sh -l -c" or
> > > equivalent, basically the gitlab-runner complains that it can't run
> > > binaries.
> >
> > Would this be better using just bin/sh for the entry point and passing the -l
> > and -c as an arguments?
> 
> Probably, but I don't think that's an option in gitlab ci and anyway
> it would be nice to support the docker options.
> 
> > > I've managed to get it working by making some changes to guix/scripts/pack.scm
> > >
> > > Adding a fn in docker-image, just before the call to
> > > build-docker-image, to create a list from the string passed in from
> > > --entry-point="bin/sh -l -c"
> > >
> > >             (define (make-docker-exec-form prefix value)
> > >               (cond
> > >                ((equal? value '())
> > >                 '())
> > >                ((equal? prefix '())
> > >                 (string-split value #\space))
> > >                (else
> > >                 (let ((values (string-split value #\space)))
> > >                   (cons
> > >                    (string-append prefix "/" (car values))
> > >                    (cdr values))))))
> >
> > If I read this right (sorry, still somewhat new to guile), you basically split
> > the --entry-point argument on spaces and use those parts as separate values to
> > invoke, is that correct?  If so, how would you pass a binary that has space in
> > the name (joke example: `/bin/ba sh') into the entry-point?
> 
> Basically, yes, and you are right about the problem.
> 
> I looked through all the guix documentation I could find and the only
> other place I saw that a list was passed in an option was for URLs and
> they were separated by spaces.
> 
> > > And replacing the setting of entry-point in the build-docker-image call to:
> > >
> > >                                 #:entry-point (make-docker-exec-form
> > > #$profile #$entry-point)
> > >
> > > The call to build-docker-image takes a list for entry-point, and it
> > > all works fine as far as I can tell.
> > >
> > > Before I send in a patch, some questions:
> > >
> > > Am I missing something?
> > >
> > > Am I on the right track?
> >
> > In my opinion (which you are free to disagree with :) ), I think it would be
> > better to either have /bin/sh as an entry-point (and pass -l -c as arguments
> > when starting the container, if required) or create a wrapper script /bin/shlc
> > that would exec /bin/sh with correct arguments.
> 
> Yep, lots of possible workarounds, but it seems to me that it would be
> better spending the time adjusting the pack command to fit the spec.
> 
> > Few random ideas: Maybe the same format Containerfiles use for cmd and
> > entrypoint directives could be used?  Maybe the --entry-point could also (in
> > addition to a string) accept a list of strings (LISP list)?
> 
> Sounds good to me. Do you have a reference for the json for this? (Not
> a big deal as I think I've worked it out from the code, but it's
> always nice to have the specs...)
> 
> From the Dockerfile reference for ENTRYPOINT
> https://docs.docker.com/engine/reference/builder/#entrypoint there are
> two fomrs:
> 
> ENTRYPOINT ["executable", "param1", "param2"] # The exec form, which
> is the preferred form:
> 
> ENTRYPOINT command param1 param2 # The shell form:
> 
> To implement the shell form I'd need to update build-docker-image in
> guix/docker.scm
> https://git.savannah.gnu.org/cgit/guix.git/tree/guix/docker.scm#n139
> to take a string instead of/ as well as the list it currently takes.
> Then update docker-image in guix/scripts/pack.scm
> https://git.savannah.gnu.org/cgit/guix.git/tree/guix/scripts/pack.scm#n592
> 
> Invocation would then simply be --entry-point="command param1 param2"
> 
> To implement the exec form (preferred according to docker) I wouldn't
> need to touch guix/docker.scm, but I would probably need to change the
> parsing for --entry-point as well as updating docker-iimge.

I did not know Guix does not currently support the shell form.  In that light I
think it should not be implemented, since once your idea (arguments for entry
point) is implemented, it will be trivial for end-user to emulate it if so
desired.

> 
> I prefer the second option, for which all I need is some guidance on
> the option syntax
> 
> .e.g. --entry-point=["command", "param1", "param2"]
> 
> Suggestions please. :)
> 
> I could implement both and test for a string or a list and choose
> between the shell and exec forms from there, but to be consistent with
> the existing implementation.
> 
> Once I'm clear about the best approach for this, I could add the CMD
> too, if that would be useful.
> https://docs.docker.com/engine/reference/builder/#cmd
> 
> One strange thing, I couldn't see the need for prefixing the profile
> to the ENTRYPOINT command:
> https://git.savannah.gnu.org/cgit/guix.git/tree/guix/scripts/pack.scm#n670
> I took it out and everything seems to work, so I'm not sure what
> problem it is fixing. Anybody any idea?

Wild guess, but it might depend on your container runtime (whether it uses execv
or execvp). Absolute path feels somewhat more robust.

> 
> Thanks,
> 
> Graham
>

W.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: guix docker on gitlab-ci

[Graham Addis]

On Thu, 1 Jun 2023 at 23:04, wolf <wolf@wolfsden.cz> wrote:
> I did not know Guix does not currently support the shell form.  In that light I
> think it should not be implemented, since once your idea (arguments for entry
> point) is implemented, it will be trivial for end-user to emulate it if so
> desired.

That was my thinking too.

> > One strange thing, I couldn't see the need for prefixing the profile
> > to the ENTRYPOINT command:
> > https://git.savannah.gnu.org/cgit/guix.git/tree/guix/scripts/pack.scm#n670
> > I took it out and everything seems to work, so I'm not sure what
> > problem it is fixing. Anybody any idea?
>
> Wild guess, but it might depend on your container runtime (whether it uses execv
> or execvp). Absolute path feels somewhat more robust.

Ah. I'll look into that, and not touch it just yet...