bug#63276: Allow channels to depend on a past Guix revision / private dependencies

bug#63276: Allow channels to depend on a past Guix revision / private dependencies

[Maxim Cournoyer]

Hi,

Recently (after the last core-updates merge), I've had the following
failure on 'guix pull', caused by my 'sfl-guix-channel' channel [0]:

[0]  https://gitlab.com/Apteryks/sfl-guix-channel

--8<---------------cut here---------------start------------->8---
$ guix pull
[...]
\builder for `/gnu/store/zliavnqgx3jxp8yc44nn1y34b3bjvlcw-guix-package-cache.drv' failed to produce output path `/gnu/store/0m2c2zrphqpmfhjb60k85yhfq704ay5r-guix-package-cache'
la compilation de /gnu/store/zliavnqgx3jxp8yc44nn1y34b3bjvlcw-guix-package-cache.drv a échoué
conseil : This usually indicates a bug in one of the channels you are pulling from, or some incompatibility
among them.  You can check the build log and report the issue to the channel developers.

The channels you are pulling from are: guix sfl-packages.

Vous trouverez le journal de compilation dans « /var/log/guix/drvs/zl/iavnqgx3jxp8yc44nn1y34b3bjvlcw-guix-package-cache.drv ».
cannot build derivation `/gnu/store/v5rszp4m1vv7zf20vrb1sghi4528d40j-profile.drv': 1 dependencies couldn't be built
guix pull: erreur : build of `/gnu/store/v5rszp4m1vv7zf20vrb1sghi4528d40j-profile.drv' failed

$ tail /var/log/guix/drvs/zl/iavnqgx3jxp8yc44nn1y34b3bjvlcw-guix-package-cache.drv
  1694:48  4 (thunk)
In sfl/packages/sflvault.scm:
    75:29  3 (propagated-inputs #<package python-sflvault-common@0.9?>)
In ice-9/boot-9.scm:
  1685:16  2 (raise-exception _ #:continuable? _)
  1780:13  1 (_ #<&compound-exception components: (#<&undefined-vari?>)
In unknown file:
           0 (backtrace #<undefined>)

(exception unbound-variable (value #f) (value "Unbound variable: ~S")
(value (python-pycrypto)) (value #f))
--8<---------------cut here---------------end--------------->8---

This is caused by the use of the 'python-pycrypto' package in the
channel, removed in commit a55e18f17cc82a01c11d03bdfb693c62cb068d5c.

It seems a valid use case to have a channel that depends on an old Guix
version.  Should this be supported?

If I could for example use the following channel dependency file at the
level of the channel in a .guix-channel, to depend on an older Guix
revision:

--8<---------------cut here---------------start------------->8---
(channel
 (version 0)
 (dependencies
  (channel
   (inherit %default-guix-channel)
   (commit "9ed65e6af77893b658a7159b091b5002892c2f95"))))
--8<---------------cut here---------------end--------------->8---

And have this used by the channel *privately*, it seems it'd solve the
problem, right?

I tried 'guix pull' from the channel using the following
~/.config/guix/channels.scm file with the above .guix-channel file
installed to the ~/src/sfl-guix-channel checkout:

--8<---------------cut here---------------start------------->8---
(cons (channel
       (name 'sfl-packages)
       ;;       (url "https://gitlab.com/Apteryks/sfl-guix-channel")
       (url "file:///home/maxim/src/sfl-guix-channel"))
      %default-channels)
--8<---------------cut here---------------end--------------->8---

But I got:

--8<---------------cut here---------------start------------->8---
$ guix pull
Mise à jour du canal « sfl-packages » depuis le dépôt Git « file:///home/maxim/src/sfl-guix-channel »...
Backtrace:
          18 (primitive-load "/home/maxim/.config/guix/current/bin/g…")
In guix/ui.scm:
   2300:7 17 (run-guix . _)
  2263:10 16 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  1752:10 15 (with-exception-handler _ _ #:unwind? _ # _)
  1747:15 14 (with-exception-handler #<procedure 7f64cc5c71e0 at ic…> …)
  1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
In guix/store.scm:
   659:37 12 (thunk)
In guix/status.scm:
    839:4 11 (call-with-status-report _ _)
In guix/store.scm:
   1321:3 10 (_)
   1298:8  9 (call-with-build-handler #<procedure 7f64cc5de690 at g…> …)
In guix/scripts/pull.scm:
   862:26  8 (_)
In guix/channels.scm:
    528:7  7 (loop _ _)
In guix/combinators.scm:
    48:26  6 (fold2 #<procedure 7f64cdcc66c0 at guix/channels.scm:5…> …)
In guix/channels.scm:
   560:37  5 (_ #<<channel> name: sfl-packages url: "file:///home/m…> …)
    528:7  4 (loop _ _)
In guix/combinators.scm:
    48:26  3 (fold2 #<procedure 7f64cdcc65a0 at guix/channels.scm:5…> …)
In guix/channels.scm:
   534:29  2 (_ #f () ())
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure struct-vtable: Wrong type argument in position 1 (expecting struct): #f
--8<---------------cut here---------------end--------------->8---

As a workaround, I can define a 'python-pycryto*' in the channel itself,
although that's kind of silly because it can only be used with a Guix
inferior pegged to commit 9ed65e6af77893b658a7159b091b5002892c2f95,
which does contain 'python-pycryto'.

Thoughts?

-- 
Thanks,
Maxim

bug#63276: Allow channels to depend on a past Guix revision / private dependencies

[Simon Tournier]

Hi Maxim,

Well, part of the message is in: :-)

    https://issues.guix.gnu.org/msgid/875y987z1m.fsf@gmail.com

On Thu, 04 May 2023 at 11:52, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

> It seems a valid use case to have a channel that depends on an old Guix
> version.  Should this be supported?
>
> If I could for example use the following channel dependency file at the
> level of the channel in a .guix-channel, to depend on an older Guix
> revision:
>
> --8<---------------cut here---------------start------------->8---
> (channel
>  (version 0)
>  (dependencies
>   (channel
>    (inherit %default-guix-channel)
>    (commit "9ed65e6af77893b658a7159b091b5002892c2f95"))))
> --8<---------------cut here---------------end--------------->8---

You want complete channel depends on a previous Guix, right?

Somehow, it would become equivalent to this channels.scm

        (list (channel
                (name 'guix)
                (url "https://git.savannah.gnu.org/git/guix.git")
                (branch "master")
                (commit
                  "9ed65e6af77893b658a7159b091b5002892c2f95"))
              (channel
                (name 'sfl)
                (url "file:///tmp/sfl-guix-channel")
                (branch "master"))))

and then run “guix pull && guix build sflvault-client” or:

    guix time-machine -C channels.scm -- shell sflvault-client


Well, I do not know if it is desirable.  Most of the time, I only want
one specific package from one specific Guix revision.


> As a workaround, I can define a 'python-pycryto*' in the channel itself,
> although that's kind of silly because it can only be used with a Guix
> inferior pegged to commit 9ed65e6af77893b658a7159b091b5002892c2f95,
> which does contain 'python-pycryto'.

Well, I do not know if we are using the time-travel the same way. :-)

Considering this:

        (define-public foo
          (package
            (name "foo")
            (inputs
             (list bar)
             (list baz))))

Most of the time, I want to build ’foo’ using a recent Guix but that
recent Guix removed ’bar’ so I want to pick it up from an inferior.
And let say I want ’baz’ from another Guix revision because some
specific version of ’baz’ is required for building ’foo’.

Basically, I am tempted to define the symbol ’bar’ and ’baz’ in my
channel and bind them to some inferior packages (here from 2 Guix
revisions).

Well, the number of inferiors should not be too much.  And it requires
some care about the propagated inputs.


Cheers,
simon

bug#63276: Allow channels to depend on a past Guix revision / private dependencies

[Maxim Cournoyer]

Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi Maxim,
>
> Well, part of the message is in: :-)
>
>     https://issues.guix.gnu.org/msgid/875y987z1m.fsf@gmail.com

Oh, a Mumi reference to a message ID!  I didn't know it supported that,
cool!

> On Thu, 04 May 2023 at 11:52, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> It seems a valid use case to have a channel that depends on an old Guix
>> version.  Should this be supported?
>>
>> If I could for example use the following channel dependency file at the
>> level of the channel in a .guix-channel, to depend on an older Guix
>> revision:
>>
>> --8<---------------cut here---------------start------------->8---
>> (channel
>>  (version 0)
>>  (dependencies
>>   (channel
>>    (inherit %default-guix-channel)
>>    (commit "9ed65e6af77893b658a7159b091b5002892c2f95"))))
>> --8<---------------cut here---------------end--------------->8---
>
> You want complete channel depends on a previous Guix, right?

That's the idea I had yes, seeing that my channel won't work with any
newer Guix revision, I thought I should be able to declare that upfront
as a dependency, and have the channels mechanism take care of treating
all things relating to this channel via a Guix inferior.  The benefit
above having to explain to users how to do this in a manifest as done in
[0] would be twofold:

1. The channel can simply be added and works out of the box, without
having users go through the hoops of configuring an inferior.

2. 'guix pull', if taught to translate a dependency on a past Guix into
an inferior, could use that at the time it runs ad avoid errors caused
by removed or moved packages in current Guix.

[0]  https://gitlab.com/Apteryks/sfl-guix-channel/-/blob/master/README.org

> Somehow, it would become equivalent to this channels.scm
>
>         (list (channel
>                 (name 'guix)
>                 (url "https://git.savannah.gnu.org/git/guix.git")
>                 (branch "master")
>                 (commit
>                   "9ed65e6af77893b658a7159b091b5002892c2f95"))
>               (channel
>                 (name 'sfl)
>                 (url "file:///tmp/sfl-guix-channel")
>                 (branch "master"))))
>
> and then run “guix pull && guix build sflvault-client” or:
>
>     guix time-machine -C channels.scm -- shell sflvault-client
>
>
> Well, I do not know if it is desirable.  Most of the time, I only want
> one specific package from one specific Guix revision.

Not exactly equivalent to that channel file.  In my idea (not thinking
about the technicalities/difficulties yet), the dependency on the Guix
channel would be made private to the package (my translating it to a
Guix inferior as mentioned above), instead of spilling into the global
package namespace (which I agree would be undesirable!).

In other words, declaring a dependency on a prior Guix channel would
cause all derivations for packages in that channel to happen in a
corresponding Guix inferior.  Does that make sense?

>
>> As a workaround, I can define a 'python-pycryto*' in the channel itself,
>> although that's kind of silly because it can only be used with a Guix
>> inferior pegged to commit 9ed65e6af77893b658a7159b091b5002892c2f95,
>> which does contain 'python-pycryto'.
>
> Well, I do not know if we are using the time-travel the same way. :-)
>
> Considering this:
>
>         (define-public foo
>           (package
>             (name "foo")
>             (inputs
>              (list bar)
>              (list baz))))
>
> Most of the time, I want to build ’foo’ using a recent Guix but that
> recent Guix removed ’bar’ so I want to pick it up from an inferior.
> And let say I want ’baz’ from another Guix revision because some
> specific version of ’baz’ is required for building ’foo’.
>
> Basically, I am tempted to define the symbol ’bar’ and ’baz’ in my
> channel and bind them to some inferior packages (here from 2 Guix
> revisions).

Interesting.  So using inferiors inside your channel does work in
general, contrary to experiments made with the sfl-guix-channel in the
other thread?

-- 
Thanks,
Maxim

bug#63276: Allow channels to depend on a past Guix revision / private dependencies

[Maxim Cournoyer]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi Simon,
>
> Simon Tournier <zimon.toutoune@gmail.com> writes:
>
>> Hi Maxim,
>>
>> Well, part of the message is in: :-)
>>
>>     https://issues.guix.gnu.org/msgid/875y987z1m.fsf@gmail.com
>
> Oh, a Mumi reference to a message ID!  I didn't know it supported that,
> cool!
>
>> On Thu, 04 May 2023 at 11:52, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>>
>>> It seems a valid use case to have a channel that depends on an old Guix
>>> version.  Should this be supported?
>>>
>>> If I could for example use the following channel dependency file at the
>>> level of the channel in a .guix-channel, to depend on an older Guix
>>> revision:
>>>
>>> --8<---------------cut here---------------start------------->8---
>>> (channel
>>>  (version 0)
>>>  (dependencies
>>>   (channel
>>>    (inherit %default-guix-channel)
>>>    (commit "9ed65e6af77893b658a7159b091b5002892c2f95"))))
>>> --8<---------------cut here---------------end--------------->8---
>>
>> You want complete channel depends on a previous Guix, right?
>
> That's the idea I had yes, seeing that my channel won't work with any
> newer Guix revision, I thought I should be able to declare that upfront
> as a dependency, and have the channels mechanism take care of treating
> all things relating to this channel via a Guix inferior.  The benefit
> above having to explain to users how to do this in a manifest as done in
> [0] would be twofold:
>
> 1. The channel can simply be added and works out of the box, without
> having users go through the hoops of configuring an inferior.
>
> 2. 'guix pull', if taught to translate a dependency on a past Guix into
> an inferior, could use that at the time it runs ad avoid errors caused
> by removed or moved packages in current Guix.
>
> [0]  https://gitlab.com/Apteryks/sfl-guix-channel/-/blob/master/README.org
>
>> Somehow, it would become equivalent to this channels.scm
>>
>>         (list (channel
>>                 (name 'guix)
>>                 (url "https://git.savannah.gnu.org/git/guix.git")
>>                 (branch "master")
>>                 (commit
>>                   "9ed65e6af77893b658a7159b091b5002892c2f95"))
>>               (channel
>>                 (name 'sfl)
>>                 (url "file:///tmp/sfl-guix-channel")
>>                 (branch "master"))))
>>
>> and then run “guix pull && guix build sflvault-client” or:
>>
>>     guix time-machine -C channels.scm -- shell sflvault-client
>>
>>
>> Well, I do not know if it is desirable.  Most of the time, I only want
>> one specific package from one specific Guix revision.
>
> Not exactly equivalent to that channel file.  In my idea (not thinking
> about the technicalities/difficulties yet), the dependency on the Guix
> channel would be made private to the package (my translating it to a
> Guix inferior as mentioned above), instead of spilling into the global
> package namespace (which I agree would be undesirable!).
>
> In other words, declaring a dependency on a prior Guix channel would
> cause all derivations for packages in that channel to happen in a
> corresponding Guix inferior.  Does that make sense?
>
>>
>>> As a workaround, I can define a 'python-pycryto*' in the channel itself,
>>> although that's kind of silly because it can only be used with a Guix
>>> inferior pegged to commit 9ed65e6af77893b658a7159b091b5002892c2f95,
>>> which does contain 'python-pycryto'.
>>
>> Well, I do not know if we are using the time-travel the same way. :-)
>>
>> Considering this:
>>
>>         (define-public foo
>>           (package
>>             (name "foo")
>>             (inputs
>>              (list bar)
>>              (list baz))))
>>
>> Most of the time, I want to build ’foo’ using a recent Guix but that
>> recent Guix removed ’bar’ so I want to pick it up from an inferior.
>> And let say I want ’baz’ from another Guix revision because some
>> specific version of ’baz’ is required for building ’foo’.
>>
>> Basically, I am tempted to define the symbol ’bar’ and ’baz’ in my
>> channel and bind them to some inferior packages (here from 2 Guix
>> revisions).
>
> Interesting.  So using inferiors inside your channel does work in
> general, contrary to experiments made with the sfl-guix-channel in the
> other thread?

The discussion has long died, and so has my original use case, but based
on what you had written I guess it could have been possible to add
code to my channel file so that all packages would have been computed
through an inferior.

I'll close this for now.

-- 
Thanks,
Maxim