Re: ‘guix pull’ and external dependencies

[ludo@gnu.org (Ludovic Courtès)]

Hello,

Chris Marusich <cmmarusich@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> So I think what we need to do is for “guix pull-ng” to build and install
>> a complete ‘guix’ package, and to manage it pretty much like other
>> packages is managed, 
>
> I think that's very reasonable.  It seems more intuitive than the
> current way 'guix pull' works.  I suspect that managing the installed
> version of Guix via the same Guix mechanisms that we use to manage any
> other package might be the best, most intuitive solution.
>
> Would it simplify the problem if we packaged the "Guix client stuff",
> the "Guix daemon stuff", and maybe the "Guix package definition stuff"
> separately?  Then a user could just install the "Guix client stuff"
> package if she wanted to upgrade the Guix client tools, or the "Guix
> package definition stuff" package if she wanted to get the latest
> package definitions.

It would be bad to separate package definitions from the rest because
they are very much intertwined: package definitions depend on the
definition of ‘package’, on build system implementations, and so on.

We could have guix-sans-daemon though, if that helps (which I suspect is
not the case).

>> except not in the user’s main profile (because that could lead to
>> undesirable behavior, 
>
> If we don't store Guix in the user's main profile, where would it go?

In “some sort of a profile” in ~/.config/guix/latest or similar.

> It's not clear to me why it's riskier to store Guix in a profile rather
> than outside the profile (but still in the store via the
> $HOME/.config/guix/latest symlink), which is what 'guix pull' does now.
> You seem to think it's riskier; I'm curious to know more about why.

There’s the manifest format change issue I mentioned, or the inability
to roll back if you install a broken Guix.

>> where upgrading Guix creates a new generation, 
>
> Why should upgrading Guix NOT create a new generation?  I thought that a
> new profile generation would be created any time you upgrade a package,
> and I thought that was a good thing because it facilitates easy,
> transactional roll-back.  Perhaps I'm missing something.

I’m suggesting that upgrading Guix creates a new generation (so we agree
here), just not in the user’s profile.

>> or, in theory, unrecoverable problems, where you cannot roll back
>> because previous generations use an old Guix that does not understand
>> the new manifest format.)
>
> Why would a change in manifest format be unrecoverable?  It looks like
> each profile generation contains a manifest file.  Assuming that the new
> Guix functions well enough to perform roll back, couldn't we just roll
> back to the previous profile generation, where we would have both (1)
> the old profile's manifest file, and (2) the previous Guix, which
> understands that format?  Since rolling back a profile is basically just
> a symlink flip, I think the new Guix could probably do that even if it
> didn't understand the old manifest format.

Yeah, I think you’re right.  :-)

In general, I think my concern is more that we cannot promise that
downgrading Guix will work, considering the potential for on-disk format
changes.  It’s a bit theoretical, but not entirely sci-fi either.

>> The difficulty is that ./configure && make && make install in Guix takes
>> some time, and we probably wouldn’t want to do that on each ‘guix pull’
>> invocation (esp. with Guile 2.2’s compilation times.)
>>
>> So we may have to provide substitutes of Guix itself, and arrange so
>> that ‘guix pull’ pulls up to a tag for which we have substitutes.
>
> What if we had a special package version of Guix (e.g., "v0.11.0") which
> we kept up to date via some mechanism?  Maybe something as simple as a
> Git hook could help increase the likelihood of that version being
> substitutable.  For example, we could have a Git hook that prevents
> someone from checking in a change if the latest Git tag does not
> correspond to a Guix package version.  Maybe we can do better.

Right, we could do something like that.  There are still non-zero
chances that someone running ‘guix pull’ at an arbitrary point in time
will have to build locally, which is not great.

> I actually think it would be a good thing if we can run "guix pull"
> without substitutes available.  But it should use a substitute by
> default, and "build from source" should be a fallback mechanism that the
> user has to explicitly request, just like when installing new packages.
> That would help avoid unexpectedly long "guix pull" invocations.

Yes, using substitutes or falling back to source builds is always the
default.

Thanks for your feedback!

Ludo’.