How to install GWL?

How to install GWL?

[Ricardo Wurmus]

Hi,

the GWL is intended to be an extension of Guix.  It uses Guix in two
ways:

1) it uses modules provided by Guix as one would use a library.  These
include (guix gexp), (guix derivations), (guix monads), (guix store),
etc.

2) it uses Guix to install packages at runtime based on whatever
workflow a user asks to be run.

The “gwl” package has the “guix” package among its inputs due to 1).
This version of Guix will always be somewhat old, and older than the
version of Guix used to install the GWL.  This is okay for using Guix
modules, but it wouldn’t be okay for 2).

How should the GWL be installed for maximum convenience and
compatibility?  Does it make sense to install it as a channel so that it
is tied to the user’s current version of Guix?  That would be pretty
awkward and less convenient than just typing “guix install gwl”.

If we stick with installing the workflow language as a package, how
should package installation be handled?  Should all workflows require a
channels definition for reproducibility, so that we could instantiate an
inferior Guix using the exact specified version?  If none is provided we
could fall back to the latest version of Guix.

How does that sound?

--
Ricardo

Re: How to install GWL?

[Kyle Meyer]

Hi Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> How should the GWL be installed for maximum convenience and
> compatibility?  Does it make sense to install it as a channel so that it
> is tied to the user’s current version of Guix?  That would be pretty
> awkward and less convenient than just typing “guix install gwl”.
>
> If we stick with installing the workflow language as a package, how
> should package installation be handled?  Should all workflows require a
> channels definition for reproducibility, so that we could instantiate an
> inferior Guix using the exact specified version?  If none is provided we
> could fall back to the latest version of Guix.
>
> How does that sound?

The last combination sounds good to me: support/encourage specifying the
channel definition in the workflow, and fall back to the latest version
of Guix.  Though perhaps it'd be better to fall back to the user's
current version instead?  My thinking is that the user's version would
be a less frequently moving target, be more likely to have higher
substitute availability, and would more closely match any debugging and
testing the user is doing with, say, `guix environment'.

Re: How to install GWL?

[zimoun]

Hi Ricardo,

It is not clear in my mind and I have not looked at the code so my
words probably do not make sense.
Well, correct me if I am wrong. :-)


On Wed, 22 Jan 2020 at 22:56, Ricardo Wurmus <rekado@elephly.net> wrote:

> 1) it uses modules provided by Guix as one would use a library.  These
> include (guix gexp), (guix derivations), (guix monads), (guix store),
> etc.
>
> 2) it uses Guix to install packages at runtime based on whatever
> workflow a user asks to be run.
>
> The “gwl” package has the “guix” package among its inputs due to 1).
> This version of Guix will always be somewhat old, and older than the
> version of Guix used to install the GWL.  This is okay for using Guix
> modules, but it wouldn’t be okay for 2).

More precisely, GWL needs the Guix Scheme library to build; quickly
speaking the symbols under the folder 'guix/'.
Even if it remains some work, we can assume that the API is stable.

Well, the issue comes from the version of the toolchain used to build GWL.

Let consider the package definition of GWL depends on the
commit/version A of Guix.
And this GWL package lives in the Guix commit B.
So all the machinery -- Guile, compilers, bootstrap, Guix builders
themself -- will use the commit B to build GWL but will compile the
commit A of the Guix library.

My first question is: when running GWL in the state B, from which
commit will the packages come from? The packages defined in the
library --commit A-- _or_ the packages in the current state --commit
B--.
And if from commit A, which toolchain is used to build the packages:
the one from A or from B?

Currently, how is it working?


> How should the GWL be installed for maximum convenience and
> compatibility?  Does it make sense to install it as a channel so that it
> is tied to the user’s current version of Guix?  That would be pretty
> awkward and less convenient than just typing “guix install gwl”.

As a user, I expect that "guix workflow" uses all the packages in the
current Guix state I am. Because it is how all "guix <command>" works.


What I naively would think it works:

1. commit B
   $ guix install gwl
=> populate the store and compile GWL with the toolchain of the commit
B using the Scheme library at commit A (as an end-user I am not to ask
myself which version of the library :-)).

2. $ guix worklow <stuff>
=> use the packages and toolchain of commit B
i.e., the Guix living in ~/.config/guix/current

3. commit C
   $ guix pull

4. $ guix workflow <other-stuff>
=> recompile workflow with the Scheme library at the state C using the
toolchain of C and so add an entry to the store. Then use the packages
at C too.

Well, the version of the Scheme library does not matter so much. But
all the version of toolchain matter when speaking about
reproducibility.


Considering classical packages, installing at commit B, then pulling,
then running is not confusing, I mean: after the pull, 'guix describe'
will return commit C and every 'guix <command>' will use this commit
C. And I know that if I use a package installed before the pull, then
I run the "old" version because I know that I need to upgrade to run
the new one. The confusion is the term 'guix' in 'guix workflow' so I
expect that the <command> 'namely workflow' acts as any other
<command>.

An easy solution to fix this kind of confusion is to rename "guix
workflow" as simple "gwl"  or whatever else.


> If we stick with installing the workflow language as a package, how
> should package installation be handled?  Should all workflows require a
> channels definition for reproducibility, so that we could instantiate an
> inferior Guix using the exact specified version?  If none is provided we
> could fall back to the latest version of Guix.

It is too late to have a clear mind. :-)
My opinion is that rename "guix worklow" as simply "gwl" would ease
the installation and reproducibility process. Deal with GWL as any
other package.
And this "gwl" command would use the package of the current Guix state
including hypothetical channels.

For example, it would reduce the head itching when using "guix time-machine".

I am not sure if my input is relevant.


Thank you for raising up this and reactivate this mailing list. :-)

Cheers,
simon

Re: How to install GWL?

[Ricardo Wurmus]

Kyle Meyer <kyle@kyleam.com> writes:

> Hi Ricardo,
>
> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> How should the GWL be installed for maximum convenience and
>> compatibility?  Does it make sense to install it as a channel so that it
>> is tied to the user’s current version of Guix?  That would be pretty
>> awkward and less convenient than just typing “guix install gwl”.
>>
>> If we stick with installing the workflow language as a package, how
>> should package installation be handled?  Should all workflows require a
>> channels definition for reproducibility, so that we could instantiate an
>> inferior Guix using the exact specified version?  If none is provided we
>> could fall back to the latest version of Guix.
>>
>> How does that sound?
>
> The last combination sounds good to me: support/encourage specifying the
> channel definition in the workflow, and fall back to the latest version
> of Guix.  Though perhaps it'd be better to fall back to the user's
> current version instead?  My thinking is that the user's version would
> be a less frequently moving target, be more likely to have higher
> substitute availability, and would more closely match any debugging and
> testing the user is doing with, say, `guix environment'.

Yes, I agree.  I suppose we could achieve this by simply spawning “guix
repl” and rely on the environment to give us the user’s current version
of Guix.  Then we connect to the spawned REPL.

-- 
Ricardo

Re: How to install GWL?

[Roel Janssen]

Hi Ricardo,

On Wed, 2020-01-22 at 22:55 +0100, Ricardo Wurmus wrote:
> Hi,
> 
> the GWL is intended to be an extension of Guix.  It uses Guix in two
> ways:
> 
> 1) it uses modules provided by Guix as one would use a
> library.  These
> include (guix gexp), (guix derivations), (guix monads), (guix store),
> etc.
> 
> 2) it uses Guix to install packages at runtime based on whatever
> workflow a user asks to be run.
> 
> The “gwl” package has the “guix” package among its inputs due to 1).
> This version of Guix will always be somewhat old, and older than the
> version of Guix used to install the GWL.  This is okay for using Guix
> modules, but it wouldn’t be okay for 2).

I initially wanted the GWL to be part of Guix, and I think that could
greatly simplify things.  So by installing GWL, you get the "workflow"
and "process" subcommands that work with the version of Guix returned
by:
$ guix --version

This would require us to package GWL in such a way that it doesn't have
Guix as one of its inputs.  Is that possible?

> How should the GWL be installed for maximum convenience and
> compatibility?  Does it make sense to install it as a channel so that
> it
> is tied to the user’s current version of Guix?  That would be pretty
> awkward and less convenient than just typing “guix install gwl”.

Or.. we merge the code from GWL into Guix, so it's automatically there;
no install needed.

I think the code is quite lightweight, and since it uses the Guix
modules, it is somewhat tied to a specific version of Guix.

What's the reason for not wanting GWL directly in Guix?

Kind regards,
Roel Janssen

Re: How to install GWL?

[zimoun]

Hi Roel,

On Thu, 23 Jan 2020 at 12:16, Roel Janssen <roel@gnu.org> wrote:

> I initially wanted the GWL to be part of Guix, and I think that could
> greatly simplify things.  So by installing GWL, you get the "workflow"
> and "process" subcommands that work with the version of Guix returned
> by:
> $ guix --version

I agree that with the current situation, it is confusing and hard to
know which "guix --version" is used when running "guix workflow".
It is what I tried to describe in my other email to this thread.


> What's the reason for not wanting GWL directly in Guix?

Maybe because it is proof-of-concept of "Guix is a Scheme library". :-)


All the best,
simon

Re: How to install GWL?

[Ricardo Wurmus]

zimoun <zimon.toutoune@gmail.com> writes:

> Hi Roel,
>
> On Thu, 23 Jan 2020 at 12:16, Roel Janssen <roel@gnu.org> wrote:
>
>> I initially wanted the GWL to be part of Guix, and I think that could
>> greatly simplify things.  So by installing GWL, you get the "workflow"
>> and "process" subcommands that work with the version of Guix returned
>> by:
>> $ guix --version
>
> I agree that with the current situation, it is confusing and hard to
> know which "guix --version" is used when running "guix workflow".
> It is what I tried to describe in my other email to this thread.

I believe that by spawning “guix repl” and using it as an inferior we
can remove the confusion, and we would make the separation of what is
library code and what is run-time code clearer.

Currently, that’s not what we’re doing.  Instead we use Guix as a
library (whatever version of the “guix” package is available in the
version of Guix used), and *also* use that version to install software.
That’s clearly not desirable, hence my email.

--
Ricardo

Re: How to install GWL?

[Ricardo Wurmus]

Roel Janssen <roel@gnu.org> writes:

> This would require us to package GWL in such a way that it doesn't have
> Guix as one of its inputs.  Is that possible?

I don’t think so.  I suppose the only way to avoid having Guix as an
input is by installing the GWL as a channel when “guix pull” is run, but
that route is fraught with problems.  Guix is just *one* of a number of
inputs, and it is not clear how to install all of them via “guix pull” —
nor would that be desirable.

>> How should the GWL be installed for maximum convenience and
>> compatibility?  Does it make sense to install it as a channel so that
>> it
>> is tied to the user’s current version of Guix?  That would be pretty
>> awkward and less convenient than just typing “guix install gwl”.
>
> Or.. we merge the code from GWL into Guix, so it's automatically there;
> no install needed.
>
> I think the code is quite lightweight, and since it uses the Guix
> modules, it is somewhat tied to a specific version of Guix.
>
> What's the reason for not wanting GWL directly in Guix?

The code won’t remain lightweight forever.  The GWL may gain support for
more advanced caching of intermediate results through external
libraries; it may gain support for deployment to virtual machines in the
cloud (e.g. via Guile AWS); it may depend on (or include) grid engine
bindings to provide better support for cluster execution.  None of this
really belongs to Guix itself, which already has a very wide scope.

I’d rather see Guix move into the opposite direction and embrace
extensibility by providing a stable API for its core modules.  With the
GWL we have an opportunity to demonstrate how Guix can be extended
without having to actually be part of it.

--
Ricardo

Re: How to install GWL?

[zimoun]

Hi Ricardo,

On Thu, 23 Jan 2020 at 14:02, Ricardo Wurmus <rekado@elephly.net> wrote:

> I believe that by spawning “guix repl” and using it as an inferior we
> can remove the confusion, and we would make the separation of what is
> library code and what is run-time code clearer.

I think we agree on a clear separation between what is library code
and what is run-time code.
I trust you that "guix repl" should fit the needs.

My concern is:

 - 'guix describe' or 'guix --version' returns the current usable Guix
 - 'guix workflow' depends on the 2 versions of Guix when it was installed:
      + version of all the building machinery
      + and this version can be different of the version of Scheme
library as inputs
 - 'guix pull' modifies the result of 'guix describe' or 'guix --version'
 - then 'guix workflow' refers to 3 versions of Guix:
      + the 2 above
      + the current version from which the packages are fetched

This happens all the times when using any package and I am not
confused. Because the separation between install-time and run-time is
clear.

Note that the 2 first version are linked so it is not an issue and the
version of Guix when installing GWL is enough for reproducibility
because this very version implies the other one.

However, it still can be confusing because 2 versions of Guix are
required for reproducibility. The version to build GWL itself and the
version at run-time.
In 'guix workflow', the first term 'guix' refers to one version of
Guix and the second term 'workflow' refers to another one.


What you are proposing probably address well my concern. :-)


> Currently, that’s not what we’re doing.  Instead we use Guix as a
> library (whatever version of the “guix” package is available in the
> version of Guix used), and *also* use that version to install software.
> That’s clearly not desirable, hence my email.

I agree.
And I agree too that "inferior" seems the right approach.


Cheers,
simon

Re: How to install GWL?

[zimoun]

Hi Ricardo,

(+Ludo)

Yesterday evening, I asked the Ludo's opinion about how to deal with
package that depends on Guix as a library.
So his is CC'd. In case, he wants to make an input. ;-)


On Wed, 22 Jan 2020 at 22:56, Ricardo Wurmus <rekado@elephly.net> wrote:

> the GWL is intended to be an extension of Guix.  It uses Guix in two
> ways:
>
> 1) it uses modules provided by Guix as one would use a library.  These
> include (guix gexp), (guix derivations), (guix monads), (guix store),
> etc.
>
> 2) it uses Guix to install packages at runtime based on whatever
> workflow a user asks to be run.
>
> The “gwl” package has the “guix” package among its inputs due to 1).
> This version of Guix will always be somewhat old, and older than the
> version of Guix used to install the GWL.  This is okay for using Guix
> modules, but it wouldn’t be okay for 2).
>
> How should the GWL be installed for maximum convenience and
> compatibility?  Does it make sense to install it as a channel so that it
> is tied to the user’s current version of Guix?  That would be pretty
> awkward and less convenient than just typing “guix install gwl”.
>
> If we stick with installing the workflow language as a package, how
> should package installation be handled?  Should all workflows require a
> channels definition for reproducibility, so that we could instantiate an
> inferior Guix using the exact specified version?  If none is provided we
> could fall back to the latest version of Guix.
>
> How does that sound?

The difference between "as a package" --which implies inferiors-- or
"as a channel" is: embed vs extend.

I was convinced that "as a package" was more convenient. Now I am not sure. :-)
Because we know what "embed" means. ;-)

For example, Julien's home-manager [1] uses the "as a channel" approach.


[1] https://framagit.org/tyreunom/guix-home-manager



All the best,
simon

Re: How to install GWL?

[Ricardo Wurmus]

Hi zimoun,

> The difference between "as a package" --which implies inferiors-- or
> "as a channel" is: embed vs extend.
>
> I was convinced that "as a package" was more convenient. Now I am not sure. :-)
> Because we know what "embed" means. ;-)
>
> For example, Julien's home-manager [1] uses the "as a channel" approach.

The home-manager *only* requires Guix.  The GWL needs wisp,
guile-commonmark, guile-gcrypt, guile-pfds (for sets),
guile-syntax-highlight, graphviz (for visualizing workflows), etc.  In
the future it might also depend on the Guile AWS library, perhaps even
some libraries that implement different schedulers, etc.  I’ve also been
thinking about adding a web interface to visualize the current state of
processes spawned by the workflow, which should make debugging workflows
much easier.

Channels don’t allow for any fancy installation or setup.  The Guile
modules provided by a channel are compiled just in time when running
“guix pull”.  It doesn’t seem like a good idea for the GWL to hook into
the channels mechanism when it doesn’t provide what channels were
designed to help implement: extensions to the package graph.

I think inferiors also give GWL users a feature that would otherwise be
missing: to run arbitrary workflows with arbitrary channel definitions
via inferiors without having to first manually execute “guix pull” to
switch to the particular channels that a workflow requires.  Using
inferiors gives a superior user experience, in my opinion.

--
Ricardo

Re: How to install GWL?

[Ricardo Wurmus]

Ricardo Wurmus <rekado@elephly.net> writes:

> Kyle Meyer <kyle@kyleam.com> writes:
>
>> Hi Ricardo,
>>
>> Ricardo Wurmus <rekado@elephly.net> writes:
>>
>>> How should the GWL be installed for maximum convenience and
>>> compatibility?  Does it make sense to install it as a channel so that it
>>> is tied to the user’s current version of Guix?  That would be pretty
>>> awkward and less convenient than just typing “guix install gwl”.
>>>
>>> If we stick with installing the workflow language as a package, how
>>> should package installation be handled?  Should all workflows require a
>>> channels definition for reproducibility, so that we could instantiate an
>>> inferior Guix using the exact specified version?  If none is provided we
>>> could fall back to the latest version of Guix.
>>>
>>> How does that sound?
>>
>> The last combination sounds good to me: support/encourage specifying the
>> channel definition in the workflow, and fall back to the latest version
>> of Guix.  Though perhaps it'd be better to fall back to the user's
>> current version instead?  My thinking is that the user's version would
>> be a less frequently moving target, be more likely to have higher
>> substitute availability, and would more closely match any debugging and
>> testing the user is doing with, say, `guix environment'.
>
> Yes, I agree.  I suppose we could achieve this by simply spawning “guix
> repl” and rely on the environment to give us the user’s current version
> of Guix.  Then we connect to the spawned REPL.

This is all more confusing than I thought.

When a user installs the GWL, the Guile module “(guix scripts workflow)”
is installed.  If the GUILE_LOAD_PATH is not set to include the
directory holding this module then the GWL won’t be available.  Let’s
assume that GUILE_LOAD_PATH is in fact set to include the modules of the
GWL and its Guile dependencies (including an old version of Guix).

(guix scripts workflow) then provides the “guix workflow” sub-command to
whatever “guix” command is invoked by the user.  What about all of the
modules that the GWL uses?  They also need to be on the load path.  The
“guix” command arranges for its own modules to appear on the load path
first.  So when using the GWL with any version of Guix *that* variant’s
modules are going to be used.

This also includes Guile JSON, guile-sqlite, Guile Gcrypt,
Bytestructures, etc in versions that match the variant of Guix — and
that might conflict with versions used in the GWL.

The first takeaway is: we don’t actually need to use an inferior Guix
when all we want is for the GWL to use the current version of Guix.
(Adding support for inferiors is still a good idea, of course, because
it makes workflows more flexible and reproducible.)

The second takeaway is: when upgrading Guix with “guix pull” it is
possible that the GWL (installed by “guix install gwl”) breaks due to
API changes in Guix or even due to changes to the dependencies of Guix —
such as the Guile JSON upgrade recently.

This makes me think that extending Guix without some API *and*
dependency guarantees is a bit more fragile than I’d like it to be.  On
the other hand, the API has been rather stable in the past and there has
only been one incompatible upgrade (that of Guile JSON) that might have
affected the GWL.

I still think that the channels feature is the wrong deployment
mechanism for the GWL; I also still think that the GWL is too big to be
a part of Guix proper.  Perhaps the GWL needs to add more runtime checks
to ensure that the Guix modules it uses are available at runtime; any
references to individual packages (such as the Guile variant to use to
run scripts, or the Bash package used for shell wrappers) should also be
replaced with more future-proof lookups by package name.

It would be desirable to have Guix releases more often, so that the
difference between the version of Guix that was used to build it and the
effective version at runtime (due to “guix pull”) cannot grow too large.

--
Ricardo

Re: How to install GWL?

[Ludovic Courtès]

Hi there!

Ricardo Wurmus <rekado@elephly.net> skribis:

>> The difference between "as a package" --which implies inferiors-- or
>> "as a channel" is: embed vs extend.
>>
>> I was convinced that "as a package" was more convenient. Now I am not sure. :-)
>> Because we know what "embed" means. ;-)
>>
>> For example, Julien's home-manager [1] uses the "as a channel" approach.
>
> The home-manager *only* requires Guix.  The GWL needs wisp,
> guile-commonmark, guile-gcrypt, guile-pfds (for sets),
> guile-syntax-highlight, graphviz (for visualizing workflows), etc.  In
> the future it might also depend on the Guile AWS library, perhaps even
> some libraries that implement different schedulers, etc.  I’ve also been
> thinking about adding a web interface to visualize the current state of
> processes spawned by the workflow, which should make debugging workflows
> much easier.

Oh, channels in their current form wouldn’t be a good fit because they
don’t allow you to ask for additional dependencies.  Perhaps a problem
to solve, dunno.

> Channels don’t allow for any fancy installation or setup.  The Guile
> modules provided by a channel are compiled just in time when running
> “guix pull”.  It doesn’t seem like a good idea for the GWL to hook into
> the channels mechanism when it doesn’t provide what channels were
> designed to help implement: extensions to the package graph.

Yeah.

> I think inferiors also give GWL users a feature that would otherwise be
> missing: to run arbitrary workflows with arbitrary channel definitions
> via inferiors without having to first manually execute “guix pull” to
> switch to the particular channels that a workflow requires.  Using
> inferiors gives a superior user experience, in my opinion.

Indeed, that makes sense to me.

Simon and I were discussing and I was saying that this inferior
vs. channel discussion reminds me of extend vs. embed in the context of
programming languages like Guile (where inferiors are a bit like
“embedding”).

But yeah, I didn’t have a good understanding of the constraints here, so
thanks for clarifying!

Ludo’.

Re: How to install GWL?

[zimoun]

Hi Ricardo,

On Fri, 31 Jan 2020 at 12:07, Ricardo Wurmus <rekado@elephly.net> wrote:

> Channels don’t allow for any fancy installation or setup.  The Guile
> modules provided by a channel are compiled just in time when running
> “guix pull”.  It doesn’t seem like a good idea for the GWL to hook into
> the channels mechanism when it doesn’t provide what channels were
> designed to help implement: extensions to the package graph.

I got it. :-)


> I think inferiors also give GWL users a feature that would otherwise be
> missing: to run arbitrary workflows with arbitrary channel definitions
> via inferiors without having to first manually execute “guix pull” to
> switch to the particular channels that a workflow requires.  Using
> inferiors gives a superior user experience, in my opinion.

I understand.

Thank you for explaining. Now, it is all clear to me.


Cheers,
simon

Re: How to install GWL?

[zimoun]

Hi,

On Sat, 1 Feb 2020 at 12:50, Ludovic Courtès <ludo@gnu.org> wrote:

> Simon and I were discussing and I was saying that this inferior
> vs. channel discussion reminds me of extend vs. embed in the context of
> programming languages like Guile (where inferiors are a bit like
> “embedding”).
>
> But yeah, I didn’t have a good understanding of the constraints here, so
> thanks for clarifying!

It was an interesting discussion. :-)
Explaining is always good way to know if something is well-understood. ;-)
Well then I have read a bit about extend vs. embed and rehashed the
explanations by Ricardo. Now it is clarified.

Cheers,
simon

Re: How to install GWL?

[zimoun]

Hi Ricardo,

On Sat, 1 Feb 2020 at 10:27, Ricardo Wurmus <rekado@elephly.net> wrote:

> When a user installs the GWL, the Guile module “(guix scripts workflow)”
> is installed.  If the GUILE_LOAD_PATH is not set to include the
> directory holding this module then the GWL won’t be available.  Let’s
> assume that GUILE_LOAD_PATH is in fact set to include the modules of the
> GWL and its Guile dependencies (including an old version of Guix).
>
> (guix scripts workflow) then provides the “guix workflow” sub-command to
> whatever “guix” command is invoked by the user.  What about all of the
> modules that the GWL uses?  They also need to be on the load p

I am thinking loudly.
ath.  The
> “guix” command arranges for its own modules to appear on the load path
> first.  So when using the GWL with any version of Guix *that* variant’s
> modules are going to be used.

Yes, it is what I have tried to spot out previously in the thread. :-)
Re-reading, I horribly worded it so it was not-understandable. My bad! Sorry.


> This also includes Guile JSON, guile-sqlite, Guile Gcrypt,
> Bytestructures, etc in versions that match the variant of Guix — and
> that might conflict with versions used in the GWL.

Oh!
This is annoying... and hard to fix because it is the classical
dependency hell we know.


> The first takeaway is: we don’t actually need to use an inferior Guix
> when all we want is for the GWL to use the current version of Guix.

Not all Guix, only the current version of the packages, right?


> (Adding support for inferiors is still a good idea, of course, because
> it makes workflows more flexible and reproducible.)
>
> The second takeaway is: when upgrading Guix with “guix pull” it is
> possible that the GWL (installed by “guix install gwl”) breaks due to
> API changes in Guix or even due to changes to the dependencies of Guix —
> such as the Guile JSON upgrade recently.
>
> This makes me think that extending Guix without some API *and*
> dependency guarantees is a bit more fragile than I’d like it to be.  On
> the other hand, the API has been rather stable in the past and there has
> only been one incompatible upgrade (that of Guile JSON) that might have
> affected the GWL.

Yes, but there is no guarantee, I mean it should work "almost" all the
time. I am not sure to agree that an "almost" remains.


> I still think that the channels feature is the wrong deployment
> mechanism for the GWL; I also still think that the GWL is too big to be
> a part of Guix proper.  Perhaps the GWL needs to add more runtime checks
> to ensure that the Guix modules it uses are available at runtime; any
> references to individual packages (such as the Guile variant to use to
> run scripts, or the Bash package used for shell wrappers) should also be
> replaced with more future-proof lookups by package name.

I thought you proposed that GWL should run in an inferior (say 1)
fixing the correct Guix version that it needs to work and another
inferior (say 2) should populate the store with any other Guix
version. And because the store is stable, then GWL using inferior 1
does the expected job. Is it not what you have in mind when speaking
about inferiors? (plural at inferior ;-))

Do I miss something?


> It would be desirable to have Guix releases more often, so that the
> difference between the version of Guix that was used to build it and the
> effective version at runtime (due to “guix pull”) cannot grow too large.

We had a session at Guix Days about improving the release process.
Well, I do not remind that someone took some notes. Sadly.
Let drop an email on guix-devel to collect the feedback (Chris
(Baines), Ludo, Clément and others were there so we should be able to
reconstitute ;-))


All the best,
simon

Re: How to install GWL?

[Ricardo Wurmus]

zimoun <zimon.toutoune@gmail.com> writes:

>> This also includes Guile JSON, guile-sqlite, Guile Gcrypt,
>> Bytestructures, etc in versions that match the variant of Guix — and
>> that might conflict with versions used in the GWL.
>
> Oh!
> This is annoying... and hard to fix because it is the classical
> dependency hell we know.

I suppose we could work around this potential problem with an extra
step.  We provide the (guix scripts workflow) module, which Guix tries
to find, but it doesn’t mean that we have to keep the environment
untouched from then on.  That module could do all sorts of things
including the execution of a new Guile process with a suitable
environment.

But how would that affect the package versions that Guix provides if we
override the environment and essentially end up using an older Guix…?
We would then have to spawn an inferior Guix, which would really just be
the Guix that originally gave us control and not the one we’ve built
with…  But we can’t just use some Guix directory as an inferior —
there’s no support for that in (guix inferior), we need to go through a
derivation…

This sounds messy.

>> The first takeaway is: we don’t actually need to use an inferior Guix
>> when all we want is for the GWL to use the current version of Guix.
>
> Not all Guix, only the current version of the packages, right?

Right, but we do end up using modules from the current Guix, even though
these are not the modules we built with.

>> I still think that the channels feature is the wrong deployment
>> mechanism for the GWL; I also still think that the GWL is too big to be
>> a part of Guix proper.  Perhaps the GWL needs to add more runtime checks
>> to ensure that the Guix modules it uses are available at runtime; any
>> references to individual packages (such as the Guile variant to use to
>> run scripts, or the Bash package used for shell wrappers) should also be
>> replaced with more future-proof lookups by package name.
>
> I thought you proposed that GWL should run in an inferior (say 1)
> fixing the correct Guix version that it needs to work and another
> inferior (say 2) should populate the store with any other Guix
> version. And because the store is stable, then GWL using inferior 1
> does the expected job. Is it not what you have in mind when speaking
> about inferiors? (plural at inferior ;-))

Oh, I didn’t think of inferior number 1 at all, but that’s a good
point.  We could arrange for the build-time modules of Guix to be made
available at runtime, independent of the version of Guix used to run
“guix workflow”.  Then we need to build packages with an inferior Guix,
which is identical to the Guix used to run “guix workflow”.

(guix inferior) does not allow us to do this yet.  The
gexp->derivation-in-inferior procedure requires a derivation that
returns some variant of Guix — we can’t just give it the directory name
of the invoking command.  I think this might be a good addition to (guix
inferior).

--
Ricardo

Re: How to install GWL?

[zimoun]

Hi Ricardo,

On Wed, 5 Feb 2020 at 20:36, Ricardo Wurmus <rekado@elephly.net> wrote:

> Oh, I didn’t think of inferior number 1 at all, but that’s a good
> point.  We could arrange for the build-time modules of Guix to be made
> available at runtime, independent of the version of Guix used to run
> “guix workflow”.  Then we need to build packages with an inferior Guix,
> which is identical to the Guix used to run “guix workflow”.

Yes something along these lines.
Let restart the discussion after the new release.
I mean, the next release could still use the old mechanism because it
is more about the description language itself than the version of the
packages.


> (guix inferior) does not allow us to do this yet.  The
> gexp->derivation-in-inferior procedure requires a derivation that
> returns some variant of Guix — we can’t just give it the directory name
> of the invoking command.  I think this might be a good addition to (guix
> inferior).

Let open a discussion on guix-devel to collect some ideas. WDYT?


All the best,
simon