Enterprise Guix Hosting?

Enterprise Guix Hosting?

[Yasuaki Kudo]

Hello!

I have been exposed to the world of Docker images and Continuous Integration that seem spend most of the time downloading and building them 😅

I can see where they are coming from though - the software dev teams will pay any reasonable money for what works!

Have you heard of Guix Hosting services jusy like the Docker or Github companies target enterprises customers?

My partners and I are just starting an IT worker cooperative and I thought this might be an interesting thing to get into! 

-Yasu

Re: Enterprise Guix Hosting?

[Olivier Dion via]

On Sat, 30 Jul 2022, Yasuaki Kudo <yasu@yasuaki.com> wrote:
> I have been exposed to the world of Docker images and Continuous
> Integration that seem spend most of the time downloading and building
> them 😅

Yup.  Tons of energy wasted and pollution generated.

> Have you heard of Guix Hosting services jusy like the Docker or Github
> companies target enterprises customers?

IMO Guix is still very niche.  Only a handful of Unix enthusiasms /
scientifics use it and companies are not in yet.  I might be wrong.

> My partners and I are just starting an IT worker cooperative and I
> thought this might be an interesting thing to get into!

I think it is.  I know that some companies pay very good for support
related to dev-op, e.g. CI.  I think that Guix could fit very well for
professional services and software as a services.  However, Guix
probably needs to be more main stream for that to work.

-- 
Olivier Dion
oldiob.dev

Re: Enterprise Guix Hosting?

[Phil]

Olivier Dion via writes:

> On Sat, 30 Jul 2022, Yasuaki Kudo <yasu@yasuaki.com> wrote:
>> I have been exposed to the world of Docker images and Continuous
>> Integration that seem spend most of the time downloading and building
>> them 😅
>
> Yup.  Tons of energy wasted and pollution generated.
>
>> Have you heard of Guix Hosting services jusy like the Docker or Github
>> companies target enterprises customers?
>
> IMO Guix is still very niche.  Only a handful of Unix enthusiasms /
> scientifics use it and companies are not in yet.  I might be wrong.

I introduced Guix to a company called Quantile where I work as the
Head of Enterprise Architecture.  Docker would have been the typical
mainstream alternative, but for the reasons you say and others Guix was
considered a more complete solution to a whole engineering ecosystem.
Nix was also considered, but we went with Guix in the end.

We have integrated it into our standard Jenkins pipeline and AWS cloud,
and it plays a central role both in how developers work (eg replacing
Python's virtual environments) and how software is deployed by our CI/CD system.

It was a bit of a punt, given Guix is not yet widely used outside
academia, but it ticked all the boxes, performed well in PoC tests, and
seemed like a solid tech decision - and one I'm still very pleased I made!

We've done a few talks on our setup and integration with more standard
commercial tooling - in case you haven't seen these they might be of interest:
https://www.cloudbees.com/videos/purely-functional-ci-cd-pipeline-using-jenkins-with-guix
https://xana.lepiller.eu/guix-days-2022/guix-days-2022-guix-aws-lambda.mkv

I'm always very interested in any discussions regarding Guix use in
mainstream and commerical projects - I think it has a bright future in
this space.


Phil

Re: Enterprise Guix Hosting?

[Yasuaki Kudo]

Oh wow!! Thank you everyone!!!    I will definitely show our conversation thread to my partners - in fact, right now!!!   Let's make it happen!!

> On Jul 31, 2022, at 01:20, Phil <phil@beadling.co.uk> wrote:
> 
> 
> Olivier Dion via writes:
> 
>>> On Sat, 30 Jul 2022, Yasuaki Kudo <yasu@yasuaki.com> wrote:
>>> I have been exposed to the world of Docker images and Continuous
>>> Integration that seem spend most of the time downloading and building
>>> them 😅
>> 
>> Yup.  Tons of energy wasted and pollution generated.
>> 
>>> Have you heard of Guix Hosting services jusy like the Docker or Github
>>> companies target enterprises customers?
>> 
>> IMO Guix is still very niche.  Only a handful of Unix enthusiasms /
>> scientifics use it and companies are not in yet.  I might be wrong.
> 
> I introduced Guix to a company called Quantile where I work as the
> Head of Enterprise Architecture.  Docker would have been the typical
> mainstream alternative, but for the reasons you say and others Guix was
> considered a more complete solution to a whole engineering ecosystem.
> Nix was also considered, but we went with Guix in the end.
> 
> We have integrated it into our standard Jenkins pipeline and AWS cloud,
> and it plays a central role both in how developers work (eg replacing
> Python's virtual environments) and how software is deployed by our CI/CD system.
> 
> It was a bit of a punt, given Guix is not yet widely used outside
> academia, but it ticked all the boxes, performed well in PoC tests, and
> seemed like a solid tech decision - and one I'm still very pleased I made!
> 
> We've done a few talks on our setup and integration with more standard
> commercial tooling - in case you haven't seen these they might be of interest:
> https://www.cloudbees.com/videos/purely-functional-ci-cd-pipeline-using-jenkins-with-guix
> https://xana.lepiller.eu/guix-days-2022/guix-days-2022-guix-aws-lambda.mkv
> 
> I'm always very interested in any discussions regarding Guix use in
> mainstream and commerical projects - I think it has a bright future in
> this space.
> 
> 
> Phil

Re: Enterprise Guix Hosting?

[Benjamin Slade]

Not something I've tried personally, but maybe PantherX (Guix-based) could be relevant here? https://www.pantherx.org/

30 Jul 2022 17:18:13 Yasuaki Kudo <yasu@yasuaki.com>:

> Oh wow!! Thank you everyone!!!    I will definitely show our conversation thread to my partners - in fact, right now!!!   Let's make it happen!!
> 
>> On Jul 31, 2022, at 01:20, Phil <phil@beadling.co.uk> wrote:
>> 
>> 
>> Olivier Dion via writes:
>> 
>>>> On Sat, 30 Jul 2022, Yasuaki Kudo <yasu@yasuaki.com> wrote:
>>>> I have been exposed to the world of Docker images and Continuous
>>>> Integration that seem spend most of the time downloading and building
>>>> them 😅
>>> 
>>> Yup.  Tons of energy wasted and pollution generated.
>>> 
>>>> Have you heard of Guix Hosting services jusy like the Docker or Github
>>>> companies target enterprises customers?
>>> 
>>> IMO Guix is still very niche.  Only a handful of Unix enthusiasms /
>>> scientifics use it and companies are not in yet.  I might be wrong.
>> 
>> I introduced Guix to a company called Quantile where I work as the
>> Head of Enterprise Architecture.  Docker would have been the typical
>> mainstream alternative, but for the reasons you say and others Guix was
>> considered a more complete solution to a whole engineering ecosystem.
>> Nix was also considered, but we went with Guix in the end.
>> 
>> We have integrated it into our standard Jenkins pipeline and AWS cloud,
>> and it plays a central role both in how developers work (eg replacing
>> Python's virtual environments) and how software is deployed by our CI/CD system.
>> 
>> It was a bit of a punt, given Guix is not yet widely used outside
>> academia, but it ticked all the boxes, performed well in PoC tests, and
>> seemed like a solid tech decision - and one I'm still very pleased I made!
>> 
>> We've done a few talks on our setup and integration with more standard
>> commercial tooling - in case you haven't seen these they might be of interest:
>> https://www.cloudbees.com/videos/purely-functional-ci-cd-pipeline-using-jenkins-with-guix
>> https://xana.lepiller.eu/guix-days-2022/guix-days-2022-guix-aws-lambda.mkv
>> 
>> I'm always very interested in any discussions regarding Guix use in
>> mainstream and commerical projects - I think it has a bright future in
>> this space.
>> 
>> 
>> Phil

Re: Enterprise Guix Hosting?

[Phil]

Benjamin Slade writes:

> Not something I've tried personally, but maybe PantherX (Guix-based) could be relevant here? https://www.pantherx.org/

This is very interesting - there is definately a niche to produce
off-the-shelf guix-based enterprise solutions that a mainstream devops
team could rollout and interface with "standard" tech tooling without
having to learn (a lot of) Scheme.

My own experience is that whilst it doesn't require a PhD to setup Guix
for the enterprise, it is a non-trivial journey, and it does require
a fair amount of time and effort to create something that regular
developers/scientists (i.e. non-Guix converts who just want to get on with their
day-jobs) accept is as good or better than regular tooling they are used
to.  There's certainly a barrier to entry for people who don't want to
do a deep-dive and just want tooling to support them in their professional
role, without them having to think about it too much.

Upselling the real benefits of Guix like rollbacks, profiles, perfectly
reproducable builds, swapping one dependency for another - even in a
scientific/tech-savvy company with lots of PhDs took a bit of persuading
from me. Even now I think our company is only using perhaps 30% of the
true power of Guix.  Making all that power accessible to people who just
want to get on with their jobs in an easy, intuitive way is a challenge
I'm continuously trying to address.  I also hope things like PantherX
might help bridge the gap in the near future! 

Re: Enterprise Guix Hosting?

[Ludovic Courtès]

Hi Phil,

Phil <phil@beadling.co.uk> skribis:

> My own experience is that whilst it doesn't require a PhD to setup Guix
> for the enterprise, it is a non-trivial journey, and it does require
> a fair amount of time and effort to create something that regular
> developers/scientists (i.e. non-Guix converts who just want to get on with their
> day-jobs) accept is as good or better than regular tooling they are used
> to.  There's certainly a barrier to entry for people who don't want to
> do a deep-dive and just want tooling to support them in their professional
> role, without them having to think about it too much.
>
> Upselling the real benefits of Guix like rollbacks, profiles, perfectly
> reproducable builds, swapping one dependency for another - even in a
> scientific/tech-savvy company with lots of PhDs took a bit of persuading
> from me. Even now I think our company is only using perhaps 30% of the
> true power of Guix.  Making all that power accessible to people who just
> want to get on with their jobs in an easy, intuitive way is a challenge
> I'm continuously trying to address.  I also hope things like PantherX
> might help bridge the gap in the near future! 

From your experience, would you say that persuading was hard primarily
because Guix was unknown (to them), or because getting started is
difficult?

Personally I think we need to make Guix approachable to a wide audience,
meaning not just developers—that goes beyond your target audience, let’s
be ambitious!  I’d like to think that ‘guix install’, ‘guix shell’, and
the likes have a rather low barrier to entry to someone who’s use the
command line before, but I’ve also seen newcomers confused because
“environment variables are hard” and get in the way.

Are there any takeaways from your experience in terms of UX/UI
improvements we could work on?

Thanks,
Ludo’.

Re: Enterprise Guix Hosting?

[Yasuaki Kudo]

Let's do this!  My partners and I are fired up about the idea and we would like this to be developed, along the way we continue to serve customers (or co-creators, in our worker coop world😄).

Realistic, step by step implementation, niche to niche, until we make it big! 😄

-Yasu

> On Aug 10, 2022, at 05:37, Ludovic Courtès <ludo@gnu.org> wrote:
> 
> Hi Phil,
> 
> Phil <phil@beadling.co.uk> skribis:
> 
>> My own experience is that whilst it doesn't require a PhD to setup Guix
>> for the enterprise, it is a non-trivial journey, and it does require
>> a fair amount of time and effort to create something that regular
>> developers/scientists (i.e. non-Guix converts who just want to get on with their
>> day-jobs) accept is as good or better than regular tooling they are used
>> to.  There's certainly a barrier to entry for people who don't want to
>> do a deep-dive and just want tooling to support them in their professional
>> role, without them having to think about it too much.
>> 
>> Upselling the real benefits of Guix like rollbacks, profiles, perfectly
>> reproducable builds, swapping one dependency for another - even in a
>> scientific/tech-savvy company with lots of PhDs took a bit of persuading
>> from me. Even now I think our company is only using perhaps 30% of the
>> true power of Guix.  Making all that power accessible to people who just
>> want to get on with their jobs in an easy, intuitive way is a challenge
>> I'm continuously trying to address.  I also hope things like PantherX
>> might help bridge the gap in the near future! 
> 
> From your experience, would you say that persuading was hard primarily
> because Guix was unknown (to them), or because getting started is
> difficult?
> 
> Personally I think we need to make Guix approachable to a wide audience,
> meaning not just developers—that goes beyond your target audience, let’s
> be ambitious!  I’d like to think that ‘guix install’, ‘guix shell’, and
> the likes have a rather low barrier to entry to someone who’s use the
> command line before, but I’ve also seen newcomers confused because
> “environment variables are hard” and get in the way.
> 
> Are there any takeaways from your experience in terms of UX/UI
> improvements we could work on?
> 
> Thanks,
> Ludo’.

Re: Enterprise Guix Hosting?

[Phil]

Hi Ludo,

Comments inline.  I'm also aiming to be at the Guix 10 Year thing in
Paris - sadly only for the Friday, so happy to discuss this informally
there too!

Ludovic Courtès writes:

> Hi Phil,
>
> Phil <phil@beadling.co.uk> skribis:
>
>
> From your experience, would you say that persuading was hard primarily
> because Guix was unknown (to them), or because getting started is
> difficult?

It's a bit of both, in the commercial space there are some mundane
practical concerns.  One example would  be when 2 companies security
audit each other before using each other's services.  If you're using a
prebuilt image of a well known OS, served by AWS or Azure, then the
reality is that this is often easier for a security team to tick this off as
a known platform - for no other reason than they've seen it before.
Auditing Guix isn't impossible but it can lead to more questions, simply
because of lack of familiarity.  This can be somewhat mitigated by using
Guix as just a package manager on top of a foreign distro, but this
doesn't fully harness the potential of Guix, so it's a trade-off.

Internally speaking the lack of familiarity wasn't as much of a barrier.
Python is the main language where I work, so I sold it as a better version
of Virtual Environments - which work for all languages not just Python.
There was an significant initial effort from me and my team to convert
all the current venvs to Guix packages and integrate it with the various
Runtimes and IDEs we use, but once we'd done this, people were largely
happy to transition.  I did have to do some tutorials and write a bit of
documentation that meant people could start using Guix without really
getting into the details of what Guix is doing.  My argument to most
developers was, "you don't really know all the details of how virtual
environments work, so why do you care about Guix's internals?".  Most
happily accepted this argument, providing you give them good docs on how
to use Guix in the workplace.

Whilst I like Guix's own documentation, some developers did feedback to
me that it was to complex for people who just wanted to get-on and use
Guix, rather than setup, understand and maintain Guix.  So this is the
area I ended-up documenting - "Guix Up-and-running for Python
Developers". One day I'd like to publish it properly, but it's very much
a WIP at the moment!

One advantage I did have is that I rewrote the CI/CD system
to work around Guix, and the old system was showing it's age, so people
were happy to trade Python venvs, for a better build and deployment experience.

We now have 5 developers working at least part of the time writing
Guix packages, or tweaking small bits of the Guix core code (I keep
meaning to make more of an effort to get our efforts back into Guix
proper!).  As more developers slowly try-out more advanced stuff in Guix
this number is growing, and most developers that invest the time end up
liking Guix - so I think there's plenty of hope to grow it further!

>
> Personally I think we need to make Guix approachable to a wide audience,
> meaning not just developers—that goes beyond your target audience, let’s
> be ambitious!  I’d like to think that ‘guix install’, ‘guix shell’, and
> the likes have a rather low barrier to entry to someone who’s use the
> command line before, but I’ve also seen newcomers confused because
> “environment variables are hard” and get in the way.

Yep I do review how Guix is being used at work, and occasionally do find
people using it in weird and wonderful ways.  All I do is build up my
documentation so we have a cookbook like format which covers recommended
ways for developers to do things, and things for them to avoid doing too!

Environment variables can be a common one, when people fiddle with their
PYTHONPATH in their code, or .bashrc, and this can have knock-on issues
with Guix.  Best practice documentation helps with this.

>
> Are there any takeaways from your experience in terms of UX/UI
> improvements we could work on?

3 things which lowers the barrier to entry in my experience commercially
would be:

- Push button WSL support (I know this has some momentum eg
  https://lists.gnu.org/archive/html/guix-patches/2022-08/msg00945.html).
  At the moment I tend to use a custom image I made which is just WSL on
  top of Ubuntu.  I have made it work with busybox, but it's not yet
  robust enough to wheel out over the enterprise like this.
- Perhaps a set of videos aimed directly at converting a vanilla Python
  environment into one running in Guix.  Try to entice the communities
  off their current tooling by making it as easy as possible to switch.
  I even went as far as writing a requirements file to guix package
  converter at work to help with this.
- Excellent Javascript support would help.  I'm aware of some of the
  difficulties this presents Guix, and am not a fan of npm, etc - but
  it's so often used by developers I think not having support for it is
  always going to be tricky to sell to a wider audience.

>
> Thanks,
> Ludo’.

Re: Enterprise Guix Hosting?

[Yasuaki Kudo]

Hello Phil!!!

What you wrote makes so much sense and sounds very familiar because I had similar discussions with my partners at our worker coop!

Sometime soon perhaps we can discuss in a video chat or something?

Our idea is at the coop is that we want to develop software development acceleration tools, and a major part would be container-less software provisioning so that composition would not mean more and more layers of technical debt...

We would like to do this alongside our regular paid software development for existing clients.  Once we have acquired enough scripts, knowhow and the enthusiasts among our colleagues, we can spin it off and sell it as a service/consultation! (software will stay as Libre and Free - enterprise users would not be interested otherwise )

BTW, I am probably the least technical person within the IT side of our coop - my other partners are pretty high level experts 😄

-Yasu

> On Aug 14, 2022, at 18:53, Phil <phil@beadling.co.uk> wrote:
> 
> Hi Ludo,
> 
> Comments inline.  I'm also aiming to be at the Guix 10 Year thing in
> Paris - sadly only for the Friday, so happy to discuss this informally
> there too!
> 
> Ludovic Courtès writes:
> 
>> Hi Phil,
>> 
>> Phil <phil@beadling.co.uk> skribis:
>> 
>> 
>> From your experience, would you say that persuading was hard primarily
>> because Guix was unknown (to them), or because getting started is
>> difficult?
> 
> It's a bit of both, in the commercial space there are some mundane
> practical concerns.  One example would  be when 2 companies security
> audit each other before using each other's services.  If you're using a
> prebuilt image of a well known OS, served by AWS or Azure, then the
> reality is that this is often easier for a security team to tick this off as
> a known platform - for no other reason than they've seen it before.
> Auditing Guix isn't impossible but it can lead to more questions, simply
> because of lack of familiarity.  This can be somewhat mitigated by using
> Guix as just a package manager on top of a foreign distro, but this
> doesn't fully harness the potential of Guix, so it's a trade-off.
> 
> Internally speaking the lack of familiarity wasn't as much of a barrier.
> Python is the main language where I work, so I sold it as a better version
> of Virtual Environments - which work for all languages not just Python.
> There was an significant initial effort from me and my team to convert
> all the current venvs to Guix packages and integrate it with the various
> Runtimes and IDEs we use, but once we'd done this, people were largely
> happy to transition.  I did have to do some tutorials and write a bit of
> documentation that meant people could start using Guix without really
> getting into the details of what Guix is doing.  My argument to most
> developers was, "you don't really know all the details of how virtual
> environments work, so why do you care about Guix's internals?".  Most
> happily accepted this argument, providing you give them good docs on how
> to use Guix in the workplace.
> 
> Whilst I like Guix's own documentation, some developers did feedback to
> me that it was to complex for people who just wanted to get-on and use
> Guix, rather than setup, understand and maintain Guix.  So this is the
> area I ended-up documenting - "Guix Up-and-running for Python
> Developers". One day I'd like to publish it properly, but it's very much
> a WIP at the moment!
> 
> One advantage I did have is that I rewrote the CI/CD system
> to work around Guix, and the old system was showing it's age, so people
> were happy to trade Python venvs, for a better build and deployment experience.
> 
> We now have 5 developers working at least part of the time writing
> Guix packages, or tweaking small bits of the Guix core code (I keep
> meaning to make more of an effort to get our efforts back into Guix
> proper!).  As more developers slowly try-out more advanced stuff in Guix
> this number is growing, and most developers that invest the time end up
> liking Guix - so I think there's plenty of hope to grow it further!
> 
>> 
>> Personally I think we need to make Guix approachable to a wide audience,
>> meaning not just developers—that goes beyond your target audience, let’s
>> be ambitious!  I’d like to think that ‘guix install’, ‘guix shell’, and
>> the likes have a rather low barrier to entry to someone who’s use the
>> command line before, but I’ve also seen newcomers confused because
>> “environment variables are hard” and get in the way.
> 
> Yep I do review how Guix is being used at work, and occasionally do find
> people using it in weird and wonderful ways.  All I do is build up my
> documentation so we have a cookbook like format which covers recommended
> ways for developers to do things, and things for them to avoid doing too!
> 
> Environment variables can be a common one, when people fiddle with their
> PYTHONPATH in their code, or .bashrc, and this can have knock-on issues
> with Guix.  Best practice documentation helps with this.
> 
>> 
>> Are there any takeaways from your experience in terms of UX/UI
>> improvements we could work on?
> 
> 3 things which lowers the barrier to entry in my experience commercially
> would be:
> 
> - Push button WSL support (I know this has some momentum eg
>  https://lists.gnu.org/archive/html/guix-patches/2022-08/msg00945.html).
>  At the moment I tend to use a custom image I made which is just WSL on
>  top of Ubuntu.  I have made it work with busybox, but it's not yet
>  robust enough to wheel out over the enterprise like this.
> - Perhaps a set of videos aimed directly at converting a vanilla Python
>  environment into one running in Guix.  Try to entice the communities
>  off their current tooling by making it as easy as possible to switch.
>  I even went as far as writing a requirements file to guix package
>  converter at work to help with this.
> - Excellent Javascript support would help.  I'm aware of some of the
>  difficulties this presents Guix, and am not a fan of npm, etc - but
>  it's so often used by developers I think not having support for it is
>  always going to be tricky to sell to a wider audience.
> 
>> 
>> Thanks,
>> Ludo’.
> 

Re: Enterprise Guix Hosting?

[Phil]

Yasuaki Kudo writes:

> What you wrote makes so much sense and sounds very familiar because I had similar discussions with my partners at our worker coop!
>
> Sometime soon perhaps we can discuss in a video chat or something?

Sure - keep us in the loop.

> We would like to do this alongside our regular paid software development for existing clients.  Once we have acquired enough scripts, knowhow and the enthusiasts among our colleagues, we can spin it off and sell it as a service/consultation! (software will stay as Libre and Free - enterprise users would not be interested otherwise )

Where I work Guix is a tool we use in IT, rather than part of the
product we package/sell.  This means I hope much of what we have learned
in terms of extending and using Guix will end-up open and freely
available over time - some in Guix itself. I don't see us offering
services or consultation in Guix as we're not a pure IT company.
However, I am interested in seeing Guix and Guix services gaining a
foothold in the commercial space, and exchanging ideas around this - as
well as growing it in academia too.

Re: Enterprise Guix Hosting?

[Olivier Dion via]

On Mon, 15 Aug 2022, Phil <phil@beadling.co.uk> wrote:
> Yasuaki Kudo writes:

> However, I am interested in seeing Guix and Guix services
> gaining a foothold in the commercial space, and exchanging ideas

I see a lot of potential there:

  - Continuous integration
  - VM generation and deployment with OpenStack
  - Container generation and deployment with K8s
  - Root filesystem generation for embedded systems (e.g. Yocto, Elbe)

> as well as growing it in academia too.

As a student, here's how I have been using Guix on the academic side:

  - CI (cuirass) of developed tools
  - Reproducible workflow for my research
  - guix-shell + manifests for volatile laboratories environment
  - Automatic correction of student's work submission (similar to a CI)

In all, Guix is a very powerful/flexible tool when in good hands and
everyone in the field would gain from mass adoption of it.  I think it
has a bright future ahead.

-- 
Olivier Dion
oldiob.dev

Re: Enterprise Guix Hosting?

[Yasuaki Kudo]

And I recently experimented with WSL2 and WSLG on Windows 11, running on a 5-year-old Surface Book.

Ubuntu works quite well on it and I can run Linux GUI applications from Windows and run Windows applications, like Excel and Powershell, from Linux.

So this indicates there is a high likelihood Guix will be a decent enterprise package manager, as long as the users' IT department allows WSL on enterprise-managed Windows PCs.  (which may be a hurdle too much to ask for though😅)

Sorry I have been wanting to follow up more but I have been too busy setting up our worker coop in Japan😅

-Yasu


> On Aug 26, 2022, at 03:37, Olivier Dion <olivier.dion@polymtl.ca> wrote:
> 
> On Mon, 15 Aug 2022, Phil <phil@beadling.co.uk> wrote:
>> Yasuaki Kudo writes:
> 
>> However, I am interested in seeing Guix and Guix services
>> gaining a foothold in the commercial space, and exchanging ideas
> 
> I see a lot of potential there:
> 
>  - Continuous integration
>  - VM generation and deployment with OpenStack
>  - Container generation and deployment with K8s
>  - Root filesystem generation for embedded systems (e.g. Yocto, Elbe)
> 
>> as well as growing it in academia too.
> 
> As a student, here's how I have been using Guix on the academic side:
> 
>  - CI (cuirass) of developed tools
>  - Reproducible workflow for my research
>  - guix-shell + manifests for volatile laboratories environment
>  - Automatic correction of student's work submission (similar to a CI)
> 
> In all, Guix is a very powerful/flexible tool when in good hands and
> everyone in the field would gain from mass adoption of it.  I think it
> has a bright future ahead.
> 
> -- 
> Olivier Dion
> oldiob.dev

Re: Enterprise Guix Hosting?

[Ricardo Wurmus]

Hi Yasu

> Our idea is at the coop is that we want to develop software
> development acceleration tools, and a major part would be
> container-less software provisioning so that composition would not
> mean more and more layers of technical debt...

Don’t discount containers too soon.  Guix has “guix system container”,
which spins up lightweight Guix System containers that share /gnu/store.
You only need to set up a bridge interface on the host and create a
network device pair and move one end into the container’s net namespace.

You can do containers and compose them without layers upon layers of
file system blobs.  The reasons why this is not commonly done on
existing commercial platforms:

- container images are often provided from different origins, so there
  is no trust and thus no way to have them share the same files or
  common packages

- without reproducible builds trust cannot be established

- container images are erroneously considered a requirement for
  isolation, but it is not actually required to use them even in the
  presence of an unshared mount namespace.

Using a shared /gnu/store as a big cache for all containers can be a
real asset.  We can learn lessons from the HPC experience here.

-- 
Ricardo

Re: Enterprise Guix Hosting?

[Thompson, David]

Hi Ricardo,

On Fri, Aug 26, 2022 at 3:43 AM Ricardo Wurmus <rekado@elephly.net> wrote:
>
>
> Hi Yasu
>
> > Our idea is at the coop is that we want to develop software
> > development acceleration tools, and a major part would be
> > container-less software provisioning so that composition would not
> > mean more and more layers of technical debt...
>
> Don’t discount containers too soon.  Guix has “guix system container”,
> which spins up lightweight Guix System containers that share /gnu/store.
> You only need to set up a bridge interface on the host and create a
> network device pair and move one end into the container’s net namespace.

I thought for sure that 'guix system container' would be something
people would love, but it doesn't seem to get much use!  Having all
containers share the store eliminates several problems that come with
Docker's primitive layer approach.  When I realized all we had to do
was bind mount store items into the container I couldn't believe it
was so simple.

> You can do containers and compose them without layers upon layers of
> file system blobs.  The reasons why this is not commonly done on
> existing commercial platforms:
>
> - container images are often provided from different origins, so there
>   is no trust and thus no way to have them share the same files or
>   common packages
>
> - without reproducible builds trust cannot be established
>
> - container images are erroneously considered a requirement for
>   isolation, but it is not actually required to use them even in the
>   presence of an unshared mount namespace.

All true.  "Container" has come to mean the image more than the
execution environment, so Guix containers not being based on disk
images makes them not fit in.

> Using a shared /gnu/store as a big cache for all containers can be a
> real asset.  We can learn lessons from the HPC experience here.

What might have a positive impact is if Guix had an answer to 'docker
compose'.  Most of the pieces are there already.  Such a tool could be
combined with 'guix shell' so you could get all the tools needed for
local development *and* automatically start any necessary daemons,
like database servers, in isolated containers.

- Dave

Re: Enterprise Guix Hosting?

[Ricardo Wurmus]

"Thompson, David" <dthompson2@worcester.edu> writes:

>> Using a shared /gnu/store as a big cache for all containers can be a
>> real asset.  We can learn lessons from the HPC experience here.
>
> What might have a positive impact is if Guix had an answer to 'docker
> compose'.  Most of the pieces are there already.  Such a tool could be
> combined with 'guix shell' so you could get all the tools needed for
> local development *and* automatically start any necessary daemons,
> like database servers, in isolated containers.

Yes, this would be useful.

Another thing that seems to be missing is a way to supervise and manage
running containers.  I use a shepherd instance for this with
container-specific actions like this:

(define %ip
  (let ((loc #f))
    (lambda ()
      "Return the absolute file name of the ip command."
      (or loc
          (let ((value (which "ip")))
            (set! loc value)
            value)))))

(define (launch container-id script)
   (herd "root" "eval"
         (format #false
           "~s"
           `(begin
              (use-modules (srfi srfi-2)
                           (ice-9 popen)
                           (ice-9 rdelim)
                           (ice-9 match))
              (define (guix-container id script)
                (make <service>
                  #:provides (list
                              (string->symbol (string-append "guix-container-" id)))
                  #:docstring "Run a Guix System container"
                  #:start
                  ;; TODO: Using /bin/sh -c is ugly, but
                  ;; without it the container would be stuck in the early boot process.
                  (make-forkexec-constructor
                   `("/bin/sh" "-c" (string-join (list "exec" script ,@args) #\space)))
                  #:stop (make-kill-destructor)
                  #:actions
                  (make-actions
                   (pid
                    "Show the PID of the system container."
                    (lambda (running)
                      (let ((pid (call-with-input-file
                                     (format #false "/proc/~a/task/~a/children"
                                             running running)
                                   read)))
                        (display (match pid
                                   ((? eof-object?) "")
                                   (_ pid))))))
                   (ip
                    "Show the IP address of the system container."
                    (lambda (running)
                      (let* ((pid (number->string
                                   (call-with-input-file
                                       (format #false "/proc/~a/task/~a/children"
                                               running running)
                                     read)))
                             (ns (format #false "guix-~a" pid))
                             (ip ,(%ip))
                             (address
                              (catch #true
                                (lambda ()
                                  (let* ((pipe (open-pipe* OPEN_READ
                                                           ip "netns" "exec" ns
                                                           "ip" "-o" "-4"
                                                           "-family" "inet"
                                                           "addr" "show"
                                                           "dev" (format #false "ceth-~a" pid)))
                                         (output (read-line pipe)))
                                    (match (string-tokenize output)
                                      ((number if "inet" ip . rest) ip)
                                      (_ ""))))
                                (lambda _ ""))))
                        (display address))))
                   (up
                    "Connect network for the system container."
                    (lambda (running)
                      (let* ((pid (number->string
                                   (call-with-input-file
                                       (format #false "/proc/~a/task/~a/children"
                                               running running)
                                     read)))
                             (ns (format #false "guix-~a" pid))
                             (host (format #false "veth-~a" pid))
                             (client (format #false "ceth-~a" pid))
                             (ip ,(%ip))
                             (sys (lambda args
                                    (or (zero? (apply system* args))
                                        (error args)))))
                        ;; Make existing network namespace available to ip netns
                        (sys ip "netns" "attach" ns pid)

                        ;; Create veth pair and move the client side into the container.
                        (sys ip "link" "add" host "type" "veth" "peer" "name" client)
                        (sys ip "link" "set" host "up")
                        (sys ip "link" "set" client "netns" ns)

                        ;; Attach host side to host bridge
                        (sys ip "link" "set" host "master" "br0")

                        ;; Bring up interface in container
                        (sys ip "netns" "exec" ns "ip" "link" "set" "lo" "up")
                        (sys ip "netns" "exec" ns "ip" "link" "set" client "up"))))
                   (down
                    "Disconnect network for the system container."
                    (lambda (running)
                      (let* ((pid (number->string
                                   (call-with-input-file
                                       (format #false "/proc/~a/task/~a/children"
                                               running running)
                                     read)))
                             (ns (format #false "guix-~a" pid))
                             (host (format #false "veth-~a" pid))
                             (ip ,(%ip))
                             (sys (lambda args
                                    (or (zero? (apply system* args))
                                        (error args)))))
                        (sys ip "netns" "delete" ns)
                        (sys ip "link" "delete" host))))
                   (netstat
                    (lambda (running)
                      (and-let* ((pid (number->string
                                       (call-with-input-file
                                           (format #false "/proc/~a/task/~a/children"
                                                   running running)
                                         read)))
                                 (template (lambda (what)
                                             (format #false "/sys/class/net/veth-~a/statistics/~a_bytes" pid what)))
                                 (rx (call-with-input-file (template "rx") read))
                                 (tx (call-with-input-file (template "tx") read)))
                        (format #true "receive:~a transmit:~a" rx tx)))))))
              (let ((service (guix-container ,vm-id ,script)))
                (register-services service)
                (start service))))))


-- 
Ricardo

Re: [EXT] Re: Enterprise Guix Hosting?

[Thompson, David]

On Wed, Aug 31, 2022 at 2:40 AM Ricardo Wurmus <rekado@elephly.net> wrote:
>
> Another thing that seems to be missing is a way to supervise and manage
> running containers.  I use a shepherd instance for this with
> container-specific actions like this:

Hey that's a real nice starting point for a container management tool!
 So maybe there should be a system service to manage containers and
then a 'docker compose'-like tool for declaratively specifying
containers and their network bridging configuration that is a client
of the service?

- Dave

Re: [EXT] Re: Enterprise Guix Hosting?

[Olivier Dion via]

On Wed, 31 Aug 2022, "Thompson, David" <dthompson2@worcester.edu> wrote:
> Hey that's a real nice starting point for a container management tool!
>  So maybe there should be a system service to manage containers and
> then a 'docker compose'-like tool for declaratively specifying
> containers and their network bridging configuration that is a client
> of the service?

I think that a container composer à la `docker compose' but with
Guile/Guix is easily done. Managing containers locally is trivial and a
little web interface could be done to view/manager your contrainers.
The service could even be a home-service.

From there, a more mature container orchestrator that can scale
to multiple nodes could be made.  Competing with Kubernetes.  Of course
at that level things are not trivial anymore.

-- 
Olivier Dion
oldiob.dev

Re: Enterprise Guix Hosting?

[Thompson, David]

Hi Olivier,

On Wed, Aug 31, 2022 at 7:42 AM Olivier Dion <olivier.dion@polymtl.ca> wrote:
>
> On Wed, 31 Aug 2022, "Thompson, David" <dthompson2@worcester.edu> wrote:
> > Hey that's a real nice starting point for a container management tool!
> >  So maybe there should be a system service to manage containers and
> > then a 'docker compose'-like tool for declaratively specifying
> > containers and their network bridging configuration that is a client
> > of the service?
>
> I think that a container composer à la `docker compose' but with
> Guile/Guix is easily done. Managing containers locally is trivial and a
> little web interface could be done to view/manager your contrainers.
> The service could even be a home-service.
>
> From there, a more mature container orchestrator that can scale
> to multiple nodes could be made.  Competing with Kubernetes.  Of course
> at that level things are not trivial anymore.

Just thinking about making something on the level of Kubernetes makes
me want to hide under a desk, but that is certainly more in line with
the subject of this email thread. :)

So who's going to start selling Guix Enterprise subscriptions?  It
would be a lot of fun.  You could have Guix Solution Architects and
other fun corporate titles!

- Dave

Re: [EXT] Re: Enterprise Guix Hosting?

[Ludovic Courtès]

Hello!

"Thompson, David" <dthompson2@worcester.edu> skribis:

> On Wed, Aug 31, 2022 at 2:40 AM Ricardo Wurmus <rekado@elephly.net> wrote:
>>
>> Another thing that seems to be missing is a way to supervise and manage
>> running containers.  I use a shepherd instance for this with
>> container-specific actions like this:
>
> Hey that's a real nice starting point for a container management tool!
>  So maybe there should be a system service to manage containers and
> then a 'docker compose'-like tool for declaratively specifying
> containers and their network bridging configuration that is a client
> of the service?

Agreed!  We could turn Ricardo’s code into ‘container-guest-service’ or
something and have ‘containerized-operating-system’ add it
automatically.

Ludo’.

Re: Enterprise Guix Hosting?

[Ludovic Courtès]

Hi Phil,

Phil <phil@beadling.co.uk> skribis:

> Comments inline.  I'm also aiming to be at the Guix 10 Year thing in
> Paris - sadly only for the Friday, so happy to discuss this informally
> there too!

Looking forward to chatting there!

[...]

> Whilst I like Guix's own documentation, some developers did feedback to
> me that it was to complex for people who just wanted to get-on and use
> Guix, rather than setup, understand and maintain Guix.  So this is the
> area I ended-up documenting - "Guix Up-and-running for Python
> Developers". One day I'd like to publish it properly, but it's very much
> a WIP at the moment!

Publishing such a document (either standalone or as part of the
cookbook) would be great; it’d certainly be a gentle way to get started
for many developers out there.

> One advantage I did have is that I rewrote the CI/CD system
> to work around Guix, and the old system was showing it's age, so people
> were happy to trade Python venvs, for a better build and deployment experience.

Yes, that too is a use case that we should document better (some years
ago I used Guix at work for CI, testing a piece of C++ code under a
variety of configurations—tedious to do without Guix.)

> We now have 5 developers working at least part of the time writing
> Guix packages, or tweaking small bits of the Guix core code (I keep
> meaning to make more of an effort to get our efforts back into Guix
> proper!).  As more developers slowly try-out more advanced stuff in Guix
> this number is growing, and most developers that invest the time end up
> liking Guix - so I think there's plenty of hope to grow it further!

Neat!

[...]

> 3 things which lowers the barrier to entry in my experience commercially
> would be:
>
> - Push button WSL support (I know this has some momentum eg
>   https://lists.gnu.org/archive/html/guix-patches/2022-08/msg00945.html).
>   At the moment I tend to use a custom image I made which is just WSL on
>   top of Ubuntu.  I have made it work with busybox, but it's not yet
>   robust enough to wheel out over the enterprise like this.
> - Perhaps a set of videos aimed directly at converting a vanilla Python
>   environment into one running in Guix.  Try to entice the communities
>   off their current tooling by making it as easy as possible to switch.
>   I even went as far as writing a requirements file to guix package
>   converter at work to help with this.
> - Excellent Javascript support would help.  I'm aware of some of the
>   difficulties this presents Guix, and am not a fan of npm, etc - but
>   it's so often used by developers I think not having support for it is
>   always going to be tricky to sell to a wider audience.

This is sorted in order of increasing difficulty (maybe exponentially
increasing, even :-)) but yes, these sound like good action items.

Thanks for your feedback!

Ludo’.

Re: Enterprise Guix Hosting?

[zimoun]

Hi Phil,

I am a bit late to the party.


On dim., 14 août 2022 at 10:53, Phil <phil@beadling.co.uk> wrote:

> Comments inline.  I'm also aiming to be at the Guix 10 Year thing in
> Paris - sadly only for the Friday, so happy to discuss this informally
> there too!

Sadly, we missed the opportunity to informally discuss at the
event.  Hope next time. :-)



> There was an significant initial effort from me and my team to convert
> all the current venvs to Guix packages and integrate it with the various
> Runtimes and IDEs we use, but once we'd done this, people were largely
> happy to transition.

Which IDEs do you use?

For what my opinion is worth, many of Guix users are Emacs users (just
look at the number of Emacs packages ;-) without speaking about examples
in the manual :-)).  It includes myself.  When I advocate Guix to
biologists or other researchers, I often use Emacs and ’M-x shell’ by
habits and then I move around various buffers again by habits.  People
are often expressing some worry faces 😰.

Therefore, now I try to run the same but with plain terminal as xterm.
But that’s not satisfying because they are not used to this interface.

When the researcher is mainly doing some R analysis, then I try to run
RStudio for demoing – the learning curve is a bit better – they focus on
Guix concepts and how to get the job done instead of being distracted by
just another foreign interface.

It could be nice if we have some starter materials for some IDEs. :-)


>                       I did have to do some tutorials and write a bit of
> documentation that meant people could start using Guix without really
> getting into the details of what Guix is doing.

Are these tutorials public?  Are you allowed to share them?

One of the main roadblock, even for developers, is the lack «Getting
Started» for some specific tasks; as Python, R, etc.

Currently, a newcomer needs some motivation to start alone and from
scratch.  For instance, in a «1h mini-tutorial*» [1,2] I have tried to
explain the pragmatic concepts of Guix that a person in academia could
be interested in for joining the Guix fun.

*mini-tutorial: sorry it is in French.

Although, at some points, this material provides too much internal
details. ;-)


1: <https://replay.jres.org/w/3TuYmocHwKtzs7q1VtL1GB>
2: <https://zimoun.gitlab.io/jres22-tuto-guix/>


>                                                  My argument to most
> developers was, "you don't really know all the details of how virtual
> environments work, so why do you care about Guix's internals?".  Most
> happily accepted this argument, providing you give them good docs on how
> to use Guix in the workplace.

I totally agree.  Is these docs too specific to your workplace or can
they be shared?


> Whilst I like Guix's own documentation, some developers did feedback to
> me that it was to complex for people who just wanted to get-on and use
> Guix, rather than setup, understand and maintain Guix.  

Yes, I agree.  If we follow these principles of documentation [3] (for
what they are worth), the Guix manual is in one quarter when 3 other
quarters are not covered yet, IMHO.

Nice, some fun to do! :-)


3: <<https://documentation.divio.com/>

>                                                         So this is the
> area I ended-up documenting - "Guix Up-and-running for Python
> Developers". One day I'd like to publish it properly, but it's very much
> a WIP at the moment!

For sure, I will be interested by this material!  Even if it is a WIP. :-)


> We now have 5 developers working at least part of the time writing
> Guix packages, or tweaking small bits of the Guix core code (I keep
> meaning to make more of an effort to get our efforts back into Guix
> proper!).  As more developers slowly try-out more advanced stuff in Guix
> this number is growing, and most developers that invest the time end up
> liking Guix - so I think there's plenty of hope to grow it further!

Oh awesome! \o/



> 3 things which lowers the barrier to entry in my experience commercially
> would be:

Some pointers for 2 things.

> - Push button WSL support (I know this has some momentum eg
>   https://lists.gnu.org/archive/html/guix-patches/2022-08/msg00945.html).
>   At the moment I tend to use a custom image I made which is just WSL on
>   top of Ubuntu.  I have made it work with busybox, but it's not yet
>   robust enough to wheel out over the enterprise like this.

Mathieu recently published [4].

4: <https://othacehe.org/wsl-images-for-guix-system.html>


> - Perhaps a set of videos aimed directly at converting a vanilla Python
>   environment into one running in Guix.  Try to entice the communities
>   off their current tooling by making it as easy as possible to switch.
>   I even went as far as writing a requirements file to guix package
>   converter at work to help with this.

There is these nice videos [5] which would require some more love.

Well, some materials for producing these kind of videos can be found
there [6].


5: <https://guix.gnu.org/en/videos/>
6: <https://git.savannah.gnu.org/cgit/guix/videos.git>



Cheers,
simon

Re: Enterprise Guix Hosting?

[Phil]

Hi Zimoun,

zimoun writes:

> On dim., 14 août 2022 at 10:53, Phil <phil@beadling.co.uk> wrote:
>
> Sadly, we missed the opportunity to informally discuss at the
> event.  Hope next time. :-)

Yes - I was only there for the day sadly, so was over too fast!
Certainly next time!

> Which IDEs do you use?

Personally I've use Emacs from before I was even introduced to Guix, so
it was an obvious choice for me; the same is true for a handful of other
guix-heavy developers at my workplace.

Across the company VS Code and Vim are more popular.  In particular,
making guix-driven development possible with VS Code was key to persuading
the department that Guix would work for everyone professionally.

This wasn't particularly difficult to do, but did require some tweaking and
setup in VS Code - so the average developer would barely know (or care)
that they were running their debug sessions in a Guix environment, or running Git
hooks through Guix, etc.

> Therefore, now I try to run the same but with plain terminal as xterm.
> But that’s not satisfying because they are not used to this interface.

Most of the developers I work with are comfortable using the console so
most of my demos are at the xterm level.  Where it makes sense, I wrap
commands into scripts that can be called from VS Code, and then demo
them from VS Code too.

So far, only 1 or 2 developers will currently go as far as writing
scripts in Guile (for example more advanced/custom manifests), but if I provide
Guile scripts most developers are comfortable knowing what to do with
them - running them from 'guix repl' from example. Developers are
getting more confident/ambitious with more advanced usage, which I
promote as where much of the real power of Guix's approach really lies!

Generally I'm taking care of packages and channels so developers only
need to know which guix-incantation to use an appropriate environment or
profile I maintain for them.

Our CI/CD system automatically updates internally developed packages
when a project feature passes a PR and is merged; so developers just
need to remember to 'guix pull' to make sure they are getting the latest
package dependencies to develop on.  The fact that their work ends up in
a guix package is seamless/transparent to them.  It took me a while to
set this up, but now it runs itself more or less.

> Are these tutorials public?  Are you allowed to share them?

They're not at the moment, but the long term aim is to make them
general (i.e. not specific to the internal setup of the company) and
public, providing I get the blessing of the company. 

>
> One of the main roadblock, even for developers, is the lack «Getting
> Started» for some specific tasks; as Python, R, etc.

Agree 100% - this is what I'd ultimately like to help address.

> Although, at some points, this material provides too much internal
> details. ;-)
>
>
> 1: <https://replay.jres.org/w/3TuYmocHwKtzs7q1VtL1GB>
> 2: <https://zimoun.gitlab.io/jres22-tuto-guix/>

Cool thanks - I will take a look at these!

> I totally agree.  Is these docs too specific to your workplace or can
> they be shared?

In there current form they are very specific to my current workplace -
i.e "to create a development environment for internal project x do these
commands".  The commands assume access to our internal channels etc.
However turning them into a general tutorial would be possible, I
think.  It's something I'd love to do, when I can find the time!

For example I go through recipes for doing things like:

- swapping out a package for a newer beta version to test it

- how to recreate a previous state of our packages to see if a newly
  discovered bug existed in a previous release

- how to test your project against some future state of guix (typically
  we lag the head of guix slightly). 

- how to get a list of dependencies for project x that have changed
  version in the head of guix since we last synchronized channels.

- how to get a list of packages we've defined internally in our private
  channel that (now) also exist in guix's public packages.

All of these things are great to understand deeply, but in reality
most developers just want the commands and a high-level idea of the
mechanics.  So my documents are set at that level - the developers who
get interested I encourage to read the Guix Manual proper and dig a bit
deeper.

> Mathieu recently published [4].
>
> 4: <https://othacehe.org/wsl-images-for-guix-system.html>

Excellent - I will look at this and incorproate into our workflow.


> There is these nice videos [5] which would require some more love.
>
> Well, some materials for producing these kind of videos can be found
> there [6].
>
>
> 5: <https://guix.gnu.org/en/videos/>
> 6: <https://git.savannah.gnu.org/cgit/guix/videos.git>

Thanks for these too.  Very useful

Cheers,
Phil.

Re: Enterprise Guix Hosting?

[zimoun]

Hi Phil,

Thanks for your feedback.  Very helpful and I would be interested by
detailing many items. :-)


Well, for now I am specifically interested in:

On sam., 08 oct. 2022 at 17:23, Phil <phil@beadling.co.uk> wrote:

> Most of the developers I work with are comfortable using the console so
> most of my demos are at the xterm level.  Where it makes sense, I wrap
> commands into scripts that can be called from VS Code, and then demo
> them from VS Code too.

Do you have a Guix recipe for building VS Code?


> So far, only 1 or 2 developers will currently go as far as writing
> scripts in Guile (for example more advanced/custom manifests), but if I provide
> Guile scripts most developers are comfortable knowing what to do with
> them - running them from 'guix repl' from example. Developers are
> getting more confident/ambitious with more advanced usage, which I
> promote as where much of the real power of Guix's approach really lies!

Do you recommend some VS Code plugins for editing Guile Scheme?


Cheers,
simon

Re: Enterprise Guix Hosting?

[(]

Hi zimoun,

On Mon Oct 10, 2022 at 8:58 AM BST, zimoun wrote:
> Do you have a Guix recipe for building VS Code?

*Highly* unlikely. VSCode is an Electron and Typescript bin fire, so packaging
it would take a *very* long time. Unless they used the official VSCode binary
downloads, but those are partly proprietary and include telemetry.

    -- (

Re: Enterprise Guix Hosting?

[zimoun]

Hi,

On Mon, 10 Oct 2022 at 12:30, ( <paren@disroot.org> wrote:

> *Highly* unlikely. VSCode is an Electron and Typescript bin fire, so packaging
> it would take a *very* long time. Unless they used the official VSCode binary
> downloads, but those are partly proprietary and include telemetry.

Thanks for the clarification.

Cheers,
simon

Re: Enterprise Guix Hosting?

[Phil]

Hi Simon

zimoun writes:

>
> Do you have a Guix recipe for building VS Code?

Not for VS Code itself.  People at my workplace tend to use the Windows
install to develop on remote Linux servers that are running Guix as a
package manager.


> Do you recommend some VS Code plugins for editing Guile Scheme?

I tried a few briefly and I was not a fan.  All fell short of the modes
available for emacs IMHO.  Most the developers using VS Code are using
Guix only to create and use environments rather than writing packages
themselves.

Re: Enterprise Guix Hosting?

[Ade Malsasa Akbar]

Hello free software community,

On 26/08/22 01.37, Olivier Dion via wrote:
>    - Root filesystem generation for embedded systems (e.g. Yocto, Elbe)

Sorry for interrupting but I'd love to thank you for mentioning Elbe. A 
new interesting project to me. And to the enterprise Guix hosting plan, 
I wish you all success.

Sincerely yours,

Malsasa

Re: Enterprise Guix Hosting?

[Olivier Dion via]

On Wed, 12 Oct 2022, Ade Malsasa Akbar <malsasa@disroot.org> wrote:
> Hello free software community,
>
> On 26/08/22 01.37, Olivier Dion via wrote:
>>    - Root filesystem generation for embedded systems (e.g. Yocto, Elbe)
>
> Sorry for interrupting but I'd love to thank you for mentioning Elbe. A 
> new interesting project to me. And to the enterprise Guix hosting plan, 
> I wish you all success.

I've been developing Elbe for a few months back in 2019!

Cheers,
old

-- 
Olivier Dion
oldiob.dev

declarative containers (was Re: [EXT] Re: Enterprise Guix Hosting?)

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 1736 bytes --]

Hello everybody,

(this is an old thread started on help-guix [1])

Ludovic Courtès <ludo@gnu.org> writes:

> "Thompson, David" <dthompson2@worcester.edu> skribis:
>
>> On Wed, Aug 31, 2022 at 2:40 AM Ricardo Wurmus <rekado@elephly.net> wrote:
>>>
>>> Another thing that seems to be missing is a way to supervise and manage
>>> running containers.  I use a shepherd instance for this with
>>> container-specific actions like this:

[...]

>> Hey that's a real nice starting point for a container management tool!
>>  So maybe there should be a system service to manage containers and
>> then a 'docker compose'-like tool for declaratively specifying
>> containers and their network bridging configuration that is a client
>> of the service?
>
> Agreed!  We could turn Ricardo’s code into ‘container-guest-service’ or
> something and have ‘containerized-operating-system’ add it
> automatically.

please there was some progress with this service?

once done, could it be possible to declaratively start a whole network
of containers using a dedicated home-service, or
containerized-operating-systems (also on foreign distros)?

right now with "guix system container" we can imperatively manage
(start/stop, connect to the console with nsenter) and connect them
to the network [2], Ricardo showed us how he do it programmatically;
having a declarative interface (os-records) whould be awesome!

I'm very interested and willing to test it, if needed

thanks! Gio'


[1] id:878rn4syql.fsf@elephly.net

[2] thank you Ricardo for the cookbook section!
https://guix.gnu.org/en/cookbook/en/guix-cookbook.html#Guix-System-Containers

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: declarative containers (was Re: [EXT] Re: Enterprise Guix Hosting?)

[Przemysław Kamiński]

[-- Attachment #1.1.1.1.1: Type: text/plain, Size: 2133 bytes --]

On 23.01.2023 16:34, Giovanni Biscuolo wrote:
> Hello everybody,
> 
> (this is an old thread started on help-guix [1])
> 
> Ludovic Courtès <ludo@gnu.org> writes:
> 
>> "Thompson, David" <dthompson2@worcester.edu> skribis:
>>
>>> On Wed, Aug 31, 2022 at 2:40 AM Ricardo Wurmus <rekado@elephly.net> wrote:
>>>>
>>>> Another thing that seems to be missing is a way to supervise and manage
>>>> running containers.  I use a shepherd instance for this with
>>>> container-specific actions like this:
> 
> [...]
> 
>>> Hey that's a real nice starting point for a container management tool!
>>>   So maybe there should be a system service to manage containers and
>>> then a 'docker compose'-like tool for declaratively specifying
>>> containers and their network bridging configuration that is a client
>>> of the service?
>>
>> Agreed!  We could turn Ricardo’s code into ‘container-guest-service’ or
>> something and have ‘containerized-operating-system’ add it
>> automatically.
> 
> please there was some progress with this service?
> 
> once done, could it be possible to declaratively start a whole network
> of containers using a dedicated home-service, or
> containerized-operating-systems (also on foreign distros)?
> 
> right now with "guix system container" we can imperatively manage
> (start/stop, connect to the console with nsenter) and connect them
> to the network [2], Ricardo showed us how he do it programmatically;
> having a declarative interface (os-records) whould be awesome!
> 
> I'm very interested and willing to test it, if needed
> 
> thanks! Gio'
> 
> 
> [1] id:878rn4syql.fsf@elephly.net
> 
> [2] thank you Ricardo for the cookbook section!
> https://guix.gnu.org/en/cookbook/en/guix-cookbook.html#Guix-System-Containers
> 


Does anyone have a simple example of a container with PostgreSQL and 
some web service like Flask? I'm new to Guix and I did see the 
PostgreSQL example that is linked in codebook but I'm missing an example 
of adding a custom service and was a bit overwhelming when I looked at 
the source code.

Best,
Przemek

[-- Attachment #1.1.1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 1227 bytes --]

[-- Attachment #1.1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[-- Attachment #1.2: publickey - cgenie@pm.me - 9cc42b0a.asc --]
[-- Type: application/pgp-keys, Size: 681 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: declarative containers (was Re: [EXT] Re: Enterprise Guix Hosting?)

[Wojtek Kosior via]

[-- Attachment #1: Type: text/plain, Size: 6248 bytes --]

Witaj, Przemku!

I don't have anything with Postgres at hand but I do have a container
definition with services that use Flask[1] :) I also didn't see any
convincing examples of WSGI/CGI applications in Guix so I figured out a
working solution by myself. Here[2] is the definition of one of my WSGI
packages (which is later imported by code in [1]).

My Guix code got quite complex by now. I explain the crucial parts
below.

1. I packaged my Flask apps so that each of them has a valid WSGI
   script in the store. That's tricky because such WSGI script is going
   to be executed by some HTTP server (Apache in my case) which by
   default does not know about the extra Guix stuff that needs to be
   put in the GUIX_PYTHONPATH. There might be different approaches to
   this problem but I solved it by embedding GUIX_PYTHONPATH in the
   very WSGI script. Below I'm quoting the relevant part of my package
   definition from [2].

(arguments
     `(#:phases
       (modify-phases %standard-phases
         (add-after 'unpack 'replace-wsgi.py
           (lambda* (#:key inputs outputs #:allow-other-keys)
             ;; In the wsgi.py file, embed the PYTHONPATH containing both the
             ;; dependencies and the python modules of this package. This will
             ;; make them available at runtime.
             (let ((pythonpath
                    (string-append (getenv "GUIX_PYTHONPATH")
                                   ":"
                                   (site-packages inputs outputs))))
               (substitute* "wsgi.py"
                 (("^from .* import .*" import-line)
                  (string-append
                   "# Make Guix-installed dependencies visible to Python.\n"
                   "import sys\n"
                   "sys.path.extend('" pythonpath "'.split(':'))\n"
                   "\n"
                   import-line))))))
         (add-after 'install 'install-wsgi-script
           (lambda* (#:key inputs outputs #:allow-other-keys)
             (let* ((out (assoc-ref outputs "out"))
                    (share-dir (string-append out "/share/koszko-org-website")))
               (mkdir-p share-dir)
               (copy-file "wsgi.py" (string-append share-dir "/wsgi.py"))))))))

2. In the operating system (well, container...) declaration I make use of
   `file-append` from `(guix gexp)` to construct the WSGI script paths
   for use in HTTP server configuration. In a simplified setting it
   could look more of less like

(define %my-store-file-object
  (file-append package "/share/koszko-org-website/wsgi.py"))

3. I ungexp such objects wherever in the configuration I need the
   actual WSGI script path. In [1] I used Apache and wrote my own helper
   functions and structures to complement its minimal configuration
   system that Guix provides. If (like probably most Guix users out
   there) you are instead using the better-supported Nginx, it should
   be easier and there's no need for you to look into my helper
   functions. If for some awkward reason you want to use Apache like I
   do, feel free to adapt my code from [1].

It might all be a bit overwhelming at first but once you get the grasp
of gexps (described in Guix manual) it gets pretty approachable :)

Feel free to ask again in case I missed some important detail.

Pozderki,
Wojtek

[1] https://git.koszko.org/koszko-org-server/plain/container.scm
[2] https://git.koszko.org/koszko-org-website/plain/guix-module-dir/koszko-org-website.scm

-- (sig_start)
website: https://koszko.org/koszko.html
PGP: https://koszko.org/key.gpg
fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A

♥ R29kIGlzIHRoZXJlIGFuZCBsb3ZlcyBtZQ== | ÷ c2luIHNlcGFyYXRlZCBtZSBmcm9tIEhpbQ==
✝ YnV0IEplc3VzIGRpZWQgdG8gc2F2ZSBtZQ== | ? U2hhbGwgSSBiZWNvbWUgSGlzIGZyaWVuZD8=
-- (sig_end)


On Mon, 23 Jan 2023 16:48:17 +0000
Przemysław Kamiński <cgenie@pm.me> wrote:

> On 23.01.2023 16:34, Giovanni Biscuolo wrote:
> > Hello everybody,
> > 
> > (this is an old thread started on help-guix [1])
> > 
> > Ludovic Courtès <ludo@gnu.org> writes:
> >   
> >> "Thompson, David" <dthompson2@worcester.edu> skribis:
> >>  
> >>> On Wed, Aug 31, 2022 at 2:40 AM Ricardo Wurmus <rekado@elephly.net> wrote:  
> >>>>
> >>>> Another thing that seems to be missing is a way to supervise and manage
> >>>> running containers.  I use a shepherd instance for this with
> >>>> container-specific actions like this:  
> > 
> > [...]
> >   
> >>> Hey that's a real nice starting point for a container management tool!
> >>>   So maybe there should be a system service to manage containers and
> >>> then a 'docker compose'-like tool for declaratively specifying
> >>> containers and their network bridging configuration that is a client
> >>> of the service?  
> >>
> >> Agreed!  We could turn Ricardo’s code into ‘container-guest-service’ or
> >> something and have ‘containerized-operating-system’ add it
> >> automatically.  
> > 
> > please there was some progress with this service?
> > 
> > once done, could it be possible to declaratively start a whole network
> > of containers using a dedicated home-service, or
> > containerized-operating-systems (also on foreign distros)?
> > 
> > right now with "guix system container" we can imperatively manage
> > (start/stop, connect to the console with nsenter) and connect them
> > to the network [2], Ricardo showed us how he do it programmatically;
> > having a declarative interface (os-records) whould be awesome!
> > 
> > I'm very interested and willing to test it, if needed
> > 
> > thanks! Gio'
> > 
> > 
> > [1] id:878rn4syql.fsf@elephly.net
> > 
> > [2] thank you Ricardo for the cookbook section!
> > https://guix.gnu.org/en/cookbook/en/guix-cookbook.html#Guix-System-Containers
> >   
> 
> 
> Does anyone have a simple example of a container with PostgreSQL and 
> some web service like Flask? I'm new to Guix and I did see the 
> PostgreSQL example that is linked in codebook but I'm missing an example 
> of adding a custom service and was a bit overwhelming when I looked at 
> the source code.
> 
> Best,
> Przemek



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 228 bytes --]