Building a software toolchain that works

Building a software toolchain that works

[Ryan Prior]

[-- Attachment #1: Type: text/plain, Size: 2523 bytes --]

I read a Twitter thread just now, which I'll link and reproduce below, that reminds me of something we're trying to build with Guix. Perhaps it'll resonate with other folks here like it did with me.

Jonathan Feinberg wrote at https://twitter.com/pheinberg/status/1503116750203797516

>I literally cannot write software outside of Google anymore. Every C++ project uses its own build engineering. There's Cmake, Ninja, bazel, configure, emsdk, etc. There is no reasonable way to make them coexist.
>
>You can use any 3rd party open source library you like, as long as you use their build, package management scheme, compiler, what have you. But you cannot then use any other open source library.You can use any 3rd party open source library you like, as long as you use their build, package management scheme, compiler, what have you. But you cannot then use any other open source library.
>
>There are 3 things that make it possible to use open source software at Google.
>1) We have 100 or so people who do nothing but work on our software building infrastructure.
>2) We have 10 or so people who do nothing but hack on toolchains.
>3) If you want to use a third-party package at Google, and it hasn't already been brought into our system, you can (well, really, must) do it yourself. You can spend the minutes/hours/days it will take to make just that one project build in our system.
>
>But, once you've done so, it will Just Work® with everything else, and the work you've done will benefit everyone.
>
>This works because there's always only one version of any particular project checked in (modulo some special snowflakes that have to apologize and explain themselves formally in special documents that live in the system with the code).
>
>But I've just tried, for the last time, to write a program outside of that system. It's too frustrating, and I'd rather write songs. This means that my Wordle (the word cloud, remember that from 2008?) will never happen again.
>
>I'll also never make anything like Python Mode for Processing again. Just trying to maintain it means many hours of downloading and configuring the latest hotness in Java versions, IDEs, build tools, etc.
>
>Not worth it. My time on Earth is too short.

I love how easy Guix makes it to build lots of tricky software, and more all the time. Thanks to everybody who's enabled this amazing work and continues to maintain it! Let's double down on that, and keep thinking of ways to improve the software toolchain for the whole world.

Cheers,
Ryan

[-- Attachment #2: Type: text/html, Size: 3296 bytes --]

Re: Building a software toolchain that works

[Pjotr Prins]

On Mon, Mar 14, 2022 at 07:05:30PM +0000, Ryan Prior wrote:
>    I love how easy Guix makes it to build lots of tricky software, and
>    more all the time. Thanks to everybody who's enabled this amazing work
>    and continues to maintain it! Let's double down on that, and keep
>    thinking of ways to improve the software toolchain for the whole world.

Ryan, I totally agree. I can't believe that mess that is Rust cargo
and npm packages today, for example. And they start out as the next
new thing to solve all problems! If they would only would have used
Guix to create a coherent build system... 

And then I spent a week's effort on a CMake build system, and CMake is
an ugly piece of work. I just kept thinking that if Guix was leveraged
you could have a really simple build generator written in Lisp... Now
wouldn't that be nice! 

Pj.

Re: Building a software toolchain that works

[David Arroyo]

A snapshot of the guix repository is similar to a snapshot of Google's
"google3" monorepo. It differs, in that rather than containing the full
source of a specific version of every software package, it encodes
instructions on how to obtain and build a specific version of each
package. This has some tradeoffs; the size of Guix's repo is greatly
reduced, and it is easier for Guix to import new packages by simply
wrapping the build systems they already use, rather than re-implementing
the build logic with blaze/bazel. On the other hand, the decentralized
nature makes it hard for any one actor to make a breaking change. At
Google, if you want to make a breaking change in a library, you can
update all the users of that library in the same commit. I think that
this feature is actually overrated, because it is often complicated by
a need to preserve the ability to roll back a release.

With guix, I know that I am able to build every package in the
distribution, and I don't need to lookup how to use cmake or bazel
or ninja or meson or dune or whatever the author decided to use. I
don't have to figure out how to obtain and build the right versions of
C libraries that a package links to. I just use `guix build`. I think
that is a wonderful thing. Conversely, I am also free to use any build
system that's implemented in guix/build-systems.

I think guix could also greatly lower the barrier to contributing to
free software projects. Since a guix package typically encodes

* How to obtain the software
* How to build the software
* (usually) how to test the software
* (usually) its source repository

One could develop a `guix contribute <package>` subcommand or standalone
script that would implement the following workflow:

1. Obtain the source and unpack it into a writable working directory,
   skipping downstream patches and snippets.
2. Drop the user into a shell with the build and test inputs present.
   Perhaps with the scripts $PWD/.guix/test and $PWD/.guix/build added
   to run the build and test actions as needed.
3. After the user has made changes, and maybe signalled completion
   somehow, run the tests and build all outputs.
4. Once the user exits, output a diff that's suitable for git send-email.

I recently used guix to make a one-line change to the dovecot-pigeonhole
package. Initially I tried to do this without guix, and had difficulties
using autoconf/automake. So instead, I made my changes, captured them with
`git diff`, and then used the `--with-patch` option of `guix install` to
build and install the modified version. It wasn't too hard, but it could
have been easier. It may have not been worth the effort in this case for
someone new to guix, but the nice thing is I could take the same steps
to modify any piece of free software for which a guix package exists,
regardless of the build system in use.

David

Re: Building a software toolchain that works

[Olivier Dion via Development of GNU Guix and the GNU System distribution.]

On Mon, 14 Mar 2022, Pjotr Prins <pjotr.public12@thebird.nl> wrote:

> I can't believe that mess that is Rust cargo and npm packages today,
> for example. And they start out as the next new thing to solve all
> problems! If they would only would have used Guix to create a coherent
> build system...
> 
> And then I spent a week's effort on a CMake build system, and CMake is
> an ugly piece ofp work. I just kept thinking that if Guix was
> leveraged you could have a really simple build generator written in
> Lisp... Now wouldn't that be nice!

I dream of a build system 100% in Scheme that integrates itself natively
with Guix.  No more Makefile, nor auto-nighmares or bloated package
managers that like NPM.  Just a simple self describing Scheme record in
a human readable file that is easily trackabled in Git.

I've made something like this for a project of mine.  Generating a
Makefile from a description of the projects using Scheme.  However, I
believe now that the true power would be to have everything done in
Scheme instead.

On another note, what I find fascinating is why Guix and Nix are not
more used in academic papers.  A quick search on the Compendex database
gives me only a handful of papers referencing Guix, mostly all from
Ludovic.  I simply can't understand this.  You have a way to factor out
the toolchain from the equation of your research's resuls -- making it
trivial to reproduce -- and yet every papers that I read is using some
Ubuntu LTS or Fedora as their build and testing environment.

The same applies for embedded systems.  Why waste time with Yocto, Elbe,
Buildroot anymore?

-- 
Olivier Dion
Polymtl

Re: Building a software toolchain that works

[Ludovic Courtès]

Hi,

"David Arroyo" <david@aqwari.net> skribis:

> One could develop a `guix contribute <package>` subcommand or standalone
> script that would implement the following workflow:
>
> 1. Obtain the source and unpack it into a writable working directory,
>    skipping downstream patches and snippets.
> 2. Drop the user into a shell with the build and test inputs present.
>    Perhaps with the scripts $PWD/.guix/test and $PWD/.guix/build added
>    to run the build and test actions as needed.
> 3. After the user has made changes, and maybe signalled completion
>    somehow, run the tests and build all outputs.
> 4. Once the user exits, output a diff that's suitable for git send-email.

I like that idea, that’d be a great tool to have!  And indeed, we have
all the building blocks.

Ludo’.

Re: Building a software toolchain that works

[Ludovic Courtès]

Hi!

Olivier Dion via "Development of GNU Guix and the GNU System
distribution." <guix-devel@gnu.org> skribis:

> On another note, what I find fascinating is why Guix and Nix are not
> more used in academic papers.  A quick search on the Compendex database
> gives me only a handful of papers referencing Guix, mostly all from
> Ludovic.  I simply can't understand this.  You have a way to factor out
> the toolchain from the equation of your research's resuls -- making it
> trivial to reproduce -- and yet every papers that I read is using some
> Ubuntu LTS or Fedora as their build and testing environment.

This is what we’re pushing as part of the Guix-HPC effort¹.  Ricardo’s
research team, for instance, has published bioinfo papers that build on
Guix for reproducibility.  Colleagues of mine in HPC (linear algebra and
run-time systems) are also starting to do this.

That’s still very much niche, but there’s growing awareness of how tools
like Guix can help build reproducible research workflows.  The next thing
for us is to provide tools and documents to make it more approachable.

Ludo’.

¹ https://hpc.guix.info

Re: Building a software toolchain that works

[Pjotr Prins]

On Tue, Mar 15, 2022 at 09:23:33AM +0100, Ludovic Courtès wrote:
> This is what we’re pushing as part of the Guix-HPC effort¹.  Ricardo’s
> research team, for instance, has published bioinfo papers that build on
> Guix for reproducibility.  Colleagues of mine in HPC (linear algebra and
> run-time systems) are also starting to do this.

We plan to submit a small 'Guix-HPC reproducible cross-build for
RISC-V' workshop paper for OSCAR. 

=> https://www.iscaconf.org/isca2022/program/workshops.php

Papers are not everything. For outreach Blogs are important and
posting to hacker news and such.

Pj.

Re: Building a software toolchain that works

[Yasuaki Kudo]

Hi,

I posted something similar recently but I know there are 100s of banks and hedge funds alike that run daily calculations on arrays of servers , and they have very tricky changes in both software and data, daily.

Do you think Guix(HPC - i don't know what specializations go there 😅) can be the prime tool for the job?

-Yasu

> On Mar 15, 2022, at 17:25, Ludovic Courtès <ludo@gnu.org> wrote:
> 
> Hi!
> 
> Olivier Dion via "Development of GNU Guix and the GNU System
> distribution." <guix-devel@gnu.org> skribis:
> 
>> On another note, what I find fascinating is why Guix and Nix are not
>> more used in academic papers.  A quick search on the Compendex database
>> gives me only a handful of papers referencing Guix, mostly all from
>> Ludovic.  I simply can't understand this.  You have a way to factor out
>> the toolchain from the equation of your research's resuls -- making it
>> trivial to reproduce -- and yet every papers that I read is using some
>> Ubuntu LTS or Fedora as their build and testing environment.
> 
> This is what we’re pushing as part of the Guix-HPC effort¹.  Ricardo’s
> research team, for instance, has published bioinfo papers that build on
> Guix for reproducibility.  Colleagues of mine in HPC (linear algebra and
> run-time systems) are also starting to do this.
> 
> That’s still very much niche, but there’s growing awareness of how tools
> like Guix can help build reproducible research workflows.  The next thing
> for us is to provide tools and documents to make it more approachable.
> 
> Ludo’.
> 
> ¹ https://hpc.guix.info
> 

Re: Building a software toolchain that works

[Josselin Poiret]

Hello everyone,

Let me chime in on a specific point.

"David Arroyo" <david@aqwari.net> writes:
> I recently used guix to make a one-line change to the dovecot-pigeonhole
> package. Initially I tried to do this without guix, and had difficulties
> using autoconf/automake. So instead, I made my changes, captured them with
> `git diff`, and then used the `--with-patch` option of `guix install` to
> build and install the modified version. It wasn't too hard, but it could
> have been easier. It may have not been worth the effort in this case for
> someone new to guix, but the nice thing is I could take the same steps
> to modify any piece of free software for which a guix package exists,
> regardless of the build system in use.

I recently did the same to produce my LUKS2 patches for GRUB [1]!
What I did was hack on the code to add what I needed, produce some
patches with `git format-patch` then define

--8<---------------cut here---------------start------------->8---
(define grub-efi-luks2
  (package/inherit grub-efi
    (source
     (origin
       (inherit (package-source grub-efi))
       (patches (cons* "0001-devmapper-getroot-Have-devmapper-recognize-LUKS2.patch"
                       "0002-devmapper-getroot-Set-up-cheated-LUKS2-cryptodisk-mo.patch"
                       (origin-patches (package-source grub-efi))))))))
--8<---------------cut here---------------end--------------->8---

With this, I could test build failures, but then I also ended up testing
the resulting GRUB in a VM by installing guix with a config.scm that had
the following:

--8<---------------cut here---------------start------------->8---
(operating-system
  (bootloader (bootloader-configuration
                (bootloader
		 (bootloader
                  (inherit grub-efi-bootloader)
                  (package grub-efi-luks2)))
                (target "/boot/efi")
                (keyboard-layout keyboard-layout)))
  ...)
--8<---------------cut here---------------end--------------->8---

This made it extremely easy to test my changes!  I don't think I
would've written these patches without Guix's help.

[1] https://lists.gnu.org/archive/html/grub-devel/2021-12/msg00076.html

Best,
-- 
Josselin Poiret

Re: Building a software toolchain that works

[Ryan Prior]

On Wednesday, March 16th, 2022 at 2:02 PM, Josselin Poiret <dev@jpoiret.xyz> wrote:

> Let me chime in on a specific point.
>[...]
> I don't think I would've written these patches without Guix's help.

This is CRUCIAL to Guix's value proposition: by abstracting away so much of the incidental complexity of software, Guix enables a nimble, carefree mode of hacking where you'll create and improve software that would never have been worth the trouble otherwise.

Proposals like David's `guix contribute` play on the strength of Guix as an enabling technology. That's exactly the kind of thing I meant when I wrote "double down." How can I encourage the improvement & implementation of such proposals? A few notions:

a. My Guile isn't strong enough to put together a proof-of-concept in Guix as a solo effort. I might be able to write a PoC in concert with a more experienced Guiler. We could schedule some strategy & pair programming sessions to get something on track.

b. I could work on a PoC using a language I'm better trained with (elisp, Ruby and JavaScript are my three strongest languages probably.) I don't know how that would be received by the Guix commons, I've felt discouraged in the past to extend Guix unless it's in Guile but perhaps that's a false anxiety.

c. I could also provide some funding if somebody has time & skills to create a PoC and just needs the financial incentive. If this is you, reach out.


I've been pleased to see ideas resonate in this thread. It makes me believe many of us have some form of this thought in our heads and that by socializing we can bring something fresh to the free software world. What are the next steps?

Re: Building a software toolchain that works

[Yasuaki Kudo]

I am very interested in this topic, too!

At work, I have been forced to use the NPM package manager and since the first day I have been unable to understand this enigma - how on earth do people work with this fuzzy system almost designed to be unaccountable??   Can Guix replace NPM?

I think we are really onto something - my hunch is that technicalities are only half of the issue and the other is the community building for democratic and accountable software(for which, concepts like Free Software will fit naturally)

-Yasu


> On Mar 17, 2022, at 00:29, Ryan Prior <rprior@protonmail.com> wrote:
> 
> On Wednesday, March 16th, 2022 at 2:02 PM, Josselin Poiret <dev@jpoiret.xyz> wrote:
> 
>> Let me chime in on a specific point.
>> [...]
>> I don't think I would've written these patches without Guix's help.
> 
> This is CRUCIAL to Guix's value proposition: by abstracting away so much of the incidental complexity of software, Guix enables a nimble, carefree mode of hacking where you'll create and improve software that would never have been worth the trouble otherwise.
> 
> Proposals like David's `guix contribute` play on the strength of Guix as an enabling technology. That's exactly the kind of thing I meant when I wrote "double down." How can I encourage the improvement & implementation of such proposals? A few notions:
> 
> a. My Guile isn't strong enough to put together a proof-of-concept in Guix as a solo effort. I might be able to write a PoC in concert with a more experienced Guiler. We could schedule some strategy & pair programming sessions to get something on track.
> 
> b. I could work on a PoC using a language I'm better trained with (elisp, Ruby and JavaScript are my three strongest languages probably.) I don't know how that would be received by the Guix commons, I've felt discouraged in the past to extend Guix unless it's in Guile but perhaps that's a false anxiety.
> 
> c. I could also provide some funding if somebody has time & skills to create a PoC and just needs the financial incentive. If this is you, reach out.
> 
> 
> I've been pleased to see ideas resonate in this thread. It makes me believe many of us have some form of this thought in our heads and that by socializing we can bring something fresh to the free software world. What are the next steps?
> 

Re: Building a software toolchain that works

[zimoun]

Hi,

On Tue, 15 Mar 2022 at 09:19:50 +0100, Ludovic Courtès <ludo@gnu.org> wrote:

> > One could develop a `guix contribute <package>` subcommand or standalone
> > script that would implement the following workflow:
> >
> > 1. Obtain the source and unpack it into a writable working directory,
> >    skipping downstream patches and snippets.
> > 2. Drop the user into a shell with the build and test inputs present.
> >    Perhaps with the scripts $PWD/.guix/test and $PWD/.guix/build added
> >    to run the build and test actions as needed.
> > 3. After the user has made changes, and maybe signalled completion
> >    somehow, run the tests and build all outputs.
> > 4. Once the user exits, output a diff that's suitable for git send-email.
>
> I like that idea, that’d be a great tool to have!  And indeed, we have
> all the building blocks.

While I agree the idea looks nice, the point #1 does not appear to me
straightforward.

a) The point #1 «skipping downstream patches and snippets» means that
the user would access to upstream source as is.  Therefore, I would like
to point the endless discussion about “guix build --source”.  Because
allowing to obtain the source skipping downstream patches and snippets
would be in contradiction with the rationale behind “guix build -S”,
IIUC.

Personally, I am in favor to have an option to be able to access to the
true upstream source – I have an half-baked script for that :-) – and
not the Guix-modified one – always for good reasons hardly justified.

Reading the lengthy backlog on this very topic, the main argument
against this option (implied by #1) is the poor written FSDG; leading to
various interpretations of the ’spirit’.  All arguments are valid. :-)

My point is, yes we have all the building blocks but, from my
understanding, we disagree about the way – how the end-user would access
to them.

b) Aside this potentially controversial point about «skipping downstream
patches and snippets», a nice workflow could be:

  i. Fetch the source that Guix builds and put it in a writable
  directory.
  
  ii. Extract the current Guix recipe and put it in guix.scm pointing to
  this working directory.

  iii. “guix build -f guix.scm“ or jump to a development environment
  cooked by ‘guix shell’ using this guix.scm

Well, iii) is already done – modulo corner cases, i) is almost ready –
missing glue for moving from the store to a writable directory; maybe a
addition to ‘guix build’

The missing, but not that hard I guess, is ii); maybe an addition to
‘guix edit’ as ‘guix edit --extract’.

The workflow would read:

    guix build --writable-source=/tmp/foo foo
    cd /tmp/foo
    guix edit --extract > guix.scm
    guix shell -Df guix.scm editor git:send-email

    editor bar baz
    git send-email

Personally, I would prefer to do it re-using existing subcommands than
only one doing all in one go.

Cheers,
simon

Re: Building a software toolchain that works

[zimoun]

Hi Olivier,

> On another note, what I find fascinating is why Guix and Nix are not
> more used in academic papers.

Indeed.

One part of the answer is, IMHO: it is difficult to spread the word.

For instance, with co-authors, we have tried to write a short paper
detailing what Guix solves, i.e., the computational environment part of
the “science crisis“, and targeting especially bioinfo folks.  We got
many refusals by the journals that bioinfo folks indeed read and we end
in a “specialized” journal.

On the top of that, add the fact that most of the time, people use what
other people in their lab or collaborators already use.

On the top of that, add the fact that the story of Guix on Windows or
Mac is not really good.  I am not arguing here, just to mention that
many people are still using Windows or Mac and few one Linux variant.

Therefore, all in all, the bootstrap of Guix is hard; as always. :-)

The initiative Guix-HPC is an attempt to address that.  The name is
probably not fully representative since now it looks like Guix in
scientific context; HPC being only one component.

From my point of view, the bootstrap of Guix in scientific world
requires more documentation materials for many common use cases and more
popular applications or usual scientific stack.  For instance PyTorch in
Guix is one step but many things are still really hard to do with Guix
when it is not elsewhere.  Another instance is RStudio for bioinfo folks
– it does not work out of the box with Guix when it does elsewhere.

Help in these both areas – howto materials and popular applications – is
very welcome. :-)

Join the fun, join guix-science@gnu.org :-)


Cheers,
simon

Re: Building a software toolchain that works

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 566 bytes --]

David Arroyo schreef op ma 14-03-2022 om 17:40 [-0400]:
> 1. Obtain the source and unpack it into a writable working directory,
>    skipping downstream patches and snippets.

Why skip downstream patches and snippets?  Often, these are necessry
to make the source actually compileable (e.g. if it uses '/bin/ls'
instead of 'ls', in case of type errrs -- e.g. abiword-explicitly-cast-
bools.patch, autotools changes (agg-am_c_prototype.patch)), remove bugs
(e.g a2ps-CVE-2001-1593)).  These patches and snippets exist for a
reason.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Building a software toolchain that works

[David Arroyo]

On Thu, Mar 17, 2022, at 11:35, Maxime Devos wrote:
> Why skip downstream patches and snippets?

The spirit of the workflow is to enable someone to easily contribute
a patch to an upstream project. With that in mind, I assumed that
our downstream patches would not be accepted by the upstream project,
because they are specific to how guix works. Otherwise, someone would have
submitted them to the upstream project, and they wouldn't be downstream
patches anymore.

Taking the dovecot-pigeonhole example that I used, one of the snippets
removes the rfcs from the source directory for licensing reasons.  If a
contributor accidentally included this change when they sent it to the
upstream, it would distract from the discussion of whatever they wanted
to do.

Of course, many of the changes are necessary to build and use the software
with guix, as you say. And it is trivial enough for the contributor
to scope their patch to the files they actually modified, rather than
whatever the snippets and patches performed. I'm not opposed to including
them.

David

Re: Building a software toolchain that works

[Katherine Cox-Buday]

I run Guix everywhere I can, and it's now the only way I develop software. Having said that, I have thought about this issue a little bit, and here's my opinion on why this happens.

Pjotr Prins <pjotr.public12@thebird.nl> writes:

>  And they start out as the next new thing to solve all problems! If
> they would only would have used Guix to create a coherent build
> system...

Is Guix not a new thing designed to solve all problems? Why aren't we all just using Nix! Or Make!

The reality is that every new system is created because its creators feel that the available systems are deficient in some way. That opinion may not be shared by everyone.

In addition, because free software is largely developed in people's spare time, they're going to use whatever tools make them most productive or even just happy. They're probably not thinking about their software against the backdrop of the larger software ecosystem.

Finally, build tools which are bespoke to a language or use-case are usually going to be easier to work with because they don't have to consider the general or aggregate cases. So if I only live in 1 ecosystem, I'm going to learn that tool and use it any time I can.

Guix solves a lot of issues, and is wonderful to use, but I don't think it solves the most difficult issues: human issues :)

-- 
Katherine

Re: Building a software toolchain that works

[Pjotr Prins]

On Thu, Mar 17, 2022 at 03:04:18PM -0500, Katherine Cox-Buday wrote:
> In addition, because free software is largely developed in people's spare time, they're going to use whatever tools make them most productive or even just happy. They're probably not thinking about their software against the backdrop of the larger software ecosystem.

Agree. Though I would not underestimate these people involved in
creating such ecosystems. Often they are not even on Linux. When the
Dlang people created dub I pointed them to Guix. Obviously I failed to
convince them.

> Guix solves a lot of issues, and is wonderful to use, but I don't think it solves the most difficult issues: human issues :)

It is a complex world out there if you look at the mix of operating
systems and compilers/interpreters. From my point of view GNU Guix
greatly simplies development and deployment - targeting Linux - at the
cost of some up-front investment. It is nice when people realise that
so much complexity goes away living in a Guix world.

Pj.

Re: Building a software toolchain that works

[Oliver Propst]

On 2022-03-18 07:27, Pjotr Prins wrote:
> On Thu, Mar 17, 2022 at 03:04:18PM -0500, Katherine Cox-Buday wrote:
>> In addition, because free software is largely developed in people's 
>> spare time, they're going to use whatever tools make them most 
>> productive or even just happy. They're probably not thinking about 
>> their software against the backdrop of the larger software ecosystem.
> 
> Agree. Though I would not underestimate these people involved in
> creating such ecosystems. Often they are not even on Linux. When the
> Dlang people created dub I pointed them to Guix. Obviously I failed to
> convince them.
> 
>> Guix solves a lot of issues, and is wonderful to use, but I don't 
>> think it solves the most difficult issues: human issues :)
> 
> It is a complex world out there if you look at the mix of operating
> systems and compilers/interpreters. From my point of view GNU Guix
> greatly simplies development and deployment - targeting Linux - at the
> cost of some up-front investment. It is nice when people realise that
> so much complexity goes away living in a Guix world.

Very interesting notes. And well don' think there is an understatement 
what so-ever that I basically agree with points here :)

*I mean just was stated during the Guix online days a few weeks ago I 
think its "simplicity"  that makes Guix such interesting and attractive 
project. And something which in turn I guess makes us motivated to 
contribute and further spread the knowledge about the power of Guix (and 
free software in general for that matter).

-- 
Kinds regards Oliver Propst
https://twitter.com/Opropst

Re: Building a software toolchain that works

[david larsson]

On 2022-03-17 13:56, zimoun wrote:
> Hi Olivier,
> 
>> On another note, what I find fascinating is why Guix and Nix are not
>> more used in academic papers.
> 
> Indeed.
> 
> One part of the answer is, IMHO: it is difficult to spread the word.
> 
> For instance, with co-authors, we have tried to write a short paper
> detailing what Guix solves, i.e., the computational environment part of
> the “science crisis“, and targeting especially bioinfo folks.  We got
> many refusals by the journals that bioinfo folks indeed read and we end
> in a “specialized” journal.
> 
> On the top of that, add the fact that most of the time, people use what
> other people in their lab or collaborators already use.
> 
> On the top of that, add the fact that the story of Guix on Windows or
> Mac is not really good.  I am not arguing here, just to mention that
> many people are still using Windows or Mac and few one Linux variant.
> 
> Therefore, all in all, the bootstrap of Guix is hard; as always. :-)
> 
> The initiative Guix-HPC is an attempt to address that.  The name is
> probably not fully representative since now it looks like Guix in
> scientific context; HPC being only one component.
> 
> From my point of view, the bootstrap of Guix in scientific world
> requires more documentation materials for many common use cases and 
> more
> popular applications or usual scientific stack.  For instance PyTorch 
> in
> Guix is one step but many things are still really hard to do with Guix
> when it is not elsewhere.  Another instance is RStudio for bioinfo 
> folks
> – it does not work out of the box with Guix when it does elsewhere.
> 
> Help in these both areas – howto materials and popular applications – 
> is
> very welcome. :-)
> 
> Join the fun, join guix-science@gnu.org :-)
> 
> 
> Cheers,
> simon

I run Guix including GUI applications from Windows via WSL2 (Windows 
Subsystem for Linux). It may help some to try it out if this setup was 
easier and more documented, though I suppose that is somewhat prevented 
to go via official channels by the FSDG guidelines.

Best regards,
David

Re: Building a software toolchain that works

[Yasuaki Kudo]

Hello David,

I am huge fan of Guix on WSL2 and I used to use it a lot 😄.  And yes, it should be documented well (or even better, the installation should be made super simple) while as you mentioned, it might be easier to do this in another community.

I don't share the ideology of hardline rejection of the use of proprietary software - I just need everything to be accountable, such as knowing which part of the system is Free and which isn't. (and of course more Free the merrier)

Guix, the software, (IMHO, not necessarily the community unless a separate one is created) does the most perfect job for this 😄

-Yasu


> On Mar 19, 2022, at 06:14, david larsson <david.larsson@selfhosted.xyz> wrote:
> 
> On 2022-03-17 13:56, zimoun wrote:
>> Hi Olivier,
>>> On another note, what I find fascinating is why Guix and Nix are not
>>> more used in academic papers.
>> Indeed.
>> One part of the answer is, IMHO: it is difficult to spread the word.
>> For instance, with co-authors, we have tried to write a short paper
>> detailing what Guix solves, i.e., the computational environment part of
>> the “science crisis“, and targeting especially bioinfo folks.  We got
>> many refusals by the journals that bioinfo folks indeed read and we end
>> in a “specialized” journal.
>> On the top of that, add the fact that most of the time, people use what
>> other people in their lab or collaborators already use.
>> On the top of that, add the fact that the story of Guix on Windows or
>> Mac is not really good.  I am not arguing here, just to mention that
>> many people are still using Windows or Mac and few one Linux variant.
>> Therefore, all in all, the bootstrap of Guix is hard; as always. :-)
>> The initiative Guix-HPC is an attempt to address that.  The name is
>> probably not fully representative since now it looks like Guix in
>> scientific context; HPC being only one component.
>> From my point of view, the bootstrap of Guix in scientific world
>> requires more documentation materials for many common use cases and more
>> popular applications or usual scientific stack.  For instance PyTorch in
>> Guix is one step but many things are still really hard to do with Guix
>> when it is not elsewhere.  Another instance is RStudio for bioinfo folks
>> – it does not work out of the box with Guix when it does elsewhere.
>> Help in these both areas – howto materials and popular applications – is
>> very welcome. :-)
>> Join the fun, join guix-science@gnu.org :-)
>> Cheers,
>> simon
> 
> I run Guix including GUI applications from Windows via WSL2 (Windows Subsystem for Linux). It may help some to try it out if this setup was easier and more documented, though I suppose that is somewhat prevented to go via official channels by the FSDG guidelines.
> 
> Best regards,
> David
> 

Windows Subsystem for Linux

[zimoun]

Hi,

> I run Guix including GUI applications from Windows via WSL2 (Windows 
> Subsystem for Linux). It may help some to try it out if this setup was 
> easier and more documented, though I suppose that is somewhat prevented 
> to go via official channels by the FSDG guidelines.

Cool!  I knew that Guix via WSL2 was possible but I was not aware that
it just works for a daily usage. :-)

Yeah, document it via one Cookbook recipe would be great, IMHO.


About FSDG, the arguments are exactly the same as GNU Emacs on
Windows. :-) No?  I mean, GNU Emacs provides a full port on Windows and
such port is available on the same side as other platforms on their
website [1].  The rationale [2]:

        The purpose of the GNU system is to give users the freedom that
        proprietary software takes away from its users. Proprietary
        operating systems (like other proprietary programs) are an
        injustice, and we aim for a world in which they do not exist.

        To improve the use of proprietary systems is a misguided
        goal. Our aim, rather, is to eliminate them. We include support
        for some proprietary systems in GNU Emacs in the hope that
        running Emacs on them will give users a taste of freedom and
        thus lead them to free themselves.

would also make sense for GNU Guix, no?


1: <https://www.gnu.org/software/emacs/>
2: <https://www.gnu.org/software/emacs/download.html#nonfree>

Cheers,
simon


(Parenthesis, if we are picky or finicky, Guix on Windows would be about
the package manager and not the distribution, thus FSDG would not apply
here, right? ;-) Closing parenthesis.)

Re: Windows Subsystem for Linux

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> About FSDG, the arguments are exactly the same as GNU Emacs on
> Windows. :-) No?  I mean, GNU Emacs provides a full port on Windows and
> such port is available on the same side as other platforms on their
> website [1].  The rationale [2]:
>
>         The purpose of the GNU system is to give users the freedom that
>         proprietary software takes away from its users. Proprietary
>         operating systems (like other proprietary programs) are an
>         injustice, and we aim for a world in which they do not exist.
>
>         To improve the use of proprietary systems is a misguided
>         goal. Our aim, rather, is to eliminate them. We include support
>         for some proprietary systems in GNU Emacs in the hope that
>         running Emacs on them will give users a taste of freedom and
>         thus lead them to free themselves.
>
> would also make sense for GNU Guix, no?

Just like the Emacs manual has an appendix on Windows support, I think
the cookbook could have a section on setting up Guix on WSL2.

Cc’ing maintainers for their thoughts!

Thanks,
Ludo’.

Re: Windows Subsystem for Linux

[Maxim Cournoyer]

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> About FSDG, the arguments are exactly the same as GNU Emacs on
>> Windows. :-) No?  I mean, GNU Emacs provides a full port on Windows and
>> such port is available on the same side as other platforms on their
>> website [1].  The rationale [2]:
>>
>>         The purpose of the GNU system is to give users the freedom that
>>         proprietary software takes away from its users. Proprietary
>>         operating systems (like other proprietary programs) are an
>>         injustice, and we aim for a world in which they do not exist.
>>
>>         To improve the use of proprietary systems is a misguided
>>         goal. Our aim, rather, is to eliminate them. We include support
>>         for some proprietary systems in GNU Emacs in the hope that
>>         running Emacs on them will give users a taste of freedom and
>>         thus lead them to free themselves.
>>
>> would also make sense for GNU Guix, no?
>
> Just like the Emacs manual has an appendix on Windows support, I think
> the cookbook could have a section on setting up Guix on WSL2.
>
> Cc’ing maintainers for their thoughts!

I think if it's worded correctly (e.g., by hinting that WSL2 is an
inferior means of running Guix than via Guix System or one of the others
GNU/Linux distributions), it can be a fine and useful addition to the
cookbook.  I have fellow coworkers (not at Savoir-faire Linux, mind you
:-)) who have yet to escape their Windows jail; giving them a taste of
what is possible out there with Guix would probably be a real eye
opener.  At the minimum it would present Guix as a technically viable
alternative to Docker & friends in these circles.

Thanks,

Maxim

Re: Windows Subsystem for Linux

[Phil]

Maxim Cournoyer writes:

> cookbook.  I have fellow coworkers (not at Savoir-faire Linux, mind you
> :-)) who have yet to escape their Windows jail; giving them a taste of
> what is possible out there with Guix would probably be a real eye
> opener.  At the minimum it would present Guix as a technically viable
> alternative to Docker & friends in these circles.
>

I totally agree with this - at the moment setting on WSL2 (AFAIK) you
have 2 options:

1. Run it on-top of Ubuntu and tweak the various daemons to start on
image boot.

2. Use Busybox to bootstrap a bare-metal install of Guix without a host
Linux.


The main problem I've had with 1 is that the resulting image is produced
is very large and does not reliably install on colleague's machines.
That and Ubuntu of course is just a distraction when trying to showcase
a Guix workflow.

The problem with 2 is that last time I tried (about a year ago), this
requires you to construct your Guix image for scratch - which is an
interesting exercise but will put off the majority of users:
https://gist.github.com/giuliano108/49ec5bd0a9339db98535bc793ceb5ab4

What we need is a WSL2 image like other OSs provided which can just be
import directly into WSL2 in a push-button fashion.  I don't think
this would be particularly difficult to do, and would encourage more
people to try Guix.

It would also be useful for people who already use Guix, but have no
option but to use Windows in some circumstances - WSL2 is already very
popular to escape Windows into Linux in day-to-day workflows, when you
are unable to have a Linux desktop, and has become widely adopted even in
companies that have a Windows-only desktop policy.

Having Guix images to download and install for WSL2 would make it very
easy to showcase Guix without asking people to risk a physical
install (which can be tricky to get hardware working), or even to adopt the
full Graphical Desktop. Users don't need admin rights to install a WSL2
image, and assuming WSL2 is already enabled (for Ubuntu or whatever) it
is trivial to add any other image.  It would be nice complement to the QCOW images
already available for QEMU.

Getting ahead of myself :-) I also think providing cloud images in
formats like Amazon's AMI would encourage people to spin-up Guix and
give it whirl.  And even looking at Ubuntu's "multipass" to easily bring
Ubuntu to Windows/Mac/Linux is another way to make it easy to try Ubuntu
on any other OS with a lightweight install.

Re: Windows Subsystem for Linux

[Oliver Propst]

Yeah I very much agree about that it could be good idea idea to have the 
Guix system available as a Wsl 2 image
as I think could serve as a good gateway to many people to learn more 
about Guix and maybe free and open source software in general for that 
matter.


Kinds regards Oliver Propst
https://twitter.com/Opropst

Re: Windows Subsystem for Linux

[Blake Shaw]

Hi,
Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> About FSDG, the arguments are exactly the same as GNU Emacs on
>> Windows. :-) No?  I mean, GNU Emacs provides a full port on Windows and
>> such port is available on the same side as other platforms on their
>> website [1].  The rationale [2]:
>>
>>         The purpose of the GNU system is to give users the freedom that
>>         proprietary software takes away from its users. Proprietary
>>         operating systems (like other proprietary programs) are an
>>         injustice, and we aim for a world in which they do not exist.
>>
>>         To improve the use of proprietary systems is a misguided
>>         goal. Our aim, rather, is to eliminate them. We include support
>>         for some proprietary systems in GNU Emacs in the hope that
>>         running Emacs on them will give users a taste of freedom and
>>         thus lead them to free themselves.
>>
>> would also make sense for GNU Guix, no?
>
> Just like the Emacs manual has an appendix on Windows support, I think
> the cookbook could have a section on setting up Guix on WSL2.
>
> Cc’ing maintainers for their thoughts!
>
> Thanks,
> Ludo’.
>

I just had a client inquire about deploying to WSL2, so it's great news
to hear that this is working!

Hopefully it will mean that as the Guix tentacles extend, the
obsolescence, redundance, inconvenience & expense of proprietary
"middleware" like Windows will become clear to users.

May Guix/Guile march forward on the path of ubiquitous extensionality! 

ez,
b

-- 
“In girum imus nocte et consumimur igni”