Internship coordination status report

Internship coordination status report

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 1490 bytes --]

Hello Guix,

Here follows a short summary on where we are regarding the internship
programs.

GSoC:

This year the GNU Project was applying again for GSoC, and
we got the word yesterday that the GNU Project was accepted.

Thanks to Jose E. Marches for taking care of this.

This basically means that we should have a look at our ideas page,
and run a quick health check on the proposals, enriching them with
the following information where needed:

(quoted from the mail on the summer of code list)
  + Title, and description.
  + Skills required.
  + A mentor with an email address.
  + Whether it is a 175 hour or a 350 hour project.
  + Difficulty: easy, medium or hard.
  + CLEAR contact method for interested students.




From now to March 19, be ready to be contacted by contributors, in
whatever contact means you specified in your ideas page.  Starting in
March 20, contributors will start submitting their proposals through the
program website.
(end quote)

Next week I am on holiday, so I asked Pjotr Prins to take over the initial
coordination tasks until I am back on 6th of March, 2023.
Outreachy:

Coming up with a working solution for funding here is far from trivial.
Thanks to Andreas Enge and Simon Touriner now all stakeholders are involved
and
we are working on a solution. I don't want to bore the list with the details
here, and I will send an update once we have an agreement on how to
handle it.

Thanks to everyone who is helping this effort.

Best regards,
g_bor

[-- Attachment #2: Type: text/html, Size: 2227 bytes --]

Re: Internship coordination status report

[Simon Tournier]

Hi,

Thanks Gábor for mastering the organisation. :-)


On Fri, 24 Feb 2023 at 19:21, Gábor Boskovits <gboskovits@gmail.com> wrote:

> This year the GNU Project was applying again for GSoC, and
> we got the word yesterday that the GNU Project was accepted.

Congrats!  \o/

> Thanks to Jose E. Marches for taking care of this.

Kudos! :-)


> This basically means that we should have a look at our ideas page,
> and run a quick health check on the proposals, enriching them with
> the following information where needed:
>
> (quoted from the mail on the summer of code list)
>   + Title, and description.
>   + Skills required.
>   + A mentor with an email address.
>   + Whether it is a 175 hour or a 350 hour project.
>   + Difficulty: easy, medium or hard.
>   + CLEAR contact method for interested students.

For now, there is this wiki page:

    https://libreplanet.org/wiki/Group:Guix/GSoC-2023

Now, we have to make sure that mentors are available (and just will).


> From now to March 19, be ready to be contacted by contributors, in
> whatever contact means you specified in your ideas page.

Maybe, it would be easy to make guix-devel@gnu.org as clear contact, or
a private alias (Gábor, Pjotr and other interested for helping in the
organization) and then privately redirect candidate to mentors; because
otherwise it appears to me difficult to be “clearly contacted”. :-)


> Outreachy:

> we are working on a solution. I don't want to bore the list with the details
> here, and I will send an update once we have an agreement on how to
> handle it.

Thanks for pushing forward.  I hope we will be able to make a temporary
fix using Guix Europe [1]. :-)

1: <http://foundation.guix.info/>

Cheers,
simon

Projects for the Google Summer of Code

[Pjotr Prins]

To follow up on Gábor: we excited to announce that the GNU project is
selected as an organisation for the Google Summer of Code (GSoc). GSoC
attracts talent and we have some people in our group who came in
through that route! I have done 10 years of GSoC and it has paid off
in my projects. You can still add ideas to:

https://libreplanet.org/wiki/Group:Guix/GSoC-2023

Currently only pukkamustard's project is listed, feel free to add
more. You can register/login yourself. Make sure to add below
information - title, email, hours etc. Tasks can be challenging - you
get more interesting applicants that way. Also add simple tasks, that
helps attract learners in Lisp, for example.

The overall GNU project proposals, including GNU Mes, are listed at:

https://www.gnu.org/software/soc-projects/ideas-2023.html

People are already inquiring, so today is a good day to add your idea.

When it comes to mentoring, in my experience it takes a few hours a
week that have a decent ROI. Before GSoC starts we'll have a get
together to share experiences and ideas.

Pj.

On Fri, Feb 24, 2023 at 07:21:51PM +0100, Gábor Boskovits wrote:
>    Hello Guix,
>    Here follows a short summary on where we are regarding the internship
>    programs.
>    GSoC:
>    This year the GNU Project was applying again for GSoC, and
>    we got the word yesterday that the GNU Project was accepted.
>    Thanks to Jose E. Marches for taking care of this.
>    This basically means that we should have a look at our ideas page,
>    and run a quick health check on the proposals, enriching them with
>    the following information where needed:
>    (quoted from the mail on the summer of code list)
>      + Title, and description.
>      + Skills required.
>      + A mentor with an email address.
>      + Whether it is a 175 hour or a 350 hour project.
>      + Difficulty: easy, medium or hard.
>      + CLEAR contact method for interested students.
>    From now to March 19, be ready to be contacted by contributors, in
>    whatever contact means you specified in your ideas page.  Starting in
>    March 20, contributors will start submitting their proposals through
>    the
>    program website.
>    (end quote)
>    Next week I am on holiday, so I asked Pjotr Prins to take over the
>    initial
>    coordination tasks until I am back on 6th of March, 2023.
>    Outreachy:
>    Coming up with a working solution for funding here is far from trivial.
>    Thanks to Andreas Enge and Simon Touriner now all stakeholders are
>    involved and
>    we are working on a solution. I don't want to bore the list with the
>    details
>    here, and I will send an update once we have an agreement on how to
>    handle it.
>    Thanks to everyone who is helping this effort.
>    Best regards,
>    g_bor

Re: Projects for the Google Summer of Code

[Kyle]

[-- Attachment #1: Type: text/plain, Size: 5256 bytes --]

Dear Guix,

 I have been reading on this list of Andreas struggles updating python packages as an outsider to that community. If python programmers are not sticking around, then that may be sign they could use more help.

Python provides access to an unparalleled amount of functionality which greatly empowers users of free software. I want that functionality in Guix for everyone to benefit from. No one person can do it all themselves. There needs to be a healthy community.

I personally would love to see more support for helping scientists port their existing workflows from conda environments to actually reproducible containers, e.g. meeting them where they are and giving them more options for working with older versions of python3.

One idea might be to write a conda importer which looks at the versions of software in the resulting environment and tries to make feasible package variants of make a manifest which matches the existing conda environment as close as possible. Simon thought often the python version usually didn't matter, but it makes users a lot less woosy to stick with software combinations they have already tested.

Another idea would be to create a python package for working with Guix more directly from inside their preferred language environment instead of through the shell. I also wouldn't mind it if such a thing existed for R as well. Both python and R extensively interface with C. On the R side there is an RcppGuile proof of concept package for example. Maybe there could be a project around curating a standard non-bash interface of guile scripts for these and other interactive languages? The rational is that users in these ecosystems often don't have much familiarity with shell programming. Learning both shell and scheme at the same time can be very intimidating and demands a large upfront effort before the value case is clear.

As an enthusiastic beginner, I want to help as much as I can to make Guix better, but I  am not an expert. I just wanted to throw those ideas out there for you real experts to consider and maybe translate into more interesting technical goals you think would help the python community (and other communities) become more engaged.

Cheers,
Kyle



On February 26, 2023 2:38:36 AM EST, Pjotr Prins <pjotr.public12@thebird.nl> wrote:
>To follow up on Gábor: we excited to announce that the GNU project is
>selected as an organisation for the Google Summer of Code (GSoc). GSoC
>attracts talent and we have some people in our group who came in
>through that route! I have done 10 years of GSoC and it has paid off
>in my projects. You can still add ideas to:
>
>https://libreplanet.org/wiki/Group:Guix/GSoC-2023
>
>Currently only pukkamustard's project is listed, feel free to add
>more. You can register/login yourself. Make sure to add below
>information - title, email, hours etc. Tasks can be challenging - you
>get more interesting applicants that way. Also add simple tasks, that
>helps attract learners in Lisp, for example.
>
>The overall GNU project proposals, including GNU Mes, are listed at:
>
>https://www.gnu.org/software/soc-projects/ideas-2023.html
>
>People are already inquiring, so today is a good day to add your idea.
>
>When it comes to mentoring, in my experience it takes a few hours a
>week that have a decent ROI. Before GSoC starts we'll have a get
>together to share experiences and ideas.
>
>Pj.
>
>On Fri, Feb 24, 2023 at 07:21:51PM +0100, Gábor Boskovits wrote:
>>    Hello Guix,
>>    Here follows a short summary on where we are regarding the internship
>>    programs.
>>    GSoC:
>>    This year the GNU Project was applying again for GSoC, and
>>    we got the word yesterday that the GNU Project was accepted.
>>    Thanks to Jose E. Marches for taking care of this.
>>    This basically means that we should have a look at our ideas page,
>>    and run a quick health check on the proposals, enriching them with
>>    the following information where needed:
>>    (quoted from the mail on the summer of code list)
>>      + Title, and description.
>>      + Skills required.
>>      + A mentor with an email address.
>>      + Whether it is a 175 hour or a 350 hour project.
>>      + Difficulty: easy, medium or hard.
>>      + CLEAR contact method for interested students.
>>    From now to March 19, be ready to be contacted by contributors, in
>>    whatever contact means you specified in your ideas page.  Starting in
>>    March 20, contributors will start submitting their proposals through
>>    the
>>    program website.
>>    (end quote)
>>    Next week I am on holiday, so I asked Pjotr Prins to take over the
>>    initial
>>    coordination tasks until I am back on 6th of March, 2023.
>>    Outreachy:
>>    Coming up with a working solution for funding here is far from trivial.
>>    Thanks to Andreas Enge and Simon Touriner now all stakeholders are
>>    involved and
>>    we are working on a solution. I don't want to bore the list with the
>>    details
>>    here, and I will send an update once we have an agreement on how to
>>    handle it.
>>    Thanks to everyone who is helping this effort.
>>    Best regards,
>>    g_bor
>

[-- Attachment #2: Type: text/html, Size: 5697 bytes --]

Re: Projects for the Google Summer of Code

[Simon Tournier]

Hi,


On Sun, 26 Feb 2023 at 16:52, Kyle <kyle@posteo.net> wrote:

> One idea might be to write a conda importer which looks at the
> versions of software in the resulting environment and tries to make
> feasible package variants of make a manifest which matches the
> existing conda environment as close as possible.

Do you mean, from ’conda env export’ Which outputs something like, 

    name: justnumpy
    channels:
      - defaults
    dependencies:
      - _libgcc_mutex=0.1=main
      - _openmp_mutex=5.1=1_gnu
      - blas=1.0=mkl
      - libuuid=1.41.5=h5eee18b_0
      - mkl=2021.4.0=h06a4308_640
      - mkl-service=2.4.0=py310h7f8727e_0
      - mkl_fft=1.3.1=py310hd6ae3a3_0
      - mkl_random=1.2.2=py310h00e6091_0
      - ncurses=6.3=h5eee18b_3
      - numpy=1.23.4=py310hd5efca6_0
      - numpy-base=1.23.4=py310h8e6c178_0
      - ...
      - ...

transform this into some Guix manifest.scm?  Well, indeed, it could help
people for transitioning.


> Another idea would be to create a python package for working with Guix
> more directly from inside their preferred language environment instead
> of through the shell. I also wouldn't mind it if such a thing existed
> for R as well.

Do you mean ’guix.install()’ from r-guix-install?

https://cran.rstudio.com/web/packages/guix.install/index.html

How do you install Python packages from the Python REPL?


Cheers,
simon

PS:

>                                                  Simon thought often
> the python version usually didn't matter, but it makes users a lot
> less woosy to stick with software combinations they have already
> tested.

This is out of context. :-)  For the context, see [1].

Well, if you have only one Python version, you only test against this
one, so being able to combine on the fly the matrix of Python versions
is not so much important – it is a collective practise inherited from
the “unstable” Python ecosystem and I am doubtful about its relevancy
concerning referencing software for reproducing.  Now Conda is more than
10 years and very very broadly used, so if its specification using
version labels and relying on some matrix of Python versions would be
enough for reproducibility, then I would have never landed in Guix. ;-)

1: https://yhetil.org/guix/86pma2t3jd.fsf@gmail.com

My 2 cents. :-)

Re: Projects for the Google Summer of Code

[Kyle Andrews]

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
>
> On Sun, 26 Feb 2023 at 16:52, Kyle <kyle@posteo.net> wrote:
>
>> One idea might be to write a conda importer which looks at the
>> versions of software in the resulting environment and tries to make
>> feasible package variants of make a manifest which matches the
>> existing conda environment as close as possible.
>
> Do you mean, from ’conda env export’ Which outputs something like, 
>
>     name: justnumpy
>     channels:
>       - defaults
>     dependencies:
>       - _libgcc_mutex=0.1=main
>       - _openmp_mutex=5.1=1_gnu
>       - blas=1.0=mkl
>       - libuuid=1.41.5=h5eee18b_0
>       - mkl=2021.4.0=h06a4308_640
>       - mkl-service=2.4.0=py310h7f8727e_0
>       - mkl_fft=1.3.1=py310hd6ae3a3_0
>       - mkl_random=1.2.2=py310h00e6091_0
>       - ncurses=6.3=h5eee18b_3
>       - numpy=1.23.4=py310hd5efca6_0
>       - numpy-base=1.23.4=py310h8e6c178_0
>       - ...
>       - ...
>
> transform this into some Guix manifest.scm?  Well, indeed, it could help
> people for transitioning.

Yes, exactly. Parse the output of this command and generate a bunch of
definitions for a manifest file but which might also be upstreamed to
the guix-past repository.

>> Another idea would be to create a python package for working with Guix
>> more directly from inside their preferred language environment instead
>> of through the shell. I also wouldn't mind it if such a thing existed
>> for R as well.
>
> Do you mean ’guix.install()’ from r-guix-install?
>
> https://cran.rstudio.com/web/packages/guix.install/index.html
>
> How do you install Python packages from the Python REPL?

With the reticulate package in R, a python environment can be readily
instantiated using R code like `use_condaenv`.

https://rstudio.github.io/reticulate/articles/versions.html

In my dreams there would be a `use_guix()` command as well. Of course,
right now that R code is limited to setting up python-specific
environments and conda environments (and installing python packages into
them). However, it still might be valuable to get the name recognition
for Guix in this limited context.

While guix would be responsible for just the python environment in the
context of the reticulate package, that isn't the only possible use
case. It would be nice to write that integration for reticulate by
having them use another R package with deeper integrations built out for
managing guix itself from R. That's why I brought up RcppGuile as a path
for how R could interface directly with the Guix scheme libraries. Such
an interface might be useful for other interactive scientific
environments like python as well. The goal would be to make Guix seem
less exotic to researchers in typical scientific languages. If it's part
of their "home" computing environment, then they might be much more
interested in trying it out. This reflects a bigger scope than just
being a replacement for install.packages.

> Cheers,
> simon
>
> PS:
>
>>                                                  Simon thought often
>> the python version usually didn't matter, but it makes users a lot
>> less woosy to stick with software combinations they have already
>> tested.
>
> This is out of context. :-)  For the context, see [1].

Thanks for the point of clarification. Sorry, I didn't add the reference
in my initial message.

> Well, if you have only one Python version, you only test against this
> one, so being able to combine on the fly the matrix of Python versions
> is not so much important – it is a collective practise inherited from
> the “unstable” Python ecosystem and I am doubtful about its relevancy
> concerning referencing software for reproducing.  Now Conda is more than
> 10 years and very very broadly used, so if its specification using
> version labels and relying on some matrix of Python versions would be
> enough for reproducibility, then I would have never landed in Guix. ;-)
>
> 1: https://yhetil.org/guix/86pma2t3jd.fsf@gmail.com
>
> My 2 cents. :-)

Later in that conversation you made the point that the purpose of the
guix-past channel was to make things like this possible. I added my
voice to this (GSOC Project) thread because I thought it would be useful
to place a fresh pair of eyes at tackling the combinatorial
configuration problems which still stand in the way of curating a large
Guix package repository with the breadth of scientific package versions
that a platform like conda provides... even if it cheats a lot by not
doing that reproducibly. Whatever they learn might also help elsewhere
in the project, such as potentially helping to curate large collections
of packages in other languages like those in Go, Rust, or even
JavaScript.

Re: Projects for the Google Summer of Code

[Simon Tournier]

Hi,

On Fri, 3 Mar 2023 at 01:51, Kyle Andrews <kyle@posteo.net> wrote:

> > Do you mean ’guix.install()’ from r-guix-install?
> >
> > https://cran.rstudio.com/web/packages/guix.install/index.html
> >
> > How do you install Python packages from the Python REPL?
>
> With the reticulate package in R, a python environment can be readily
> instantiated using R code like `use_condaenv`.
>
> https://rstudio.github.io/reticulate/articles/versions.html

IIUC, this install Python packages from R REPL and not from Python
REPL.  Maybe I am missing something but I think 'guix.install' already
does that.

> In my dreams there would be a `use_guix()` command as well. Of course,
> right now that R code is limited to setting up python-specific
> environments and conda environments (and installing python packages into
> them). However, it still might be valuable to get the name recognition
> for Guix in this limited context.

Again, maybe I am missing a point but, IMHO, 'guix.install' is the
'use_guix' you are describing.

> While guix would be responsible for just the python environment in the
> context of the reticulate package, that isn't the only possible use
> case. It would be nice to write that integration for reticulate by
> having them use another R package with deeper integrations built out for
> managing guix itself from R. That's why I brought up RcppGuile as a path
> for how R could interface directly with the Guix scheme libraries. Such
> an interface might be useful for other interactive scientific
> environments like python as well. The goal would be to make Guix seem
> less exotic to researchers in typical scientific languages. If it's part
> of their "home" computing environment, then they might be much more
> interested in trying it out. This reflects a bigger scope than just
> being a replacement for install.packages.

From my understanding, the path would be 'guix.install', but maybe I
am missing details.  And I am not following the point of RcppGuile,
instead I would go via some 'emulate-fhs' as "guix shell" provides.

Anyway, in all cases, it could nice to fix these bugs related to Conda
on the top of Guix:

https://issues.guix.gnu.org/issue/59776
https://issues.guix.gnu.org/issue/59775
https://issues.guix.gnu.org/issue/59774
https://issues.guix.gnu.org/issue/59772


> Later in that conversation you made the point that the purpose of the
> guix-past channel was to make things like this possible. I added my
> voice to this (GSOC Project) thread because I thought it would be useful
> to place a fresh pair of eyes at tackling the combinatorial
> configuration problems which still stand in the way of curating a large
> Guix package repository with the breadth of scientific package versions
> that a platform like conda provides... even if it cheats a lot by not
> doing that reproducibly. Whatever they learn might also help elsewhere
> in the project, such as potentially helping to curate large collections
> of packages in other languages like those in Go, Rust, or even
> JavaScript.

I agree that working around Conda could pay off for Reproducible
Research.  For example, you might be interested by the thread starting
here:

https://lists.gnu.org/archive/html/guix-science/2022-11/msg00009.html

continuing there:

https://lists.gnu.org/archive/html/guix-science/2022-12/msg00000.html

Last, Thibault launched a Git repository with CI trying to find after
which weeks or months or years? a Conda environment as you described
earlier breaks, see:

https://framagit.org/tlestang/conda-python-example/-/pipelines

The current CI tests the resolver part of Conda as discussed in the
thread above and that's cool!  :-)

Other tests could be added, as changing the "base" computational
environment (image) [1].  What happens if the Docker image
'continuumio/miniconda3:4.12.0' is lost?  For example, it could also
be tested with other 'continuumio/miniconda3' images.  Or even install
Conda on the top of say various Debian or other "base" image.  Well,
using a simple example (numpy+matplotlib) and try to run it on various
computational environments management by Conda; mimicking what we do
most of the time as scientific researchers.

1: https://framagit.org/tlestang/conda-python-example/-/blob/main/.gitlab-ci.yml#L1

Well, for what it is worth, I am trying to do these kind of tests for
Guix -- rebuild an old computational environment from 2020 using
current state of the world -- as reported here:

https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00398.html
https://lists.gnu.org/archive/html/guix-devel/2023-03/msg00007.html

And I also proposed to work on that as GSoC:
https://libreplanet.org/wiki/Group:Guix/GSoC-2023
Feel free to add you as co-mentor. :-)

Cheers,
simon