Guix to participate in the Google Summer of Code

Guix to participate in the Google Summer of Code

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

Hello Guix!

GNU has been accepted as an umbrella organization for GSoC.  That means
that Guix can participate in GSoC under the aegis of GNU, like we did in
previous years.

For that we need mentors: see below the info you need to send if you
wish to be a mentor.  Also, don’t hesitate to ask if you’re unsure what
it means to be a mentor, or if you plan to have vacations during summer
(I do :-)), we can certainly find a co-mentor.

We also need topics: consider updating
<https://libreplanet.org/wiki/Group:Guix/GSoC-2018>.

Last but not least, we need candidates: if you’re a student lurking,
let’s get in touch; if you’re a contributor, spread the word!

Ludo’.


[-- Attachment #2: Type: message/rfc822, Size: 5484 bytes --]

From: Giuseppe Scrivano <gscrivano@gnu.org>
To: GNU Summer Of Code <summer-of-code@gnu.org>
Subject: [IMPORTANT] GNU has been accepted & mentors registration
Date: Mon, 12 Feb 2018 19:37:20 +0100
Message-ID: <87y3jy836n.fsf@redhat.com>

Hi everyone,

I just got the confirmation the GNU project was accepted this year.

Now we can add mentors.  Everyone who wants to be a mentor should fill
and send this information off-list to Darshit, Jose, myself:

-----------------------------
Name:
Email:
AltEmail:
Phone:
Project:
-----------------------------

Thanks,
Giuseppe

Re: Guix to participate in the Google Summer of Code

[Alex Vong]

Hello,

ludo@gnu.org (Ludovic Courtès) writes:

[...]
>
> We also need topics: consider updating
> <https://libreplanet.org/wiki/Group:Guix/GSoC-2018>.
>
> Last but not least, we need candidates: if you’re a student lurking,
> let’s get in touch; if you’re a contributor, spread the word!
>
> Ludo’.
>
[...]

I want to be a student this year since this will be my final chance (I
am graduating this year). I look at the libreplanet page and I think
only the build daemon project is suitable for me since I don't have
knowledge in networking. Is it possible for me to work on last-year
Bournish project?

I am taking a compiler course right now (sadly it is in Java not ML). I
am more confident in working on Bournish than working on the build
daemon. Well, I would have to study how garbage collection works if I
were to work on the build daemon (of course this could be fun as well).

Cheers,
Alex

Re: Guix to participate in the Google Summer of Code

[Ricardo Wurmus]

Hi Alex,

> I want to be a student this year since this will be my final chance (I
> am graduating this year). I look at the libreplanet page and I think
> only the build daemon project is suitable for me since I don't have
> knowledge in networking. Is it possible for me to work on last-year
> Bournish project?

The Bournish project is poorly defined now that gash exists, a Bash
interpreter written in Guile.  It wouldn’t be a lot of work to integrate
gash into Guix and replace Bournish; in fact I wanted to do this myself
in a few weeks.

I’m afraid that the Bournish project would no longer be suitable for
GSoC, and that’s why it had been removed from the list of project ideas.

The guix-daemon project is larger but also much more useful to us.
Garbage collection in the context of Guix is not very difficult to
understand.  The garbage collector may remove any item that has zero
users; the link count on a store item is indicative of the number of
users that item has.  For garbage collection you essentially go through
all items and check if the link count is zero.

The reference scanner is already implemented in the form of grafts.  The
code opens each file and searches efficiently for store references.

For that project it is best to spend a day or two browsing the existing
Guile sources and noting the many things that already implement large
parts of the daemon features.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Guix to participate in the Google Summer of Code

[Alex Vong]

Hello Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Alex,
>
>> I want to be a student this year since this will be my final chance (I
>> am graduating this year). I look at the libreplanet page and I think
>> only the build daemon project is suitable for me since I don't have
>> knowledge in networking. Is it possible for me to work on last-year
>> Bournish project?
>
> The Bournish project is poorly defined now that gash exists, a Bash
> interpreter written in Guile.  It wouldn’t be a lot of work to integrate
> gash into Guix and replace Bournish; in fact I wanted to do this myself
> in a few weeks.
>
OK, so we already have a working Bash interpreter written in Guile. It
makes sense to drop the old plan then.

> I’m afraid that the Bournish project would no longer be suitable for
> GSoC, and that’s why it had been removed from the list of project ideas.
>
I see. So this is why I think I have seen it before. It was on the list
of idea but then get removed.

> The guix-daemon project is larger but also much more useful to us.
> Garbage collection in the context of Guix is not very difficult to
> understand.  The garbage collector may remove any item that has zero
> users; the link count on a store item is indicative of the number of
> users that item has.  For garbage collection you essentially go through
> all items and check if the link count is zero.
>
I will join the guix-daemon project then. My understading is that the
garbage collector also uses info from 'list-runtime-roots', right? So
together we have registered roots and runtime roots.

> The reference scanner is already implemented in the form of grafts.  The
> code opens each file and searches efficiently for store references.
>
Does this mean we have procedure to do a single find-reference and the
remaining work is to build a recursive find-reference?

> For that project it is best to spend a day or two browsing the existing
> Guile sources and noting the many things that already implement large
> parts of the daemon features.
>
> --
> Ricardo
>
> GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
> https://elephly.net

Cheers,
Alex

Re: Guix to participate in the Google Summer of Code

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]

Ricardo Wurmus <rekado@elephly.net> writes:

> Garbage collection in the context of Guix is not very difficult to
> understand.  The garbage collector may remove any item that has zero
> users; the link count on a store item is indicative of the number of
> users that item has.  For garbage collection you essentially go through
> all items and check if the link count is zero.

Is this correct?  My understanding is that the link count of a store
path (i.e., the count of hard links for a given file in the store) is
unrelated to whether or not that store path is live.  It looks like, in
nix/libstore/gc.cc, we basically walk the references graph starting from
the GC roots and delete anything that can't be reached from a root.
This makes sense, since anything that can be reached from a root is live
and must not be deleted, and anything that can't be reached from any
root can be deleted.  Link counts (e.g., the number of hard links) do
not seem to play a role here.

In addition to the C++ code, the Nix thesis has a good discussion of the
garbage collection algorithm.  I don't know if it's accurate, but it
helps shed light on the intent.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Guix to participate in the Google Summer of Code

[Ricardo Wurmus]

Chris Marusich <cmmarusich@gmail.com> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> Garbage collection in the context of Guix is not very difficult to
>> understand.  The garbage collector may remove any item that has zero
>> users; the link count on a store item is indicative of the number of
>> users that item has.  For garbage collection you essentially go through
>> all items and check if the link count is zero.

[…]

> It looks like, in nix/libstore/gc.cc, we basically walk the references
> graph starting from the GC roots and delete anything that can't be
> reached from a root.

Yes, this is more accurate.  Thanks for the correction.

(Link counts only matter for deduplication.)

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Guix to participate in the Google Summer of Code

[Ricardo Wurmus]

Hi Alex,

>> The reference scanner is already implemented in the form of grafts.  The
>> code opens each file and searches efficiently for store references.
>>
> Does this mean we have procedure to do a single find-reference and the
> remaining work is to build a recursive find-reference?

The current way to obtain references is through the daemon.  In (guix
store) there is a procedure “references” that is used whenever we want
to get a list of references for a store item.  But how does the daemon
itself obtain those references in the first place?

“nix/libstore/references.cc” implements a reference scanner and it is
quite similar to how “replace-store-references” in (guix build graft)
works.

So a first step towards an implementation of the reference scanner would
be to liberate “replace-store-references” from its grafting context;
possibly turn it into a higher order function that finds references and
optionally invokes a function when a reference is found.  Then it could
be used for grafting as well as for collecting references in the daemon.

But I would recommend to revive the work from the last GSoC first.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net