GSoC proposal last-minute double-check

GSoC proposal last-minute double-check

[Caleb Ristvedt]

[-- Attachment #1: Type: text/plain, Size: 7553 bytes --]

I realize this is very last-minute indeed, but thought I'd see if I could
get some feedback before I submit. I'm looking at working on the Guile
Scheme guix-daemon - it seems much more concrete in scope than the build
tool. I wrote my proposal based on
https://www.gnu.org/software/soc-projects/guidelines.html, but I think that
might be a bit outdated - it's still got links to 2013 stuff and talks
about mid-semester evaluations even though the schedule currently is to
divide the summer in to 3 work periods. I'm least confident about the
timeline of my proposal - is the time for those goals reasonable, evenly
spaced, etc?

Here's the full proposal as I have it now:

* Name
  Caleb Ristvedt (irc nickname: reepca)

* email address
  caleb.ristvedt@cune.org

* name of project
  Guix Rewrite build daemon in Guile Scheme

* Summary
  Currently the build daemon of Guix is written in C++, inherited from Nix.
It
  works fine but is not as hackable as we'd like, and has poor integration
with
  the rest of Guix. For instance, the daemon calls out to the 'guix
substitute',
  'guix authenticate', and 'guix offload' commands, but its interface to
these
  commands is very limited. Furthermore, a large part of the daemon's code
is
  already implemented in Scheme: container functionality is available with
  'call-with-container', archive creation is implemented in (guix nar),
writing
  derivation files (.drv) is done in (guix derivations), and so on.

  The goal of this project would be to rewrite the daemon in Guile Scheme
using
  the building blocks already available. Important missing bits include the
  garbage collector and its scanner (which scans files for references to
  /gnu/store items), the scheduler, which schedules derivation builds in
  topological order, using the specified number of cores, etc.

* Benefits
  This would benefit users indirectly by benefiting developers. The easier
it is
  to understand and modify the code, the better the quality of the resulting
  software (I leave this as an axiom). Better-quality software benefits
  users. So this project will benefit users.
** Better hackability
   By "hackability", I mean ease of modifying. Where more than one language
   needs to be known to make changes, the code is more difficult to
   change. Where one of those languages is C++, it's even more difficult to
   change. Where multiple languages need to talk to each other, changing the
   interface is more difficult than if the interface involves only one
   language.
** Reduce duplication of code
   Duplicated code is more code than if there is not duplication. It's
easier to
   understand less code. If we can replace the C++ implementation with one
in
   Guile Scheme, there won't be duplication (we can get rid of the C++
   implementation). So replacing the C++ implementation with one in Guile
Scheme
   will make the code easier to understand.

* Deliverables
  A program which, when run, will perform the same function as the current
  guix-daemon, and is written by me solely in Guile Scheme. Additionally,
  documentation explaining how the garbage collector and scheduler work and
  analyzing the time and space requirements of the algorithms used as well
as
  profiling the real-world performance of the daemon.

* Plan
  (I can't find any mention of mid-term evaluations in the 2017 GSoC
timeline,
  so I'm guessing that
https://www.gnu.org/software/soc-projects/guidelines.html
  is outdated?)

  The missing pieces according to the suggestion description include the
garbage
  collector, scanner, and scheduler. Presumably those are either the only
  missing pieces or the only non-trivial missing pieces.

  I will read through the current C++ code implementing garbage collection
and
  scanning during the "Community Bonding Period" to learn how it works.

  The first step should be to implement the missing pieces. I will be on
  schedule if I have the garbage collector and scanner working ("working"
here
  means that given sufficient time, all packages lacking references will be
  deleted or otherwise noticed by a program I wrote in Guile Scheme) by the
end
  of the first work period, and if the scheduler is working by the end of
the
  second work period ("working" here means that a program I wrote in Guile
  Scheme can order derivations topologically and schedule their building
with
  the same characteristics as the current guix-daemon can). By the end of
the
  third work period I will be on schedule if a complete executable,
  functioning in the same way as the current guix-daemon and written by me
in
  Guile Scheme, exists.

  The results from the first two work periods should be usable by
themselves in
  the event that I am unable to complete the entire project.

** Other commitments
   During the summer I will be unable to work from 1300 to 1800 UTC on
Sundays
   and 0030 to 0400 UTC on Mondays (Sunday morning and evening for me), and
from
   0030 to 0400 UTC on Fridays (Thursday evening for me). I currently don't
   anticipate any extended time of unavailability, but should it arise it
will
   either be an emergency or communicated several weeks in advance.

* Communication
  I will communicate using email (individually and/or via the guix-devel
mailing
  list, as seems appropriate) and irc in #guix on freenode. My progress can
be
  monitored via a public github repository to be set up at a later date. I
will
  seek my mentor's input regarding the quality and content of my code at
least
  once a week.

* Qualification
  This project appealed to me for a number of reasons:
  - It is a GNU project
  - It uses a Lisp, a family of languages I appreciate
  - It is clearly-defined (you can't get much clearer than "there is already
    working code that does what you are trying to do")
  - I have some experience with Guix and GuixSD before, from the user side
(the
    only one silly enough to accidentally cause a kernel panic in Shepherd
by
    using incorrect command-line syntax!). The idea of a .emacs-style
    configuration file for the entire operating system especially appeals
to me
    a lot.

  This project will benefit me by teaching me about garbage collection and
at
  least some of the algorithms used for it. It will also teach me a lot
about
  developing larger-scale programs, and especially about developing programs
  meant to be used by others, which I expect will involve a lot more input
  validation than anything I've written before.

  I'm not /especially/ qualified to work on this. I haven't devoted much
time to
  studying garbage collection, and I haven't looked at the guix-daemon code
  yet. But I've been told that I am an effective independent learner, and
this
  /does/ interest me. I have learned Common Lisp on my own time over the
past
  two years, so Scheme shouldn't be too far off. I don't have C++
experience,
  but do have C and Java experience, and the union of concepts from those
two
  should cover most of the C++ concepts. That just leaves the syntax to
learn.

* Background
  I am a sophomore computer science major at Concordia University,
Nebraska. I
  started learning programming in high school with Java (specifically,
  Processing). I learned some Python when my older brother had to use it in
his
  first year of college, then more last year when I arrived. I don't
remember
  exactly when I learned C, but I did. Over the last two years I learned
Forth
  and Common Lisp. I've been using emacs since right before I started at
  college, and using GNU/Linux for about two years before that.

[-- Attachment #2: Type: text/html, Size: 8472 bytes --]

Re: GSoC proposal last-minute double-check

[Ludovic Courtès]

Hi Caleb,

Thank you for your proposal.

Caleb Ristvedt <caleb.ristvedt@cune.org> skribis:

> * Plan
> (I can't find any mention of mid-term evaluations in the 2017 GSoC timeline,
> so I'm guessing that https://www.gnu.org/software/soc-projects/guidelines.html
> is outdated?)
>
> The missing pieces according to the suggestion description include the garbage
> collector, scanner, and scheduler. Presumably those are either the only
> missing pieces or the only non-trivial missing pieces. 
>
> I will read through the current C++ code implementing garbage collection and
> scanning during the "Community Bonding Period" to learn how it works. 
>
> The first step should be to implement the missing pieces. I will be on
> schedule if I have the garbage collector and scanner working ("working" here
> means that given sufficient time, all packages lacking references will be
> deleted or otherwise noticed by a program I wrote in Guile Scheme) by the end
> of the first work period, and if the scheduler is working by the end of the
> second work period ("working" here means that a program I wrote in Guile
> Scheme can order derivations topologically and schedule their building with
> the same characteristics as the current guix-daemon can). By the end of the
> third work period I will be on schedule if a complete executable,
> functioning in the same way as the current guix-daemon and written by me in
> Guile Scheme, exists.
>
> The results from the first two work periods should be usable by themselves in
> the event that I am unable to complete the entire project.

Cool!  I think you should first start playing with Guix and get a good
understanding of what derivations are, what happens when a client asks
the daemon to build them, and what happens when one runs ‘guix gc’; as a
first approach you don’t need into the details of how garbage-collection
works.

Only once you have experience with this and a good understanding does it
make sense to go look at the C++ code IMO.  Of course people on the
mailing lists and on IRC will be happy to help you find your way in the
code.  The manual also contains introductory material about these
concepts (see “The Store”, “Derivations”, “Setting Up the Daemon”, and
“Invoking guix gc”.)  You can also take a look at
<https://arxiv.org/abs/1305.4584>, which gives an overview.

> ** Other commitments
> During the summer I will be unable to work from 1300 to 1800 UTC on Sundays
> and 0030 to 0400 UTC on Mondays (Sunday morning and evening for me), and from
> 0030 to 0400 UTC on Fridays (Thursday evening for me). I currently don't
> anticipate any extended time of unavailability, but should it arise it will
> either be an emergency or communicated several weeks in advance.

As long as you don’t have other commitments (studies, etc.), that should
be fine!

> * Communication
> I will communicate using email (individually and/or via the guix-devel mailing
> list, as seems appropriate) and irc in #guix on freenode. My progress can be
> monitored via a public github repository to be set up at a later date. I will
> seek my mentor's input regarding the quality and content of my code at least
> once a week.

Awesome.  Communication is really important for a project like this, so
we’d expect you to make sure we hear from you, both about the good
things and things that don’t work well.  Sometimes just coming on IRC
and saying “how does this work?” can save you a lot of time.

> * Qualification
> This project appealed to me for a number of reasons:
> - It is a GNU project
> - It uses a Lisp, a family of languages I appreciate
> - It is clearly-defined (you can't get much clearer than "there is already
> working code that does what you are trying to do")
> - I have some experience with Guix and GuixSD before, from the user side (the
> only one silly enough to accidentally cause a kernel panic in Shepherd by
> using incorrect command-line syntax!). The idea of a .emacs-style
> configuration file for the entire operating system especially appeals to me
> a lot.

If everything goes well you’d be fluent in Scheme by the end of the
GSoC.  :-)

I’d expect you to try out the Guix both from the command line and via
its Scheme interface so you get a feel of how it works.  The (guix
store) module in particular contains the client remote procedure call
(RPC) stubs that talk to the daemon; the corresponding side is
implemented in nix-daemon.cc.

Make sure to take advantage of the “Community Bonding Period” to try
things out and to ask any questions you may have!

Thanks,
Ludo’.