Hello Ludovic,

Some of my 2-cents, wish-list, and crazy ideas.  Unfortunately no
code at this time :(.  Of course, correct me if some of these things
are already solved/not an actual issue.

----- Original Message -----
> Hello,
> 
> I’m trying to gather a “wish list” of things to be done to facilitate
> the use of Guix on clusters and for high-performance computing (HPC).
> 
> Ricardo and I wrote about the advantages, shortcomings, and
> perspectives
> before:
> 
>   http://elephly.net/posts/2015-04-17-gnu-guix.html
>   https://hal.inria.fr/hal-01161771/en
> 
> I know that Pjotr, Roel, Ben, Eric and maybe others also have
> experience
> and ideas on what should be done (and maybe even code? :-)).
> 
> So I’ve come up with an initial list of work items going from the
> immediate needs to crazy ideas (batch scheduler integration!) that
> hopefully make sense to cluster/HPC people.  I’d be happy to get
> feedback, suggestions, etc. from whoever is interested!
> 
> (The reason I’m asking is that I’m considering submitting a proposal
> at
> Inria to work on some of these things.)
> 
> TIA!  :-)
> 
> Ludo’.
> 
> 
>   - non-root usage

The Singularity project advertises that it does not use a root-owned
daemon http://singularity.lbl.gov/about#no-root-owned-daemon-processes
but it does not in the same section explain that it uses a setuid
helper instead: http://singularity.lbl.gov/docs-security which also 
summarizes some of the current limitations and trade-offs of user namespaces.

>   - central daemon usage (like at MDC, but improved)

For many-user systems, I think we'd need to put in place some controls
to keep users from stepping on each others feet when it comes to interacting
with the deamon.  E.g. One user spends a bunch of time building her
application; before she gets a chance to use it, another user comes along
and runs 'guix gc'.

Can a user run 'nice 10 guix build ...' and have it DTRT?

On existing systems, the root partition may not be as large as Guix might
like and there may not be opporunities to mount a separate partition for the
store.  While it's nice that Guix would give users the change to share
package build results, often disk partitions are weighted in favor of /home
(possibly because of the current widespread practice of users building their
own packages in $HOME).  Until that changes, sysadmins might like some more
powerful tools for auditing store disk usage to answer questions such as
"which user profiles are exclusively occupying the most store space?" or even
some way to specify expiration dates for certain profiles.

>     + admin/social issues
>       * daemon runs as root

So, Singularity uses a setuid helper, and Shifter needs to run the Docker
daemon.  It may be easier to convince sysadmins to run Guix's daemon
given those other examples.  Of course, if we can do what we need to
with even fewer privileges, that'd be great.

>       * daemon needs Internet access

There are many HPC sites that are air-gapped for security reasons.  Of those
sites that I know, the ones that allow any outside software to be put on the
machine after the initial system installation require CD/DVD installation media.
IMO for such sites, and for other users wishing to take Guix off the grid, it
would be nice to be able to prepopulate the installation media, whether USB or
CD, with more package outputs and/or source (e.g. like Trisquel's "Sources DVD").
Or similarly a way to "mount" some media that contains a bunch of package
definitions for GUIX_PACKAGE_PATH as well as the corresponding source or output
for a specific Guix release.

>   - package variants, experimentation
>     + for experiments, as in Section 4.2 of
>     [[https://hal.inria.fr/hal-01161771/en][the RepPar paper]]
>       * in the meantime we added
>       [[https://www.gnu.org/software/guix/manual/html_node/Package-Transformation-Options.html][--with-input
>       et al.]]; need more?
>     + for
>     [[https://lists.gnu.org/archive/html/guix-devel/2016-10/msg00005.html][CPU-specific
>     optimizations]]
>     + somehow support -mtune=native (and even profile-guided
>       optimizations?)
>     + simplify the API to switch compilers, libcs, etc.

+1 for all these

Even though we intend to not specifically support proprietary compilers, some
users may still want to explore building their packages with other compilers,
like e.g. Clang and Rose

>   - workflow, reproducible science
>     + implement
>     [[http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22629][channels]]

Perhaps what I discussed above re installation media could fold into this.

`~Eric