gnunet-guile reboot & guix (take two)

gnunet-guile reboot & guix (take two)

[amirouche]

Hello all,


After discussing gnunet & guix at fosdem with gnunet
people I have better picture of where things can go.

The short story is:

1) There is no way to know the gnunet hash aka. gnunet uri
   of a substitute before the build.

2) There is no way to associate gnunet hash and guix hash
   in a secure/trusted manner over gnunet. Except maybe
   if we use GNS to publish guix hash as subdomains of
   substitute-server.guix.gnu?

Possible solutions:

a) Add the gnunet-uri of the substitute in the package
   definition. This can only work if the package is
   reproducible aka. the build is always the same given
   the same package definition. For reproducible builds,
   it will be possible to offload the build and
   the download over gnunet.

b) Use a central repository (!) which must be trusted and
   which will provide a map of guix hash <-> gnunet hash
   based on builds done locally. This way we can offload
   the download of the files to gnunet...
   That said, the central repository is still a SPOF.

Solution b) is not a massive improvement over the current
situation, that said maybe that is good enough. It's the
easy solution. We must:

 i) change the substitute server to publish over gnunet
    new builds and add the gnunet hash to a local
    database.

 ii) change the substitute server to publish
     guix hash <-> gnunet hash association file

 iii) change guix, to fetch the association file from
      a trusted server and then download over gnunet
      the files.

Solution a) is my prefered because it's truly peer-to-peer
but it leads to complicated workflow for builds that are
not reproducible since we must reset the gnunet uri in
the package definition from a trusted build server.
I am not sure how it's possible to rewrite a package
definition in guile right now.

WDYT?

Re: gnunet-guile reboot & guix (take two)

[Adam Van Ymeren]

On February 3, 2018 8:10:36 AM EST, amirouche <amirouche@hypermove.net> wrote:
>Hello all,
>
>
>After discussing gnunet & guix at fosdem with gnunet
>people I have better picture of where things can go.
>
>The short story is:
>
>1) There is no way to know the gnunet hash aka. gnunet uri
>   of a substitute before the build.
>
>2) There is no way to associate gnunet hash and guix hash
>   in a secure/trusted manner over gnunet. Except maybe
>   if we use GNS to publish guix hash as subdomains of
>   substitute-server.guix.gnu?
>
>Possible solutions:
>
>a) Add the gnunet-uri of the substitute in the package
>   definition. This can only work if the package is
>   reproducible aka. the build is always the same given
>   the same package definition. For reproducible builds,
>   it will be possible to offload the build and
>   the download over gnunet.

Since reproducible builds are a goal of Guix, as far as I know, maybe it would make sense to only do solution A, and only do it for packages which are reproducible.

That would provide extra motivation for fixing non reproducible packages, and would also prevent having to do a bunch of work for use cases that are not the final goal.

Re: gnunet-guile reboot & guix (take two)

[Ludovic Courtès]

Hi amirouche,

amirouche <amirouche@hypermove.net> skribis:

> After discussing gnunet & guix at fosdem with gnunet
> people I have better picture of where things can go.
>
> The short story is:
>
> 1) There is no way to know the gnunet hash aka. gnunet uri
>   of a substitute before the build.
>
> 2) There is no way to associate gnunet hash and guix hash
>   in a secure/trusted manner over gnunet. Except maybe
>   if we use GNS to publish guix hash as subdomains of
>   substitute-server.guix.gnu?

Before jumping to conclusions, did you check the narinfo format that we
use on HTTP?  A narinfo is essentially a statement that a given store
item, resulting from a .drv build, has a specific content hash.

Lookup by content hash over GNUnet’s AFS is all we’d need.

See also the discussion in
<https://www.gnu.org/software/guix/blog/2017/reproducible-builds-a-status-update/>.

HTH!

Ludo’.

_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@gnu.org
https://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: gnunet-guile reboot & guix (take two)

[Ricardo Wurmus]

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi amirouche,
>
> amirouche <amirouche@hypermove.net> skribis:
>
>> After discussing gnunet & guix at fosdem with gnunet
>> people I have better picture of where things can go.
>>
>> The short story is:
>>
>> 1) There is no way to know the gnunet hash aka. gnunet uri
>>   of a substitute before the build.
>>
>> 2) There is no way to associate gnunet hash and guix hash
>>   in a secure/trusted manner over gnunet. Except maybe
>>   if we use GNS to publish guix hash as subdomains of
>>   substitute-server.guix.gnu?
>
> Before jumping to conclusions, did you check the narinfo format that we
> use on HTTP?  A narinfo is essentially a statement that a given store
> item, resulting from a .drv build, has a specific content hash.

This may have been due to my unclear communication at the Guix Meeting
on Friday.  What I said there is that we cannot possibly know the hash
of the output *ahead of time*, which is why it cannot be recorded as
part of the package definition.

The file format of the narinfo files accompanying substitutes, however,
(as provided by files served through “guix publish”) *does* contain the
hash of the contents.

> Lookup by content hash over GNUnet’s AFS is all we’d need.

Indeed.

For items that are *not* reproducible (or that are marked as not
substitutable) a different mechanism would be needed, but it is much
less important to have it.  For those items a keyword lookup (where the
keyword is the name of the derivation or the name of the item’s path in
the store) with filtering by GNS trust would be possible; but
considering that this case is less important and harder to implement I
don’t think we need to bother with it.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net



_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@gnu.org
https://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: [GNUnet-developers] gnunet-guile reboot & guix (take two)

[amirouche]

Le sam. 3 févr. 2018 à 14:10, amirouche <amirouche@hypermove.net> a 
écrit :
> Hello all,
> 
> Possible solutions:
> 
> a) Add the gnunet-uri of the substitute in the package
>   definition. This can only work if the package is
>   reproducible aka. the build is always the same given
>   the same package definition. For reproducible builds,
>   it will be possible to offload the build and
>   the download over gnunet.


I am not sure I will have time to invest in this project right now.

So I created an entry in Guix 2018 GSoC page @ 
https://libreplanet.org/wiki/Group:Guix/GSoC-2018#GNUnet_integration

Feel free to edit / pick the task.