Guix - GNUnet binary distribution roadmap

Guix - GNUnet binary distribution roadmap

[Deepan Sekar]

[-- Attachment #1.1: Type: text/plain, Size: 1531 bytes --]

Hi

This is the implementation strategy(work that would be involved) I've come
up with. If it is ok, then I'll proceed to develop the timeline for the
work. I need your critics to improve it.

1. Users will search in the Guix store for the package. This will cause a
Guix daemon to run which will return the installed version and the date of
installation. This has to be worked upon.

2. Then, the user runs the command "guix get filename version_no." (new
command which has to be implemented). This generates the hash with the
filname and the version number as the values.

3. This will in turn execute the "gnunet-dht-get" which will probe GNUnet
DHT for the possible key/value tuples. The values will be the GNUnet peer
which contains the file requested and the version available in the peer. A
new implementation should be written which verifies if the version in the
peer is greater than the existing version. This is all to be done by the
same "gnunet-dht-get" command.

5. If so, then a MESH channel should be opened using the "gnu-mesh-connect
gnunet_peer_id filename version". This does opening the connection,
accessing the file and downloading the complete archive so that it
simplifies the job for the user.

6. Once, the file has been downloaded, the implementation should verify the
signature in the file, and call the existing guix commands to install the
pacakage. All these features are already available. Hence, no work need to
be done on this.

7. Finally test this entire process on a testbed.

Regards,
Deepan

[-- Attachment #1.2: Type: text/html, Size: 1828 bytes --]

[-- Attachment #2: Type: text/plain, Size: 162 bytes --]

_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@gnu.org
https://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: Guix - GNUnet binary distribution roadmap

[Ludovic Courtès]

Hi, Deepan,

Deepan Sekar <dee93.jr@gmail.com> skribis:

> This is the implementation strategy(work that would be involved) I've come
> up with. If it is ok, then I'll proceed to develop the timeline for the
> work. I need your critics to improve it.

Thanks for your interest!

Note that another student, Pierre-Antoine Rault, is also interested in
this project for GSoC.  Please read the discussions we’ve had if you
haven’t already.

> 1. Users will search in the Guix store for the package. This will cause a
> Guix daemon to run which will return the installed version and the date of
> installation. This has to be worked upon.
>
> 2. Then, the user runs the command "guix get filename version_no." (new
> command which has to be implemented). This generates the hash with the
> filname and the version number as the values.

This is not the way things work in Guix.  Please read
<http://www.gnu.org/software/guix/manual/guix.html#Introduction> and
Section 2 of <http://arxiv.org/abs/1305.4584> for more info about the
hash that’s embedded in store file names.

> 3. This will in turn execute the "gnunet-dht-get" which will probe GNUnet
> DHT for the possible key/value tuples. The values will be the GNUnet peer
> which contains the file requested and the version available in the peer. A
> new implementation should be written which verifies if the version in the
> peer is greater than the existing version. This is all to be done by the
> same "gnunet-dht-get" command.
>
> 5. If so, then a MESH channel should be opened using the "gnu-mesh-connect
> gnunet_peer_id filename version". This does opening the connection,
> accessing the file and downloading the complete archive so that it
> simplifies the job for the user.
>
> 6. Once, the file has been downloaded, the implementation should verify the
> signature in the file, and call the existing guix commands to install the
> pacakage. All these features are already available. Hence, no work need to
> be done on this.

We have been refining this in the discussions with Pierre-Antoine.
Could you please start from where we left it?  I would like the proposal
to be closer to the implementation in order to avoid bad surprises
later.  In particular, see
<https://lists.gnu.org/archive/html/guix-devel/2014-03/msg00189.html>.

Thanks,
Ludo’.

_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@gnu.org
https://lists.gnu.org/mailman/listinfo/gnunet-developers

Guix - GNUnet binary distribution roadmap

[Deepan Sekar]

[-- Attachment #1: Type: text/plain, Size: 3113 bytes --]

Hi

Im sorry for coming up with an update in the proposal for the binary
distribution for GSoC 14 this late. I had my university exams. I hope the
shortlisting is not already over and we can still continue the discussions.

Ludo, I tried to edit the proposal in the melange page like you said but
its not possible. It seems the organization has to enable that option.

So, here's the thing:

1. The "guix package -i .." command currently looks up the package
definition and installs the package with the substituter downloading the
binaries from hydra server. If thats not possible, the derivations are
built on the local system.

modifications:
a. Change the package definitions field "url-fetch" of all the packages to
reference that the download is to be made using gnunet. This will make the
daemon run the necessary steps discussed below. The users can update to
this model after its released by using "guix pull".

b. The package definition can be retained and we can change the source code
of the substituter to by default lookup for the binaries in the GNUnet
service and if its not available, then go for the hydra server.

2. Now, whichever method above is implemented, the subsituter can be
reprogrammed to open a gnunet dht connection and get the necessary details.
The substituter finds out the dependencies of the given package from the
package definition and computes the SHA256 hash of the depencies.

3. Since the DHT uses key/value pairs to get and put data, we can use the
SHA256 hash of the dependencies of the package as the key. The name of the
package (since there is a possibility that two packages can have the same
dependencies), gnunet peer id of the user having the package and the mesh
channel identifier as the values associated with that particular key.

4. Once the values from the DHT nodes are received, the substituter filters
out the tuples with a different filename which might be present due to the
match with the hash of the dependencies used as key.

5. From the resulting entries of the corresponding values, the first peer
in the list is used to open a p2p connection in gnunet.

6. The filename and the hash are used to request for the necessary binary
from the peer. Or maybe only the hash can be used. The filename alone is
not sufficient because various versions might be present in the peer's
system. The hash on the other hand is unique.

Note: The gnunet features in the previous steps are available presently.
Only the substituter code in guix has to be written.

6. The downloaded binary is then used by guix to install the package as it
currently does.

Note: For authentication purposes, we can use signatures of the peers along
with the files.

7. Finally, with the users consent, the guix daemon can publish in the
gnunet dht pool of nodes to "put" the key/values of the package so that
other users can now use this peer as a source for the binary file later.


This is the idea for the implementation. The timeline can easily be formed
once the implementation idea is approved. The above can be divided into
phases and assigned a deadline.

Regards,
Deepan

[-- Attachment #2: Type: text/html, Size: 3664 bytes --]

Guix - GNUnet binary distribution roadmap

[Deepan Sekar]

[-- Attachment #1: Type: text/plain, Size: 3113 bytes --]

Hi

Im sorry for coming up with an update in the proposal for the binary
distribution for GSoC 14 this late. I had my university exams. I hope the
shortlisting is not already over and we can still continue the discussions.

Ludo, I tried to edit the proposal in the melange page like you said but
its not possible. It seems the organization has to enable that option.

So, here's the thing:

1. The "guix package -i .." command currently looks up the package
definition and installs the package with the substituter downloading the
binaries from hydra server. If thats not possible, the derivations are
built on the local system.

modifications:
a. Change the package definitions field "url-fetch" of all the packages to
reference that the download is to be made using gnunet. This will make the
daemon run the necessary steps discussed below. The users can update to
this model after its released by using "guix pull".

b. The package definition can be retained and we can change the source code
of the substituter to by default lookup for the binaries in the GNUnet
service and if its not available, then go for the hydra server.

2. Now, whichever method above is implemented, the subsituter can be
reprogrammed to open a gnunet dht connection and get the necessary details.
The substituter finds out the dependencies of the given package from the
package definition and computes the SHA256 hash of the depencies.

3. Since the DHT uses key/value pairs to get and put data, we can use the
SHA256 hash of the dependencies of the package as the key. The name of the
package (since there is a possibility that two packages can have the same
dependencies), gnunet peer id of the user having the package and the mesh
channel identifier as the values associated with that particular key.

4. Once the values from the DHT nodes are received, the substituter filters
out the tuples with a different filename which might be present due to the
match with the hash of the dependencies used as key.

5. From the resulting entries of the corresponding values, the first peer
in the list is used to open a p2p connection in gnunet.

6. The filename and the hash are used to request for the necessary binary
from the peer. Or maybe only the hash can be used. The filename alone is
not sufficient because various versions might be present in the peer's
system. The hash on the other hand is unique.

Note: The gnunet features in the previous steps are available presently.
Only the substituter code in guix has to be written.

6. The downloaded binary is then used by guix to install the package as it
currently does.

Note: For authentication purposes, we can use signatures of the peers along
with the files.

7. Finally, with the users consent, the guix daemon can publish in the
gnunet dht pool of nodes to "put" the key/values of the package so that
other users can now use this peer as a source for the binary file later.


This is the idea for the implementation. The timeline can easily be formed
once the implementation idea is approved. The above can be divided into
phases and assigned a deadline.

Regards,
Deepan

[-- Attachment #2: Type: text/html, Size: 3664 bytes --]