* gnunet-guile reboot & guix
@ 2018-01-12 21:23 Amirouche Boubekki
2018-01-13 0:49 ` ng0
2018-01-13 21:31 ` Ludovic Courtès
0 siblings, 2 replies; 6+ messages in thread
From: Amirouche Boubekki @ 2018-01-12 21:23 UTC (permalink / raw)
To: Gnunet Developers, Guix Devel
Héllo all,
I restarted from scratch the gnunet-guile bindings. It was made
much easier thanks to the work of ng0 on gnunet documentation and
guile-bytestructures to handle C structs and unions.
You need guix from today with latest guile-bytestructures 1.0.1.
You can get the code using the following command:
git clone git://gnunet.org/gnunet-guile2.git
You can install a recent-ish gnunet using the following command:
guix package -f guix.scm
Then, you can do usual cli dance:
./bootstrap && ./configure && make
Ahem, now it's time to run gnunet services. I put together
gnunet configuration that might not be very good even if it
works. For instance, it seems like gnunet manages to reach
the outside world. It's based on configuration files found
in gnunet distribution:
gnunet-arm -c etc/p2.conf -s
At this point you will be able to test the bindings.
To publish a FILE, use the following command:
./pre-inst-env ./gnunet-guile publish etc/p2.conf FILE
To download the above file into OUT, you need to copy paste
the gnunet:// URI from the previous command output and execute
the following command:
./pre-inst-env ./gnunet-guile download etc/p2.conf URI OUT
That is all!
There is no support for identity and various stuff are missing.
There might be memory leaks and other issues (like proper disjoint
types for pointers). I just finished the code.
I think I need to know what's the plan/design for gnunet/guix
integration to continue.
TIA!
_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@gnu.org
https://lists.gnu.org/mailman/listinfo/gnunet-developers
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: gnunet-guile reboot & guix 2018-01-12 21:23 gnunet-guile reboot & guix Amirouche Boubekki @ 2018-01-13 0:49 ` ng0 2018-01-13 9:26 ` Amirouche Boubekki 2018-01-13 21:31 ` Ludovic Courtès 1 sibling, 1 reply; 6+ messages in thread From: ng0 @ 2018-01-13 0:49 UTC (permalink / raw) To: Amirouche Boubekki; +Cc: Guix Devel, Gnunet Developers [-- Attachment #1.1: Type: text/plain, Size: 3367 bytes --] Amirouche Boubekki transcribed 1.5K bytes: > Héllo all, > > I restarted from scratch the gnunet-guile bindings. It was made > much easier thanks to the work of ng0 on gnunet documentation and > guile-bytestructures to handle C structs and unions. > > You need guix from today with latest guile-bytestructures 1.0.1. > > You can get the code using the following command: > > git clone git://gnunet.org/gnunet-guile2.git > > You can install a recent-ish gnunet using the following command: > > guix package -f guix.scm > > Then, you can do usual cli dance: > > ./bootstrap && ./configure && make > > Ahem, now it's time to run gnunet services. I put together Would gnunet-service (in GuixSD, and shepherd as user-process on other systems) help here? I wanted to start debugging my service again in february, maybe earlier. I could delegate it to whoever wants to work on it though. > gnunet configuration that might not be very good even if it > works. For instance, it seems like gnunet manages to reach > the outside world. It's based on configuration files found > in gnunet distribution: > > gnunet-arm -c etc/p2.conf -s > > At this point you will be able to test the bindings. > > To publish a FILE, use the following command: > > ./pre-inst-env ./gnunet-guile publish etc/p2.conf FILE > > To download the above file into OUT, you need to copy paste > the gnunet:// URI from the previous command output and execute > the following command: > > ./pre-inst-env ./gnunet-guile download etc/p2.conf URI OUT > > That is all! Hey! Pretty good progress for just a couple of days, very nice! > There is no support for identity and various stuff are missing. > There might be memory leaks and other issues (like proper disjoint > types for pointers). I just finished the code. > > I think I need to know what's the plan/design for gnunet/guix > integration to continue. If you want a (relative) unprocessed summary of its history, it's collected in here (org-mode recommended): https://gnunet.org/git/infotropique-artwork.git/tree/doc/guix-past-notes.txt Now for the current goals I should manage to reply within a week (searching my notes and past offlist emails). That is of course from my perspective as a GNUnet developer. On various occasions it was made clear to me that FS isn't ready for such usage and we would need to extend it if we start working on it. I could be wrong, but to some degree our implementation of our own designs has some mistakes, if I remember Grothoff right. Bits and pieces without what I came up with in winter 2016: * anonymous levels aren't necessary for sharing code ** update on this: one (or more?) other OS' is looking into Guix+GNUnet as well and I'm not sure if they would rely on anonymity. Imo it makes no sense for the start. anonymity setting 0 would work for a start. Before I share my part of the ideas, maybe someone who wasn't involved in the on-and-off discussions could add their ideas? Fresh ideas could bring new perspectives we haven't seen. For the Guix part I just know that it should be an option, not a default. > > TIA! > > -- GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588 GnuPG: https://c.n0.is/ng0_pubkeys/tree/keys WWW: https://n0.is/a/ :: https://ea.n0.is [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] [-- Attachment #2: Type: text/plain, Size: 162 bytes --] _______________________________________________ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: gnunet-guile reboot & guix 2018-01-13 0:49 ` ng0 @ 2018-01-13 9:26 ` Amirouche Boubekki 2018-01-13 11:14 ` ng0 0 siblings, 1 reply; 6+ messages in thread From: Amirouche Boubekki @ 2018-01-13 9:26 UTC (permalink / raw) To: Amirouche Boubekki, Gnunet Developers, Guix Devel On 2018-01-13 01:49, ng0 wrote: > Amirouche Boubekki transcribed 1.5K bytes: >> Héllo all, >> >> I restarted from scratch the gnunet-guile bindings. It was made >> much easier thanks to the work of ng0 on gnunet documentation and >> guile-bytestructures to handle C structs and unions. ... >> I think I need to know what's the plan/design for gnunet/guix >> integration to continue. > > If you want a (relative) unprocessed summary of its history, it's > collected in here > (org-mode recommended): > https://gnunet.org/git/infotropique-artwork.git/tree/doc/guix-past-notes.txt Here is an extract relevant to the current discussion: * Design of guix/gnunet integration >> So I can see two milestones now, as we discussed before: >> 1. Create a variant of ‘guix publish’ that publishes over GNUnet’s >> file sharing (FS) service, using the neat bindings that you wrote. >> For that you can literally copy guix/scripts/publish.scm as a >> starting point, and simply remove the HTTP-related code (which is a >> small fraction.) The rest will be mostly identical: narinfo >> meta-data generation, signing. I guess it will be easy to test it >> using gnunet-search and similar tools. >> As a 2nd step, we’ll see how we can refactor things to allow code >> sharing between the existing ‘guix publish’ and its GNUnet variant. >> 2. Create a variant of ‘guix substitute’ for searches through GNUnet. >> Again, a large part of the code is about narinfo and signature >> checking, and it can be reused. >> I’m not completely clear on how search for substitutes will work, >> though. Currently, when the user wants to build /gnu/store/xyz, ‘guix >> substitute’ simply fetches http://hydra.gnu.org/xyz.narinfo. How will >> that work with GNUnet? Are we going to look up their /gnu/store file >> name? >> I’m not completely clear on how search for substitutes will work, >> though. Currently, when the user wants to build /gnu/store/xyz, ‘guix >> substitute’ simply fetches http://hydra.gnu.org/xyz.narinfo. How will >> that work with GNUnet? Are we going to look up their /gnu/store file >> name? > I’ve considered a solution for that: GNUnet allows one to create > specific namespace and publish files under this namespace. Unlike > publishing under the “global namespace” where keywords are used to > identify a file, when publishing under a specific namespace files are > identified with a choosen identifier. Moreover, as a namespace is > basically a cryptographic key pair, and publishing a file under your > namespace means signing, one’s assured nobody else will publish under > her or his namespace. By the way, the private key associated with a > namespace is named “ego” or “pseudonym”. > It’s easy to test this feature: > # create a `test` ego/namespace > $ gnunet-identity -C test > # list the known egos in the form: `name - public key` > $ gnunet-identity -d > test - M2OC987U9LFJHQ8LC9SLCV4Q0ONHJV7FMTFQ2VRPE0M9R9MK5860 > … > # index the file `foo.txt` under the `test` namespace > $ gnunet-publish -P test -t foobarbaz foo.txt > # find the file `foo.txt` > $ gnunet-search gnunet://fs/sks/M2OC987U9LFJHQ8LC9SLCV4…/foobarbaz > #0: > gnunet-download -o "foo.txt" gnunet://fs/chk/PL217ODD8EDSMOIQ3UT0… > Now if Alice wants to publish her binaries, she creates an > ego/namespace and publishes everything under it; Bob adds her > namespace’s public key to his authorized substitutes list, and when > installing `/gnu/store/xyz` the substitute will search for > `gnunet://fs/sks/<Alice’s key>/xyz`. > Instead of publishing an archive we might also directly publish/index > the build, but I don’t know if it’s viable. > Does it seem right to you? * Another discusion about guix integration >> Instead of using ‘file-system-tree’, this variant should probably use >> ‘live-paths’ from (guix store), which returns the list of live store >> items. > Well, `file-system-tree` is only used to recursively index a random > directory’s content (in our case, a single store item). It looked > viable > for publishing a single store item, but won’t be good for indexing at > once the entire set of live paths; I should ask the GNUnet team how to > properly index such a huge amount of directories. > On my machine, running `live-paths` takes ~2 seconds, but the > publication of the entire store will probably take much longer anyway. >> BTW, I noticed there’s quite a bunch of global variables that are >> ‘set!’. It would be better to avoid that, but I suppose the >> continuation-passing style that the GNUnet libraries impose makes it >> difficult. > Hopefully, using the “closure” parameters of the GNUnet API in the > bindings should reduce the need for global variables, and improve > elegance of end-user programs. >>> Nitpick: it’s a bit annoying that we have to specify a GNUnet >>> configuration file. >> >> Yes, GNUnet programs usually look for `~/.config/gnunet.conf`, and >> `publish-gnunet` does the same. Now, maybe `publish-gnunet` could >> somehow obtain the config file used by `gnunet-arm`? > No, it would need the config file to figure out how to talk to > gnunet-service-arm (or any other service). And we do support running > many instances of peers on the same host, which really means the config > is the only way to find out. The rest of ng0 reply: > > On various occasions it was made clear to me that FS isn't ready for > such > usage and we would need to extend it if we start working on it. I could > be > wrong, but to some degree our implementation of our own designs has > some > mistakes, if I remember Grothoff right. > > Bits and pieces without what I came up with in winter 2016: > > * anonymous levels aren't necessary for sharing code > ** update on this: one (or more?) other OS' is looking into Guix+GNUnet > as well and I'm not sure if they would rely on anonymity. Imo it > makes > no sense for the start. anonymity setting 0 would work for a start. Anonymity is easily configurable. > Before I share my part of the ideas, maybe someone who wasn't involved > in the on-and-off discussions could add their ideas? Fresh ideas could > bring new perspectives we haven't seen. > > > For the Guix part I just know that it should be an option, not a > default. > Oh really? I don't see that appear in the discussion with Remi. _______________________________________________ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: gnunet-guile reboot & guix 2018-01-13 9:26 ` Amirouche Boubekki @ 2018-01-13 11:14 ` ng0 0 siblings, 0 replies; 6+ messages in thread From: ng0 @ 2018-01-13 11:14 UTC (permalink / raw) To: Amirouche Boubekki; +Cc: Guix Devel, Gnunet Developers [-- Attachment #1.1: Type: text/plain, Size: 7778 bytes --] Amirouche Boubekki transcribed 6.3K bytes: > On 2018-01-13 01:49, ng0 wrote: > > Amirouche Boubekki transcribed 1.5K bytes: > > > Héllo all, > > > > > > I restarted from scratch the gnunet-guile bindings. It was made > > > much easier thanks to the work of ng0 on gnunet documentation and > > > guile-bytestructures to handle C structs and unions. > > ... > > > > I think I need to know what's the plan/design for gnunet/guix > > > integration to continue. > > > > If you want a (relative) unprocessed summary of its history, it's > > collected in here > > (org-mode recommended): > > https://gnunet.org/git/infotropique-artwork.git/tree/doc/guix-past-notes.txt > > Here is an extract relevant to the current discussion: > > * Design of guix/gnunet integration > > > > So I can see two milestones now, as we discussed before: > > > > 1. Create a variant of ‘guix publish’ that publishes over GNUnet’s > > > file sharing (FS) service, using the neat bindings that you wrote. > > > > For that you can literally copy guix/scripts/publish.scm as a > > > starting point, and simply remove the HTTP-related code (which is a > > > small fraction.) The rest will be mostly identical: narinfo > > > meta-data generation, signing. I guess it will be easy to test it > > > using gnunet-search and similar tools. > > > > As a 2nd step, we’ll see how we can refactor things to allow code > > > sharing between the existing ‘guix publish’ and its GNUnet variant. > > > > 2. Create a variant of ‘guix substitute’ for searches through GNUnet. > > > Again, a large part of the code is about narinfo and signature > > > checking, and it can be reused. > > > > I’m not completely clear on how search for substitutes will work, > > > though. Currently, when the user wants to build /gnu/store/xyz, ‘guix > > > substitute’ simply fetches http://hydra.gnu.org/xyz.narinfo. How will > > > that work with GNUnet? Are we going to look up their /gnu/store file > > > name? > > > > > I’m not completely clear on how search for substitutes will work, > > > though. Currently, when the user wants to build /gnu/store/xyz, ‘guix > > > substitute’ simply fetches http://hydra.gnu.org/xyz.narinfo. How will > > > that work with GNUnet? Are we going to look up their /gnu/store file > > > name? > > > I’ve considered a solution for that: GNUnet allows one to create > > specific namespace and publish files under this namespace. Unlike > > publishing under the “global namespace” where keywords are used to > > identify a file, when publishing under a specific namespace files are > > identified with a choosen identifier. Moreover, as a namespace is > > basically a cryptographic key pair, and publishing a file under your > > namespace means signing, one’s assured nobody else will publish under > > her or his namespace. By the way, the private key associated with a > > namespace is named “ego” or “pseudonym”. > > > It’s easy to test this feature: > > > # create a `test` ego/namespace > > $ gnunet-identity -C test > > > # list the known egos in the form: `name - public key` > > $ gnunet-identity -d > > test - M2OC987U9LFJHQ8LC9SLCV4Q0ONHJV7FMTFQ2VRPE0M9R9MK5860 > > … > > > # index the file `foo.txt` under the `test` namespace > > $ gnunet-publish -P test -t foobarbaz foo.txt > > > # find the file `foo.txt` > > $ gnunet-search gnunet://fs/sks/M2OC987U9LFJHQ8LC9SLCV4…/foobarbaz > > #0: > > gnunet-download -o "foo.txt" gnunet://fs/chk/PL217ODD8EDSMOIQ3UT0… > > > Now if Alice wants to publish her binaries, she creates an > > ego/namespace and publishes everything under it; Bob adds her > > namespace’s public key to his authorized substitutes list, and when > > installing `/gnu/store/xyz` the substitute will search for > > `gnunet://fs/sks/<Alice’s key>/xyz`. > > > Instead of publishing an archive we might also directly publish/index > > the build, but I don’t know if it’s viable. > > > Does it seem right to you? > > * Another discusion about guix integration > > > > Instead of using ‘file-system-tree’, this variant should probably use > > > ‘live-paths’ from (guix store), which returns the list of live store > > > items. > > > Well, `file-system-tree` is only used to recursively index a random > > directory’s content (in our case, a single store item). It looked viable > > for publishing a single store item, but won’t be good for indexing at > > once the entire set of live paths; I should ask the GNUnet team how to > > properly index such a huge amount of directories. > > > On my machine, running `live-paths` takes ~2 seconds, but the > > publication of the entire store will probably take much longer anyway. > > > > BTW, I noticed there’s quite a bunch of global variables that are > > > ‘set!’. It would be better to avoid that, but I suppose the > > > continuation-passing style that the GNUnet libraries impose makes it > > > difficult. > > > Hopefully, using the “closure” parameters of the GNUnet API in the > > bindings should reduce the need for global variables, and improve > > elegance of end-user programs. > > > > > > Nitpick: it’s a bit annoying that we have to specify a GNUnet > > > > configuration file. > > > > > > Yes, GNUnet programs usually look for `~/.config/gnunet.conf`, and > > > `publish-gnunet` does the same. Now, maybe `publish-gnunet` could > > > somehow obtain the config file used by `gnunet-arm`? > > > No, it would need the config file to figure out how to talk to > > gnunet-service-arm (or any other service). And we do support running > > many instances of peers on the same host, which really means the config > > is the only way to find out. > > The rest of ng0 reply: > > > > > On various occasions it was made clear to me that FS isn't ready for > > such > > usage and we would need to extend it if we start working on it. I could > > be > > wrong, but to some degree our implementation of our own designs has some > > mistakes, if I remember Grothoff right. > > > > Bits and pieces without what I came up with in winter 2016: > > > > * anonymous levels aren't necessary for sharing code > > ** update on this: one (or more?) other OS' is looking into Guix+GNUnet > > as well and I'm not sure if they would rely on anonymity. Imo it > > makes > > no sense for the start. anonymity setting 0 would work for a start. > > Anonymity is easily configurable. Sure. > > Before I share my part of the ideas, maybe someone who wasn't involved > > in the on-and-off discussions could add their ideas? Fresh ideas could > > bring new perspectives we haven't seen. > > > > > > For the Guix part I just know that it should be an option, not a > > default. > > > > Oh really? I don't see that appear in the discussion with Remi. The copied file was a collection of work not only of Remi. There's 2 or 3 threads at least. The idea that GNUnet FS distribution should be an option, not a default for Guix is my understanding of how more recent discussions (on IRC and the mailinglist, possibly even AFK chats I had with some of the other developers). My personal idea is to make it a standard in my redistribution of GuixSD, but primarily we want to work on what Guix wants, not what a distro framework in development close to GNUnet wants, for now. Of course I wouldn't complain if Guix would pick GNUnet as default, but we'd need a good amount of testing before we'd make it so. -- GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588 GnuPG: https://c.n0.is/ng0_pubkeys/tree/keys WWW: https://n0.is/a/ :: https://ea.n0.is [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] [-- Attachment #2: Type: text/plain, Size: 162 bytes --] _______________________________________________ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: gnunet-guile reboot & guix 2018-01-12 21:23 gnunet-guile reboot & guix Amirouche Boubekki 2018-01-13 0:49 ` ng0 @ 2018-01-13 21:31 ` Ludovic Courtès 2018-01-14 0:45 ` ng0 1 sibling, 1 reply; 6+ messages in thread From: Ludovic Courtès @ 2018-01-13 21:31 UTC (permalink / raw) To: Amirouche Boubekki; +Cc: Guix Devel, Gnunet Developers Hello, Amirouche Boubekki <amirouche@hypermove.net> skribis: > I restarted from scratch the gnunet-guile bindings. It was made > much easier thanks to the work of ng0 on gnunet documentation and > guile-bytestructures to handle C structs and unions. > > You need guix from today with latest guile-bytestructures 1.0.1. > > You can get the code using the following command: > > git clone git://gnunet.org/gnunet-guile2.git Thanks a lot for giving it some love! Perhaps we should retire the repo at <https://git.savannah.gnu.org/cgit/guix/gnunet.git/>? (The ‘wip-monad’ branch might be of interest though.) Ludo’. _______________________________________________ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: gnunet-guile reboot & guix 2018-01-13 21:31 ` Ludovic Courtès @ 2018-01-14 0:45 ` ng0 0 siblings, 0 replies; 6+ messages in thread From: ng0 @ 2018-01-14 0:45 UTC (permalink / raw) To: Ludovic Courtès; +Cc: Guix Devel, Gnunet Developers [-- Attachment #1.1: Type: text/plain, Size: 1731 bytes --] Ludovic Courtès transcribed 1.0K bytes: > Hello, > > Amirouche Boubekki <amirouche@hypermove.net> skribis: > > > I restarted from scratch the gnunet-guile bindings. It was made > > much easier thanks to the work of ng0 on gnunet documentation and > > guile-bytestructures to handle C structs and unions. > > > > You need guix from today with latest guile-bytestructures 1.0.1. > > > > You can get the code using the following command: > > > > git clone git://gnunet.org/gnunet-guile2.git > > Thanks a lot for giving it some love! > > Perhaps we should retire the repo at > <https://git.savannah.gnu.org/cgit/guix/gnunet.git/>? > (The ‘wip-monad’ branch might be of interest though.) > > Ludo’. > > _______________________________________________ > GNUnet-developers mailing list > GNUnet-developers@gnu.org > https://lists.gnu.org/mailman/listinfo/gnunet-developers I think both Amirouche as well as myself (and gnunet.org) have a copy of the old code base. If I understand savannah hosting correctly 'retiring' just means making it read-only, so that would be alright. Development happens on gnunet.org and future releases will be placed on the GNU Ftp folder of gnunet. Our mailinglist at gnunet-developers is open for patches (no sign-up required) and whoever wants to help developing it can get push access (of course Amirouche has to decide on this project, etc). Eventually we should remove the old repository https://gnunet.org/git/gnunet-guile, or rename the current one (gnunet-guile2). That is my position. I'm not working on this right now, Amirouche does. -- ng0 :: https://ea.n0.is A88C8ADD129828D7EAC02E52E22F9BBFEE348588 :: https://ea.n0.is/keys/ [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] [-- Attachment #2: Type: text/plain, Size: 162 bytes --] _______________________________________________ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-01-14 0:45 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-01-12 21:23 gnunet-guile reboot & guix Amirouche Boubekki 2018-01-13 0:49 ` ng0 2018-01-13 9:26 ` Amirouche Boubekki 2018-01-13 11:14 ` ng0 2018-01-13 21:31 ` Ludovic Courtès 2018-01-14 0:45 ` ng0
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).