Lightning talk at IPFS camp

Lightning talk at IPFS camp

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 332 bytes --]

Hi,

I'm going to the IPFS camp (https://camp.ipfs.io/) on June 27th and I've
been asked to give a (5 minute) lightning talk about IPFS & Guix.  Yaik! :)

I'll have to prepare some slides (probably in Org reveal?) soon, so now
is the time to pitch in some ideas ;)

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Lightning talk at IPFS camp

[Ludovic Courtès]

Hi Pierre,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> I'm going to the IPFS camp (https://camp.ipfs.io/) on June 27th and I've
> been asked to give a (5 minute) lightning talk about IPFS & Guix.  Yaik! :)

Excellent, I’m sure you’re going to have a great time!

> I'll have to prepare some slides (probably in Org reveal?) soon, so now
> is the time to pitch in some ideas ;)

I guess you could explain why it makes sense for Guix as a project to
turn to decentralized solutions, and roughly how we plan to make it
work?

Thanks,
Ludo’.

Re: Lightning talk at IPFS camp

[Konrad Hinsen]

Hi Pierre,

> I'm going to the IPFS camp (https://camp.ipfs.io/) on June 27th and I've
> been asked to give a (5 minute) lightning talk about IPFS & Guix.  Yaik! :)

That sounds like a great opportunity. Guix and IPFS are in my humble
opinion two of the most interesting ongoing projects in the computing
world.. Bringing the two together can only make this better.

> I'll have to prepare some slides (probably in Org reveal?) soon, so now
> is the time to pitch in some ideas ;)

A common issue: augmenting trust in digital information
  - IPFS: unalterable references to arbitrary data
  - Guix: provenance tracking for software builds

What IPFS can do for Guix:
  - A better way to archive and distribute Guix and the software
    it builds.
  - A unified way to refer to stuff (I am thinking of IPLD here)
    No more tarballs, git commits, etc. CIDs everywhere.
  - A unified storage scheme for all data, both "system" and "user".

What Guix can do for IPFS:
  - Provenance tracking for data that has been processed by software

A vision for a (remote) future:
  - All data lives in IPFS: no local filesystem, no Guix store.
    A personal computing device only stores references to information
    that its owner cares about.
  - All computations are equal: no distinction between "software builds"
    and everything else.
  - All data comes with provenance tracking:
     - computations are tracked via Guix
     - human input is logged (interactivity) or version controlled

Konrad.

Re: Lightning talk at IPFS camp

[Pronaip]

Hi, not to rain on the parade, but

> -   All data lives in IPFS: no local filesystem, no Guix store.
>     A personal computing device only stores references to information
>     that its owner cares about.
>
> -   All computations are equal: no distinction between "software builds"
>     and everything else.
>
> -   All data comes with provenance tracking:
>     -   computations are tracked via Guix
>     -   human input is logged (interactivity) or version controlled

unless you vision of the distant future somehow also includes most social problems having been solved, these would be ripe for abuse by malicious actors.

Not having any local file system is also just plain wasteful.

Your ideas make sense in some environments (eg. tracking what government infrastructure is doing) but for personal computers? Come on....

Re: Lightning talk at IPFS camp

[Konrad Hinsen]

Am 03.06.19 um 18:19 schrieb Pronaip:

>> -   All data comes with provenance tracking:
>>      -   computations are tracked via Guix
>>      -   human input is logged (interactivity) or version controlled
> 
> unless you vision of the distant future somehow also includes most social problems having been solved, these would be ripe for abuse by malicious actors.

I didn't say that provenance information should be made public, even if 
the data itself is. It could well be encrypted. More generally, 
encryption is an essential ingredient for dealing with non-public data 
in IPFS. Which is indeed a weak point considering that you never know 
for how long encryption will be safe.

> Not having any local file system is also just plain wasteful.

Note that I said file system, not storage. There should be local storage 
for efficiency, but it would act like a cache, not as a separately 
managed storage space.

BTW, data management is not really part of IPFS at all, it needs to be 
handled by another layer and so far there is little more than 
experiments. Textile looks promising, for example.

Konrad.

Re: Lightning talk at IPFS camp

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

Thank you all for your comments!

> That sounds like a great opportunity. Guix and IPFS are in my humble
> opinion two of the most interesting ongoing projects in the computing
> world.. Bringing the two together can only make this better.

My humble opinion is on the same page ;)

>	 - A unified way to refer to stuff (I am thinking of IPLD here)
>		 No more tarballs, git commits, etc. CIDs everywhere.

Do you have a concrete use case?

>	 - A unified storage scheme for all data, both "system" and "user".

Can you elaborate?


I'm not too sure about how human input would be logged, but at the very
least the idea of distributing the store seems amazing.

In a not so distant future, we could simply distribute the store of
everyone.
So if Alice built qtwebkit locally (at last!), then Bob will be able to
fetch the substitute.  There would be no difference between substitute
servers and Guix users connected to the Internet.

Anonymity should not really be a problem here, and even then it could be
done over Tor.

Thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Lightning talk at IPFS camp

[Konrad Hinsen]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

>>	 - A unified way to refer to stuff (I am thinking of IPLD here)
>>		 No more tarballs, git commits, etc. CIDs everywhere.
>
> Do you have a concrete use case?

I was thinking of the Guix package definitions. In the long run,
assuming IPFS turns out to be reliable enough, we could put all source
into IPFS with a CID reference, rather then today's many ways to
download source files.

>>	 - A unified storage scheme for all data, both "system" and "user".
>
> Can you elaborate?

Again in the long run, if we don't mind depending on IPFS, we don't need
the Guix store any more. Package installation would amount to local
pinning. Anyone could then build a package anywhere (home directory,
...) and just add it to IPFS. Since that also eliminates the technical
constraints of the store, the same mechanism could be used for any kind
of data processing, with the results stored in IPFS. Reproducibility of
any kind of computation via Guix, with building software just an
important special case.

> I'm not too sure about how human input would be logged, but at the very
> least the idea of distributing the store seems amazing.

For human input, Git would be OK, with repositories stored in IPFS
(there's already some support for that, see
https://github.com/ipfs/go-ipld-git). A more radical redesign is
Radicle (http://www.radicle.xyz/), which uses IPFS as a collaboration
platform (still at the git level). I guess Radicle could be used for
much more than that in Guix, but I haven't looked at that in detail.

Konrad.

Re: Lightning talk at IPFS camp

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1688 bytes --]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> I was thinking of the Guix package definitions. In the long run,
> assuming IPFS turns out to be reliable enough, we could put all source
> into IPFS with a CID reference, rather then today's many ways to
> download source files.

There would be nothing special about it beside implementing an IPFS
fetcher, or would it?  Let me know if I misunderstood.

> Again in the long run, if we don't mind depending on IPFS, we don't need
> the Guix store any more. Package installation would amount to local
> pinning. Anyone could then build a package anywhere (home directory,
> ...) and just add it to IPFS. Since that also eliminates the technical
> constraints of the store, the same mechanism could be used for any kind
> of data processing, with the results stored in IPFS. Reproducibility of
> any kind of computation via Guix, with building software just an
> important special case.

Very good point, I like it.  I think I'll mention this in the talk.

> For human input, Git would be OK, with repositories stored in IPFS
> (there's already some support for that, see
> https://github.com/ipfs/go-ipld-git). A more radical redesign is
> Radicle (http://www.radicle.xyz/), which uses IPFS as a collaboration
> platform (still at the git level). I guess Radicle could be used for
> much more than that in Guix, but I haven't looked at that in detail.

Didn't know Radicle, it looks fantabulous!  And... it uses (or plans to)
a Scheme-based language! :)

So you are saying that we could move the guix.git to a Radicle project,
right?
Makes sense to me.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Lightning talk at IPFS camp

[Konrad Hinsen]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

>> I was thinking of the Guix package definitions. In the long run,
>> assuming IPFS turns out to be reliable enough, we could put all source
>> into IPFS with a CID reference, rather then today's many ways to
>> download source files.
>
> There would be nothing special about it beside implementing an IPFS
> fetcher, or would it?  Let me know if I misunderstood.

What's special about it is that it could replace all the other ones.

> Didn't know Radicle, it looks fantabulous!  And... it uses (or plans to)
> a Scheme-based language! :)

Exactly.

> So you are saying that we could move the guix.git to a Radicle project,
> right?

Yes, assuming all that becomes stable at some point.

Konrad.

Re: Lightning talk at IPFS camp

[Ludovic Courtès]

Hi!

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

[...]

>> For human input, Git would be OK, with repositories stored in IPFS
>> (there's already some support for that, see
>> https://github.com/ipfs/go-ipld-git). A more radical redesign is
>> Radicle (http://www.radicle.xyz/), which uses IPFS as a collaboration
>> platform (still at the git level). I guess Radicle could be used for
>> much more than that in Guix, but I haven't looked at that in detail.
>
> Didn't know Radicle, it looks fantabulous!  And... it uses (or plans to)
> a Scheme-based language! :)

It looks like the “right” design, or at least one that I find super
interesting!

(They could have chosen Guile instead of a custom Lisp, but that’s an
“implementation detail”.  :-))

Ludo’.

Re: Lightning talk at IPFS camp

[Konrad Hinsen]

Ludovic Courtès <ludo@gnu.org> writes:

> (They could have chosen Guile instead of a custom Lisp, but that’s an
> “implementation detail”.  :-))

From my reading of the whitepaper, no. They have pretty strict
constraints on their language because they send live code updates
to running instances and want to be able to make certain guarantees.
Guile, or any other Scheme, would be too flexible.

Konrad.

Re: Lightning talk at IPFS camp

[Ludovic Courtès]

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> (They could have chosen Guile instead of a custom Lisp, but that’s an
>> “implementation detail”.  :-))
>
> From my reading of the whitepaper, no. They have pretty strict
> constraints on their language because they send live code updates
> to running instances and want to be able to make certain guarantees.
> Guile, or any other Scheme, would be too flexible.

Oh right, I had overlooked that aspect of the project.  Interesting!

Ludo’.

Re: Lightning talk at IPFS camp

[Ludovic Courtès]

Hey!

I stumbled upon this:

  https://github.com/ipfs/package-managers#readme

Hopefully that’s something you’ll discuss at IPFS Camp!

Ludo’.

Re: Lightning talk at IPFS camp

[Konrad Hinsen]

Ludovic Courtès <ludo@gnu.org> writes:

> Hey!
>
> I stumbled upon this:
>
>   https://github.com/ipfs/package-managers#readme

And I just stumbled on this one:

   https://github.com/ipfs/roadmap

Quote:

   2019 Priority

   We will be focusing our efforts into a single (lazer focus)
   priority. 📦 Package Managers

This looks like just the right moment to tell them about the one true
package manager ;-)

Konrad.

Re: Lightning talk at IPFS camp

[ng0]

Pierre Neidhardt transcribed 1.7K bytes:
> Thank you all for your comments!
> 
> > That sounds like a great opportunity. Guix and IPFS are in my humble
> > opinion two of the most interesting ongoing projects in the computing
> > world.. Bringing the two together can only make this better.
> 
> My humble opinion is on the same page ;)
> 
> >	 - A unified way to refer to stuff (I am thinking of IPLD here)
> >		 No more tarballs, git commits, etc. CIDs everywhere.
> 
> Do you have a concrete use case?
> 
> >	 - A unified storage scheme for all data, both "system" and "user".
> 
> Can you elaborate?
> 
> 
> I'm not too sure about how human input would be logged, but at the very
> least the idea of distributing the store seems amazing.
> 
> In a not so distant future, we could simply distribute the store of
> everyone.
> So if Alice built qtwebkit locally (at last!), then Bob will be able to
> fetch the substitute.  There would be no difference between substitute
> servers and Guix users connected to the Internet.
> 
> Anonymity should not really be a problem here, and even then it could be
> done over Tor.
> 
> Thoughts?

Skipping all the rest, my brief comment on the tor point:
You don't want to use tor for this.
1) it depends on the country if it is safe at all to use
2) tor has some recent RFCs and/or papers which hint at what we've been
   talking about for years, tor in its current form does not scale for
   applications like this (even when the throughput of the servers in the
   tor network got faster globally).

At best make it an opt-in choice, legal reasons and safety considerations
of your entire user base / community call for (1) as a reason,
but also for (2) because you want to actively work on the relevant RFCs
to not make this an annoyance both for you and tor.
While some operating systems have the official blessing, I seem to recall
(paper notes...) that in no case tor said it is a good decision to sent
package manager traffic through it.

Since you have no covertraffic, the point for anonymity is moot anyways.

What's more, you buy into a couple of more points of authority to trust
(in particular directory server operators, their ISPs, and so forth).

I leave it up to you to gather the sources, I don't want to provide them
right now as it is doable with enough time for yourself to see this is
a bad idea.

> -- 
> Pierre Neidhardt
> https://ambrevar.xyz/

Re: Lightning talk at IPFS camp

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 571 bytes --]

Update: I've been asked if I wanted to conduct a "Deep dive"
(https://github.com/ipfs/camp/blob/master/DEEP_DIVES/README.md) about
IPFS and Guix.  In other words: what problem we've encountered, how to
solve them, what we'd like to do.

I think it would be a good time to discuss a work out bug
33899	"[PATCH 0/5] Distributing substitutes over IPFS" that Ludovic
submitted a while ago.

What do you people think?

Ludovic, what are the most pressing issues with this patch that you'd
like to raise?

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Lightning talk at IPFS camp

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 229 bytes --]

I've uploaded the slides to git.savannah.gnu.org:/srv/git/guix/maintenance.git
in the talks/ipfs-camp-2019 subfolder.

Let me know if you'd like to see some changes.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Lightning talk at IPFS camp

[Ludovic Courtès]

Hello!

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Update: I've been asked if I wanted to conduct a "Deep dive"
> (https://github.com/ipfs/camp/blob/master/DEEP_DIVES/README.md) about
> IPFS and Guix.  In other words: what problem we've encountered, how to
> solve them, what we'd like to do.

Neat.

> I think it would be a good time to discuss a work out bug
> 33899	"[PATCH 0/5] Distributing substitutes over IPFS" that Ludovic
> submitted a while ago.
>
> What do you people think?

Sounds like a good idea!

> Ludovic, what are the most pressing issues with this patch that you'd
> like to raise?

I think the patches provide a reasonable way forward.  One issue that
was discussed with Hector at <https://issues.guix.gnu.org/issue/33899>
is the fact that my patch would use its own implementation of a
directory index instead of using IPFS’s “UnixFS” abstraction.  It seems
that “UnixFSv2” would fit the bill, but it was not implemented at the
time.

There might be other issues, but I’ve paged that out in the meantime.  :-)
Hopefully if you take a look at that thread and the relevant patches,
you can find potential issues and things that need to be discussed.

Really happy you’ll be discussing these things!

Ludo’.

Re: Lightning talk at IPFS camp

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 324 bytes --]

All good, I had a look at the discussion and I'll study the patch a bit
more.

Today I learnt that Nix also had tried the same thing some 2 years
back.  And they ran into scalability issues.  Maybe we should ask the
Nix folks.  Does anyone know more details about this?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Lightning talk at IPFS camp

[Ludovic Courtès]

Hi,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> All good, I had a look at the discussion and I'll study the patch a bit
> more.
>
> Today I learnt that Nix also had tried the same thing some 2 years
> back.  And they ran into scalability issues.  Maybe we should ask the
> Nix folks.  Does anyone know more details about this?

I remember discussing it with lewo of NixOS at the R-B Summit.  My
recollection is that the prototype that had been developed was using
IPFS in a brute-force fashion, something like directly storing nars and
narinfos, as opposed to storing individual files from the store using
“UnixFS” & co.  I can no longer find the code that we looked at though,
that was somewhere on GitHub.

Note also that inserting files in IPFS takes quite a bit of CPU time and
memory, so that also could take ‘guix publish’ busy.

Ludo’.