all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Vijaya Anand <sunrockers8@gmail.com>
To: Attila Lendvai <attila@lendvai.name>
Cc: pukkamustard <pukkamustard@posteo.net>, guix-devel@gnu.org
Subject: Re: [GSoC 23] distributed substitutes, cost of storage
Date: Mon, 27 Mar 2023 01:36:29 +0530	[thread overview]
Message-ID: <CAKg+GqaXGA+erSWFKRxQmxsnHm8-JJhffH2W1WTnbfkK4soqkQ@mail.gmail.com> (raw)
In-Reply-To: <sf9aTCKoDqrpSn5OUOGPujCbirKQFxG41GR_3Q3iEB1CqdhUjPL9lt7gC-JiMdQYruX33bAYlZw2YtrJh-JBKLJRoQW-61Ns9ljFllmEnkI=@lendvai.name>

[-- Attachment #1: Type: text/plain, Size: 3899 bytes --]

Hi Attila

Thanks for the welcome!
I agree that the responsibility of re-uploading the blocks back to the
network should be with the clients rather than the substitute server. Also
I didn't really think about the point about having to pay for the p2p
services at some point of time. In this case we will have to pay for the
storage of substitutes both on the p2p storage backend as well as for
storage in the substitute server am I right? So ideally we will want to
eliminate the usage of these substitute servers and shift totally to p2p
services and in this case we will have to shift the responsibility of
re-uploading the blocks to the clients itself.
Also if we dont keep the re-uploading blocks option as default for the
users, won't users usually not choose to enable it? Maybe we can keep it on
as default and resource conscious users can choose to turn it off? Please
let me know your thoughts on these points and I will change the
implementation point of my proposal accordingly.

Thank you
Vijaya Anand

On Sun, 26 Mar 2023 at 00:30, Attila Lendvai <attila@lendvai.name> wrote:

> welcome on board Anand!
>
>
> > In case a user requests for a substitute and there is a missing
> > block in the decoding process, a HTTP request for block would sent
> > to the substitute server and the server will encode the
> > corresponding block in real time and push it back into the
> > network. The block will be searched again and retrieved.
>
>
> something to consider here: whose responsibility should it be that a
> block, that is missing from a p2p network, is (re-)uploaded there? the
> clients? or the current substitute server?
>
> my gut instinct says that it's better if the clients do the (re-)upload of
> the blocks.
>
> in this architecture the substitute server is just another storage
> mechanism along the other storage backends (although with a different
> reliability characteristics), and it's the clients that are doing the
> mirroring/spreading/distribution of the blocks among the various backends.
> the clients of course will/should keep the current substitute servers at
> the bottom of their list of backends in their configuration.
>
> this way the load is distributed, and we don't need to add (too much)
> extra complexity to the substitute server codebase, and the actors are less
> tightly coupled.
>
> it's another question whether this mirroring should be enabled by default
> in the clients. probably it shouldn't, and the project infrastructure
> should be running clients where it is turned on. altruistic third parties
> could also enable this mirroring feature, and donate their
> bandwidth/resources.
>
> there's an issue with this, though:
>
> some p2p storage backends will require some form of payment/credentials to
> use their resources. arguably, all p2p storage networks that will survive
> into the future will have some mechanism to limit the infinite abuse of
> their resources. it is to be researched how these payment mechanisms work
> on the various p2p networks, and whether it is possible that the Guix
> project pays for the storage globally, and then the random clients will
> have the necessary credentials to (re-)upload the missing blocks.
>
> this architecture shouldn't be impossible, because the content is
> authenticated by its hash, and if the payment/authorization mechanism is
> based on the hashes of the blocks (probably), then any client could
> (re-)upload a missing block that was already paid for.
>
> i'll look into this, especially in the context of Swarm.
>
> meta: i think such specific discussions should be kept off-list, but the
> financing of the storage fees is probably something that should be known
> about more widely.
>
> --
> • attila lendvai
> • PGP: 963F 5D5F 45C7 DFCD 0A39
> --
> Every lie is a debt to the truth.
>
>

[-- Attachment #2: Type: text/html, Size: 4355 bytes --]

  reply	other threads:[~2023-03-26 20:07 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-25 19:00 [GSoC 23] distributed substitutes, cost of storage Attila Lendvai
2023-03-26 20:06 ` Vijaya Anand [this message]
2023-03-26 21:19   ` Attila Lendvai
2023-03-28 20:19     ` Vijaya Anand
2023-03-29  8:45       ` Andreas Enge
2023-03-29  9:26         ` pukkamustard
2023-03-29  9:34       ` pukkamustard
2023-03-30 11:08 ` Maxime Devos
2023-04-04 10:53   ` Attila Lendvai
2023-04-04 18:51     ` Maxime Devos
2023-04-05  7:19       ` Attila Lendvai
2023-04-06  8:13     ` Simon Tournier
2023-04-07 22:45       ` Attila Lendvai
2023-04-08  0:46         ` Csepp
2023-04-08 16:05           ` Attila Lendvai
2023-04-08  9:30         ` Simon Tournier
2023-04-08 15:53           ` Attila Lendvai

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAKg+GqaXGA+erSWFKRxQmxsnHm8-JJhffH2W1WTnbfkK4soqkQ@mail.gmail.com \
    --to=sunrockers8@gmail.com \
    --cc=attila@lendvai.name \
    --cc=guix-devel@gnu.org \
    --cc=pukkamustard@posteo.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.