From: Csepp <raingloom@riseup.net>
To: Csepp <raingloom@riseup.net>
Cc: guix-devel@gnu.org, pukkamustard <pukkamustard@posteo.net>
Subject: Re: distributed substitutes: file slicing
Date: Thu, 22 Jun 2023 01:06:50 +0200 [thread overview]
Message-ID: <87zg4sl8d8.fsf@riseup.net> (raw)
In-Reply-To: <87jzvxoicg.fsf@riseup.net>
Csepp <raingloom@riseup.net> writes:
> I have a question / suggestion about the distributed substitutes
> project: would downloads be split into uniformly sized chunks or could
> the sizes vary?
> Specifically, in an extreme case where an update introduced a single
> extra byte at the beginning of a file, would that result in completely
> new chunks?
>
> An alternative I've been thinking about is this:
> find the store references in a file and split it along these references,
> optionally apply further chunking to the non-reference blobs.
>
> It's probably best to do this at the NAR level??
>
> Storing reference offsets is already something that we should be doing to
> speed other operations up, so this could tie in nicely with that.
I was sent this link that others might find interesting:
https://alternativebit.fr/posts/nixos/future-of-nix-substitution/
next prev parent reply other threads:[~2023-06-21 23:11 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-20 22:44 distributed substitutes: file slicing Csepp
2023-06-21 14:32 ` Attila Lendvai
2023-06-21 23:06 ` Csepp [this message]
2023-06-25 9:48 ` pukkamustard
2023-06-26 13:41 ` Florian Klink
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87zg4sl8d8.fsf@riseup.net \
--to=raingloom@riseup.net \
--cc=guix-devel@gnu.org \
--cc=pukkamustard@posteo.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).