Re: Identical files across subsequent package revisions

[zimoun <zimon.toutoune@gmail.com>]

Hi,

On Wed, 23 Dec 2020 at 14:10, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:

> Probably you're already aware of it, but I want to mention that
> Tridgell's thesis[1] contains a very neat approach to this problem.

This thesis is a must to read! :-)


> A naive prototype would be copying of the latest available nar of the
> package on the client side and using it as the destination for a copy
> using rsync.  Either the protocol used by the rsync application, or a
> protocol based on those ideas, could be implemented over the HTTP layer;
> client and server implementation and cooperation would be needed
> though.

I could misunderstand and miss something, one part of the problem is how
to detect “latest”; other said how to know it is different.  From my
memories, and I have drunk couple of beers since I read the thesis :-),
the ’rsync’ approach uses timestamp and size.  And if you switch to
checksum instead, the performances are poor, because of IO.  Well, it
depends on the number of files and their size, if this checksum are
computed ahead, etc.


> Another idea that might fit well into that kind of protocol---with
> harder impact on the design, and probably with a high cost on the
> runtime---would be the "upgrade" of the deduplication process towards a
> content-based file system as git does[2].  This way the a description of
> the nar contents (size, hash) could trigger the retrieval only of the
> needed files not found in the current store.

Is it not related to Content-Addressed Store?  i.e, «intensional model»?

Chap. 6: <https://edolstra.github.io/pubs/phd-thesis.pdf>
Nix FRC: <https://github.com/tweag/rfcs/blob/cas-rfc/rfcs/0062-content-addressed-paths.md>


Cheers,
simon