Re: Experimental nar-herder support for serving fixed output files by hash

[Christopher Baines <mail@cbaines.net>]

[-- Attachment #1: Type: text/plain, Size: 2087 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

>>> IWBN to share as much code as possible with ‘guix publish’, which has
>>> great test suite coverage and is being hammered every day.  Clearly the
>>> bit about extracting nars is specific to the nar-herder though, so that
>>> may prove difficult.
>>
>> I'm going to look at the Guile Fibers web server, hopefully that can be
>> improved to support streaming responses, which would allow removing a
>> lot of custom code from guix publish.
>
> By “streaming responses”, do you mean pipelining?

Streaming might not be the best word, but I'm referring to not having to
have the entire response body in memory before sending the first byte.

I've now done an initial implementation of this:

  https://github.com/wingo/fibers/pull/63

> How would that affect ‘guix publish’?

My reading of the concurrent-http-server in the publish script suggests
it could be replaced by the fibers web server, if it supports
"streaming" response bodies, as I've described above.

That might introduce some fibers related issues as I'm guessing there
might be some native code used for compression, but it would be a way to
remove the workarounds related to the web server part.

>> There isn't all that much code to the nar-herder though, and most of
>> waht is there is doing different things to guix publish, so I'm not sure
>> there's all that much to share.
>>
>> What I was getting at here though, ignoring the implementation, was
>> whether this is worthwhile to do? As in, is there benefit to having this
>> and being able to extend the content addressed mirrors that Guix uses?
>
> Having more content-addressed mirrors is worthwhile IMO, yes.

Cool :)

> Having two different implementations of the same interfaces may not be
> ideal, though, in terms of long-term maintenance cost.

Indeed, and I'm not set on keeping the nar-herder separate from guix
publish, but the nar-herder does seem to be doing a good job, and I
haven't seen a way of unifying the tooling yet.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]