Using HTTP pipelining for substitutes

[ludo@gnu.org (Ludovic Courtès)]

Hello!

The new ‘wip-http-pipelining’ branch changes ‘guix substitute-binary’ so
that, instead of using the current brute force approach—a bunch of
threads performing HTTP GETs[*] in parallel to hydra.gnu.org–, it sends all
HTTP GET requests in a row and then reads responses (“HTTP pipeline”.)

I suspect it performs more or less the same as the brute force approach,
but it uses much less resources on the client (memory, file descriptors,
etc.), and likewise on the server (doesn’t have to handle a large number
of parallel connections.)

It should also be more robust, and will make it easy to have progress
report when updating the list of substitutes.

I’ll test it some more and make sure it works with Guile 2.0.5.

Comments welcome!

Ludo’.

[*] The GETs are to check whether hydra.gnu.org has a substitute for a
    given store item.