unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* ‘guix publish’ now compresses archives
@ 2016-07-18 22:22 Ludovic Courtès
  2016-07-19  6:29 ` Tomáš Čech
  2016-07-21 10:12 ` Andy Wingo
  0 siblings, 2 replies; 17+ messages in thread
From: Ludovic Courtès @ 2016-07-18 22:22 UTC (permalink / raw)
  To: guix-devel

Hi!

‘guix publish --compression’ can now compress archives in gzip format,
using zlib, which makes it much more usable in real life.  See commit
4a1fc562ae5eedf40f6ae4eabe30580b0983b8f6.

‘guix publish’ is cache-friendly: it uses /nar/gzip URLs for gzipped
archives, and /nar for uncompressed archives.  That way, even if ‘guix
publish’ is restarted with different compression parameters, previously
announced /nar/* URLs remain valid.

Please try it and report back.

To recap, the difficulty with all this is that we couldn’t just call out
the ‘gzip’ command because ‘guix publish’ uses threads, and threads and
fork(2) don’t go together well.  So we needed zlib bindings, and when
discussing it earlier with Dave, I thought we’d have to wrap zlib’s
low-level API, which is painful, so we were sad and all.  Next we had
this hydra.gnu.org outage, which entailed more sadness.

Then I realized that zlib has this high-level ‘gz’ API, which is easy
and does what we need; hence, (guix zlib) provides bindings to that.
The only limitation is that it needs a file descriptor as its source or
sink and cannot compress into memory.  That’s enough for ‘guix publish’
though.

Comments welcome!

Ludo’.

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2016-07-21 20:50 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-07-18 22:22 ‘guix publish’ now compresses archives Ludovic Courtès
2016-07-19  6:29 ` Tomáš Čech
2016-07-19 10:03   ` Pjotr Prins
2016-07-19 13:15   ` Ludovic Courtès
2016-07-19 13:42     ` Tomáš Čech
2016-07-19 14:23       ` Ricardo Wurmus
2016-07-19 15:50         ` Tomáš Čech
2016-07-20 11:20           ` Ricardo Wurmus
2016-07-20 13:05             ` Tomáš Čech
2016-07-20 13:12               ` Thompson, David
2016-07-20 16:10                 ` Tomáš Čech
2016-07-21  5:33                 ` Ricardo Wurmus
2016-07-21 15:58                   ` Thompson, David
2016-07-21  5:53               ` Ricardo Wurmus
2016-07-21 20:50                 ` Tomáš Čech
2016-07-21 10:12 ` Andy Wingo
2016-07-21 12:43   ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).