From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?utf-8?B?VG9tw6HFoSDEjGVjaA==?= Subject: Re: =?utf-8?B?4oCYZ3VpeCBwdWJsaXNo?= =?utf-8?B?4oCZ?= now compresses archives Date: Tue, 19 Jul 2016 08:29:15 +0200 Message-ID: <20160719062915.3cdpmb6xhcg3l6mw@venom> References: <878twytwlp.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="jq5rn2jmkvibb7fq" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:43341) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bPOWx-0005f9-Fr for guix-devel@gnu.org; Tue, 19 Jul 2016 02:29:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bPOWt-0001PA-78 for guix-devel@gnu.org; Tue, 19 Jul 2016 02:29:23 -0400 Received: from mx2.suse.de ([195.135.220.15]:57537) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bPOWs-0001Oj-Rc for guix-devel@gnu.org; Tue, 19 Jul 2016 02:29:19 -0400 Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 5BB00AAEF for ; Tue, 19 Jul 2016 06:29:17 +0000 (UTC) Content-Disposition: inline In-Reply-To: <878twytwlp.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel --jq5rn2jmkvibb7fq Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 19, 2016 at 12:22:42AM +0200, Ludovic Court=C3=A8s wrote: >Hi! > >=E2=80=98guix publish --compression=E2=80=99 can now compress archives in = gzip format, >using zlib, which makes it much more usable in real life. See commit >4a1fc562ae5eedf40f6ae4eabe30580b0983b8f6. > >=E2=80=98guix publish=E2=80=99 is cache-friendly: it uses /nar/gzip URLs f= or gzipped >archives, and /nar for uncompressed archives. That way, even if =E2=80=98= guix >publish=E2=80=99 is restarted with different compression parameters, previ= ously >announced /nar/* URLs remain valid. > >Please try it and report back. > >To recap, the difficulty with all this is that we couldn=E2=80=99t just ca= ll out >the =E2=80=98gzip=E2=80=99 command because =E2=80=98guix publish=E2=80=99 = uses threads, and threads and >fork(2) don=E2=80=99t go together well. So we needed zlib bindings, and w= hen >discussing it earlier with Dave, I thought we=E2=80=99d have to wrap zlib= =E2=80=99s >low-level API, which is painful, so we were sad and all. Next we had >this hydra.gnu.org outage, which entailed more sadness. > >Then I realized that zlib has this high-level =E2=80=98gz=E2=80=99 API, wh= ich is easy >and does what we need; hence, (guix zlib) provides bindings to that. >The only limitation is that it needs a file descriptor as its source or >sink and cannot compress into memory. That=E2=80=99s enough for =E2=80=98= guix publish=E2=80=99 >though. > >Comments welcome! When I saw archive improvement I remembered I have a comment about that (but not associated to compression). Recently when Snappy and Flatpak news flew around the world I saw some people were thinking about Guix as possible and cleaner alternative to new package types. As I am happy to see enthusiasm about Guix, I don't think that current archive format is usable for distribution of binary packages of whatever origin. Problem is that we have package definition as part of package manager GIT snapshot or local files but this information is not distributed along with the package archive. If this area is interesting for us, we can do better. It would be great to be able to verify such archive, maintain the reproducibility or provide recipe how to bake missing dependencies. I was thinking that we could (and should) add some optional support for metadata which could describe the origin of the package. It could contain: - Guix GIT revision used - differences against GIT revision - local package recipe - exported dependency tree of package recipes - optionally URL where missing dependencies can be found What do you think? S_W --jq5rn2jmkvibb7fq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJXjci0AAoJEEoj40+gM0NtL0MP/RJ9NIFQuhVZByPMHAJEd/B8 aQy/fxzGCV+xhdr8Qu6LFFJ0QVta1POVu7s/SHpdwm7cl7qj9wU3dM5SMiK0MGSP 78gqmQl3SyyjlCrjdT109EKgPxzoxQdxKnVX18fMaSRqlbaV2r+NfPGu9lzwmLIH 88SP8oFInpeN6bXO7PZ/EapZQFeTnvRWPjTQ/D3iInnwFWDi0PwAaU9Sx1/Shodt 0Vke7hBTzXiibaTexODLdS/6qLOoOvM4yET+rbqp0KOJgNzqaHBRRFKeHxwON0Dp xfArOqU6nC9mUWJVaxsAbOi/m2Bln0mU09E0wnbv5mC93HXpu/YkfgGeWGhyuxVt ROoyfpemV/Epwjo83c6V4xmeMCxRm+2IF7jWNtQQyAEk7q42PkqKa3Mj6QJDYVTh X2vFuYrXj1ghvCsLmTqM+ihUmjh+7a1MabdJ+PxzEw5ykgMZ5qhCHyHklRAyZyEH foRRxH3dzUSR3oQRDDs5DNjwWo4P6213Brkf6986fTqnYwn4nQymU9bEoUFLEPj0 RIvPf2brIJ4hAIoW/q6storgIFsHRlUWwZRWNzp4c/aNJUy2BPVKmmzxxEAkDgRO qenMOqYXUdkzO1ZR+W1rF9zlVN+p6JldOzCIYipYxlxeGrot4fgpkfdVUifUM+3h 7mgjUo2jgzcNQ3LNG8KU =eyNC -----END PGP SIGNATURE----- --jq5rn2jmkvibb7fq--