From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 6BQzOW+v0F8zeQAA0tVLHw (envelope-from ) for ; Wed, 09 Dec 2020 11:05:19 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id wOP5NG+v0F/ZDwAAB5/wlQ (envelope-from ) for ; Wed, 09 Dec 2020 11:05:19 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A8B88940111 for ; Wed, 9 Dec 2020 11:05:19 +0000 (UTC) Received: from localhost ([::1]:45130 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmxHa-0000vh-8b for larch@yhetil.org; Wed, 09 Dec 2020 06:05:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50700) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmxHB-0000vL-5N for guix-devel@gnu.org; Wed, 09 Dec 2020 06:04:53 -0500 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21177) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmxH0-0008Du-Da; Wed, 09 Dec 2020 06:04:51 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1607511871; cv=none; d=zohomail.com; s=zohoarc; b=Cs4+ZPkEZUryGxrRJcxvp4+tIQj2aq16kSkJed+3Ejb1yUcj7328gP3ZG8kQTSAGsQDrHbAh88wRPCnn46Lut5Uzr6mvZKVPITcMVIsQWUGrYS3n0wU7aFXFHxPMxb7jR6g2sA+lbtcmD94S7QJFe5SCNiqa62x2iqFt3KzbKfQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1607511871; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=LWSN5awEzfa84A56Bl8+gIrcnKtLmolvVTTdRlojgK0=; b=jDOzY4DRFbMkExZgIMOlak8Fz3CansKFdwQF4OA0DNVsdztAguzBi/k9X3OzlNm7gb3fRfASqk8F9iwLdsynsIY5Ohx35KhtBxxCoGB4halxeTwJVkvvwaFqdKqn26TiwbiqqpovAe4F5qhHe5YqMa1wePxHCbb2Q7qZljutID4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1607511871; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=LWSN5awEzfa84A56Bl8+gIrcnKtLmolvVTTdRlojgK0=; b=JIIgPlc3eGfyiysW4zwMoVxJmzwm6nEPK7ATEyEHcQaY+cS5ksFqYnY+ns6Fy4Qq yEtZ3ScoNcQY37rgYH4qpcPIvZ1U4lbVos5/SOtNE1g5ZBduNaSAxbSxNWozGNO9Na6 JoS2gFn84IzLR/NE3qDkafIe7v8PswwAWt/PMVEE= Received: from localhost (p54ad4a4f.dip0.t-ipconnect.de [84.173.74.79]) by mx.zohomail.com with SMTPS id 1607511870078413.1354791221695; Wed, 9 Dec 2020 03:04:30 -0800 (PST) References: <875z5f6z1a.fsf@elephly.net> <877dpswnxl.fsf@gnu.org> User-agent: mu4e 1.4.13; emacs 27.1 From: Ricardo Wurmus To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: guix pack file enumerator? In-reply-to: <877dpswnxl.fsf@gnu.org> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Wed, 09 Dec 2020 12:04:26 +0100 Message-ID: <87a6unnsud.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.01 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=JIIgPlc3; arc=pass (zohomail.com:s=zohoarc:i=1); dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: A8B88940111 X-Spam-Score: -1.01 X-Migadu-Scanner: ns3122888.ip-94-23-21.eu X-TUID: efNBRtFRDA2/ Ludovic Court=C3=A8s writes: > Hi, > > Ricardo Wurmus skribis: > >> =E2=80=9Cguix pack=E2=80=9D is great for deployment of applications to s= ervers that >> don=E2=80=99t have Guix. For a project I have a =E2=80=9Cdeploy=E2=80= =9D target in my Makefile >> that essentially does this: >> >> cat $(shell guix pack -RR -e '(load "guix.scm")' -S /bin=3Dbin) | ss= h remote-server "tar xvzf - -C /where/i/want/it" >> >> This is fine for small deployments, but it=E2=80=99s a little annoying t= hat it >> transfer *all* the files, even those that haven=E2=80=99t changed. So I= thought >> I could use rsync here, but it=E2=80=99s inconvenient that =E2=80=9Cguix= pack=E2=80=9D will do >> what it was designed for and produce a single file bundle. >> >> What do you think about adding an output format that is no format at all >> but a file enumeration printed to stdout? That way I could use =E2=80= =9Cguix >> pack=E2=80=9D to produce a list of files to transfer and use that to tra= nsfer >> only the unchanged files. Alternatively, perhaps we could have a >> =E2=80=9Cdirectory=E2=80=9D format that merely copies (or links) the fil= es to a new >> directory root. > > You could untar the pack and rsync it. We could have a =E2=80=98director= y=E2=80=99 > format but I=E2=80=99m afraid it would duplicate too much of the tarball = code > (we=E2=80=99d have to check). I=E2=80=99m unpacking to rsync now, but it=E2=80=99s not great to duplicate= files on the local disk only to send them over the network. The ideal workflow for me would treat the store on my local machine as the source and the remote machine as the target, without any copying from source to source in between. The local-unpack method also requires an empty directory to be maintained; it needs to be emptied before the next time we try to unpack anything. Another downside is the creation of the tarball itself. I don=E2=80=99t ne= ed it but it=E2=80=99s still created only to be destroyed in the next step. This= is unfortunate, because Guix could use a different mechanism to =E2=80=9Ccheck= out=E2=80=9D files from the store to a new location. > When you think about it, you could just as well have Guix on the other > side and then use =E2=80=98guix copy=E2=80=99=E2=80=A6 or maybe use Guix = directly there. (As in > =E2=80=98guix pack --localstatedir -RR guix=E2=80=99.) I didn=E2=80=99t consider this, but yes: this is an option. It requires the target to be actively involved, though, which means that I need to spend more time preparing it. There=E2=80=99s a certain appeal in treating the t= arget as a mere dumping ground and coordinating everything from my development machine. > I know it=E2=80=99s not what you had in mind, but it seems to me that the= re=E2=80=99s a > continuum here, and maybe there=E2=80=99s just a gap to bridge to allow f= or > smoother workflows? Probably. The only difficulty I see is that none of the formats that =E2=80=9Cguix pack=E2=80=9D supports allow for arbitrary variations in the = packing process, such as using a different tool for the packing. To be fair: none of the currently supported formats need this level of flexibility. But extending =E2=80=9Cguix pack=E2=80=9D to domains that it was not primar= ily designed to cover (such as deploying a pack before it is produced as a local artifact) may require some of that flexibility. --=20 Ricardo