From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: running a script in a post-build hook? Date: Fri, 07 Oct 2016 21:42:13 +0200 Message-ID: <87a8egj696.fsf@gnu.org> References: <877f9kfar7.fsf@mdc-berlin.de> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:41491) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bsb2E-0001I7-1u for help-guix@gnu.org; Fri, 07 Oct 2016 15:42:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bsb2A-0006CZ-TT for help-guix@gnu.org; Fri, 07 Oct 2016 15:42:22 -0400 In-Reply-To: <877f9kfar7.fsf@mdc-berlin.de> (Ricardo Wurmus's message of "Fri, 7 Oct 2016 17:18:36 +0200") List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+gcggh-help-guix=m.gmane.org@gnu.org Sender: "Help-Guix" To: Ricardo Wurmus Cc: help-guix Hello! Ricardo Wurmus skribis: > here=E2=80=99s my problem: I need to have the store on a big slow NFS ser= ver > with online compression and deduplication. This means that *everything* > in Guix is slow: downloading binaries, building packages from source, > building a new profile generation =E2=80=94 it=E2=80=99s all *very* slow. This is a tricky use case=E2=80=A6 > So I thought: I should be able to just have the store on fast local > disks and copy over store items to the slow NFS server when they are > done. This is not easy because the store is very big and at the > filesystem level we cannot take advantage of the fact that the store is > append-only for the most part. So running rsync, for example, won=E2=80= =99t cut > it. > > The daemon, however, does have information about completed builds and > new store items. Does the daemon have a post-build hook that can be > called with the names of the new store items which I could then copy > over? (Or even the names of store items I need to delete after =E2=80=9C= guix > gc=E2=80=9D.) > > Could the build hook feature be used for this, maybe by wrapping the > normal build such that a script is run when it finishes? The build hook =E2=80=9Cprotocol=E2=80=9D doesn=E2=80=99t work like this. = The daemon sends a build request, which the hook can accept, postpone, or decline (see (guix scripts offload)). When it accepts, the hook cannot invoke the daemon (it=E2=80=99s not =E2=80=9Creentrant=E2=80=9D.) The substituter pro= tocol is similar. > Finally, is this a good idea? Or is /gnu/store/.links going to be a > problem? Should a post-build hook also inform about deduplication > decisions? Or should I just turn of deduplication in my case? Turning off deduplication would probably help because deduplication is I/O intensive. Otherwise maybe a file system level hack? Like making /gnu/store a unionfs that writes elsewhere? HTH! Ludo=E2=80=99.