From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mathieu Othacehe Subject: Using cuirass to build your own manifest. Date: Wed, 08 Mar 2017 09:19:02 +0100 Message-ID: <87r328w58p.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:54033) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1clWoP-0004IQ-Tc for guix-devel@gnu.org; Wed, 08 Mar 2017 03:19:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1clWoL-0004NG-4X for guix-devel@gnu.org; Wed, 08 Mar 2017 03:19:09 -0500 Received: from mail-wm0-x234.google.com ([2a00:1450:400c:c09::234]:34407) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1clWoK-0004N9-Qc for guix-devel@gnu.org; Wed, 08 Mar 2017 03:19:05 -0500 Received: by mail-wm0-x234.google.com with SMTP id 196so28576105wmm.1 for ; Wed, 08 Mar 2017 00:19:04 -0800 (PST) Received: from arch_pc ([46.218.109.88]) by smtp.gmail.com with ESMTPSA id d42sm3121860wrd.37.2017.03.08.00.19.02 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 Mar 2017 00:19:02 -0800 (PST) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel Hi, Here's a small tutorial on how to setup cuirass to build your own manifest. I see two major reasons for this kind of setup: * When you pull latest guix, hydra and bayfront may not have finished building all the packages you use. * Hydra and bayfront won't build your custom packages. For this reasons I installed cuirass to build my manifest, using a configuration very similar to the one used on bayfront (http://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/bayfront.scm). I guess it may not be the easier way to do it so don't hesiste to comment. Anyway, here's how I set it up. 1) config.scm --8<---------------cut here---------------start------------->8--- (define %cuirass-specs ;; Cuirass specifications to build Guix. #~(list `((#:name . "guix") (#:url . "git://git.savannah.gnu.org/guix.git") (#:branch . "master") (#:no-compile? #t) (#:load-path . ".") ;; Eval function drv-list in file guix-drv.scm (#:proc . drv-list) (#:file . #$(local-file "/home/mathieu/conf/guix/cuirass/guix-drv.scm"))))) (services ... (service cuirass-service-type (cuirass-configuration (interval 30) ;; git pull guix repo every 30 seconds. (use-substitutes? #t) (port 8082) (load-path '("/home/mathieu/conf/guix/packages" "/home/mathieu/conf/guix/common_packages")) (specifications %cuirass-specs)))) --8<---------------cut here---------------end--------------->8--- With this configuration, cuirass service will wake-up every 30 seconds. It will pull branch master of guix.git. If new commits have appeared since last evaluation, a build will be triggered. Cuirass will evaluate the function specified with #:proc in the file #:file. This function is supposed to return the list of derivations you want to build. Note that if this list contains custom packages, you have to setup load-path to point to the directories containing the custom packages definitions. 2) guix-drv.scm --8<---------------cut here---------------start------------->8--- (use-modules (guix config) (guix store) (guix grafts) (guix packages) (guix ui) (guix derivations) (guix monads) (guix profiles) (gnu packages) (srfi srfi-1)) (define (drv-package store package) (lambda () `((#:job-name . ,(string-append (package-name package) "-" (package-version package) "-job")) (#:derivation . ,(derivation-file-name (parameterize ((%graft? #f)) (package-derivation store package #:graft? #f))))))) (define (drv-list store arguments) (let* ((manifest (load* "/home/mathieu/conf/guix/manifest.scm" (make-user-module '((guix profiles) (gnu))))) (packages (map manifest-entry-item (manifest-entries manifest)))) (parameterize ((%graft? #f)) (map (lambda (package) (drv-package store package)) (delete-duplicates! packages))))) --8<---------------cut here---------------end--------------->8--- The drv-list procedure loads the file manifest.scm which content is detailed below. The list produced by drv-list looks like : --8<---------------cut here---------------start------------->8--- (((#:job-name . "acpi-1.7-job") (#:derivation . "/gnu/store/r9s5x0ksj02hsw4n3acdxab8ggjp4z7y-acpi-1.7.drv")) ...) --8<---------------cut here---------------end--------------->8--- 3) manifest.scm --8<---------------cut here---------------start------------->8--- (define (spec->packages spec) (call-with-values (lambda () (specification->package+output spec)) list)) (define packages-list '("acpi" "acpica" ... "yasm" "zip" )) (packages->manifest (map spec->packages packages-list)) --8<---------------cut here---------------end--------------->8--- Now you may want to run cuirass on a dedicated machine and access packages built by cuirass on other machines. You just need to run guix publish to do that. 4) config.scm --8<---------------cut here---------------start------------->8--- (services ... (guix-publish-service #:host "0.0.0.0")) --8<---------------cut here---------------end--------------->8--- 5) other machines config.scm --8<---------------cut here---------------start------------->8--- (services (modify-services %base-services (guix-service-type config => (guix-configuration (inherit config) (substitute-urls '("https://bayfront.guixsd.org" "https://mirror.hydra.gnu.org" "https://cuirass-machine-name")))))) --8<---------------cut here---------------end--------------->8--- Where "cuirass-machine-name" is the URL of the machine running cuirass. Note that the order in substitute list has an importance. Here, the substitutes from cuirass-machine-name are only downloaded if they do not exist on bayfront and hydra. Depending on the upload speed of your build machine you may want to put it on top of the list or not ... At last, you need to authorize "cuirass-machine-name" key with this command : --8<---------------cut here---------------start------------->8--- guix archive --authorize < cuirass-machine-name.pub --8<---------------cut here---------------end--------------->8--- You can refer to https://www.gnu.org/software/guix/manual/html_node/Substitutes.html#Substitutes for precisions. That's it, I hope it will be helful. To finish, the downsides of this setup are : * You need to keep your manifest up-to-date on your build machine. * There are no easy ways to track cuirass status but to do some sql on cuirass database. Besides that, it's working fine. Happy building, Mathieu