From mboxrd@z Thu Jan 1 00:00:00 1970 From: ng0 Subject: Re: Using cuirass to build your own manifest. Date: Wed, 8 Mar 2017 09:48:32 +0000 Message-ID: <20170308094832.tjxcm47d3pk7e4qv@abyayala> References: <87r328w58p.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:59735) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1clX8W-0001SK-KT for guix-devel@gnu.org; Wed, 08 Mar 2017 03:39:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1clX8T-0001Hd-Ew for guix-devel@gnu.org; Wed, 08 Mar 2017 03:39:56 -0500 Received: from latitanza.investici.org ([82.94.249.234]:63406) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1clX8T-0001HU-2L for guix-devel@gnu.org; Wed, 08 Mar 2017 03:39:53 -0500 Content-Disposition: inline In-Reply-To: <87r328w58p.fsf@gmail.com> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Mathieu Othacehe Cc: guix-devel Mathieu Othacehe transcribed 5.9K bytes: > > Hi, > > Here's a small tutorial on how to setup cuirass to build your own > manifest. Nice :) I am working on a similar documentation, it would just extend this by providing examples of how to do this via .onion, via OpenNIC dns name, and inside the freifunk meshnetwork. If I mention you as the original author may I just extend the text, and add my changes? > I see two major reasons for this kind of setup: > > * When you pull latest guix, hydra and bayfront may not have finished > building all the packages you use. > > * Hydra and bayfront won't build your custom packages. > > For this reasons I installed cuirass to build my manifest, using a > configuration very similar to the one used on bayfront > (http://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/bayfront.scm). > > I guess it may not be the easier way to do it so don't hesiste to > comment. Anyway, here's how I set it up. > > 1) config.scm > > --8<---------------cut here---------------start------------->8--- > (define %cuirass-specs > ;; Cuirass specifications to build Guix. > #~(list `((#:name . "guix") > (#:url . "git://git.savannah.gnu.org/guix.git") > (#:branch . "master") > (#:no-compile? #t) > (#:load-path . ".") > ;; Eval function drv-list in file guix-drv.scm > (#:proc . drv-list) > (#:file . #$(local-file "/home/mathieu/conf/guix/cuirass/guix-drv.scm"))))) > (services > ... > (service cuirass-service-type > (cuirass-configuration > (interval 30) ;; git pull guix repo every 30 seconds. > (use-substitutes? #t) > (port 8082) > (load-path '("/home/mathieu/conf/guix/packages" > "/home/mathieu/conf/guix/common_packages")) > (specifications %cuirass-specs)))) > --8<---------------cut here---------------end--------------->8--- > > With this configuration, cuirass service will wake-up every 30 > seconds. It will pull branch master of guix.git. If new commits have > appeared since last evaluation, a build will be triggered. > > Cuirass will evaluate the function specified with #:proc in the file > #:file. This function is supposed to return the list of derivations you > want to build. > > Note that if this list contains custom packages, you have to setup > load-path to point to the directories containing the custom packages > definitions. > > 2) guix-drv.scm > > --8<---------------cut here---------------start------------->8--- > (use-modules (guix config) > (guix store) > (guix grafts) > (guix packages) > (guix ui) > (guix derivations) > (guix monads) > (guix profiles) > (gnu packages) > (srfi srfi-1)) > > (define (drv-package store package) > (lambda () > `((#:job-name . ,(string-append > (package-name package) > "-" > (package-version package) > "-job")) > (#:derivation . ,(derivation-file-name > (parameterize ((%graft? #f)) > (package-derivation store package #:graft? #f))))))) > > (define (drv-list store arguments) > (let* ((manifest > (load* "/home/mathieu/conf/guix/manifest.scm" > (make-user-module > '((guix profiles) (gnu))))) > (packages > (map manifest-entry-item > (manifest-entries manifest)))) > (parameterize ((%graft? #f)) > (map (lambda (package) > (drv-package store package)) > (delete-duplicates! packages))))) > --8<---------------cut here---------------end--------------->8--- > > The drv-list procedure loads the file manifest.scm which content is > detailed below. The list produced by drv-list looks like : > > --8<---------------cut here---------------start------------->8--- > (((#:job-name . "acpi-1.7-job") (#:derivation > . "/gnu/store/r9s5x0ksj02hsw4n3acdxab8ggjp4z7y-acpi-1.7.drv")) ...) > --8<---------------cut here---------------end--------------->8--- > > 3) manifest.scm > > --8<---------------cut here---------------start------------->8--- > (define (spec->packages spec) > (call-with-values (lambda () > (specification->package+output spec)) list)) > > (define packages-list > '("acpi" > "acpica" > ... > "yasm" > "zip" > )) > > (packages->manifest > (map spec->packages packages-list)) > --8<---------------cut here---------------end--------------->8--- > > Now you may want to run cuirass on a dedicated machine and access > packages built by cuirass on other machines. You just need to run guix > publish to do that. > > 4) config.scm > > --8<---------------cut here---------------start------------->8--- > (services > ... > (guix-publish-service #:host "0.0.0.0")) > --8<---------------cut here---------------end--------------->8--- > > 5) other machines config.scm > > --8<---------------cut here---------------start------------->8--- > (services > (modify-services %base-services > (guix-service-type > config => > (guix-configuration > (inherit config) > (substitute-urls '("https://bayfront.guixsd.org" > "https://mirror.hydra.gnu.org" > "https://cuirass-machine-name")))))) > --8<---------------cut here---------------end--------------->8--- > > Where "cuirass-machine-name" is the URL of the machine running cuirass. > > Note that the order in substitute list has an importance. Here, the > substitutes from cuirass-machine-name are only downloaded if they do not > exist on bayfront and hydra. > > Depending on the upload speed of your build machine you may want to put > it on top of the list or not ... > > At last, you need to authorize "cuirass-machine-name" key with this > command : > > --8<---------------cut here---------------start------------->8--- > guix archive --authorize < cuirass-machine-name.pub > --8<---------------cut here---------------end--------------->8--- > > You can refer to > https://www.gnu.org/software/guix/manual/html_node/Substitutes.html#Substitutes > for precisions. > > That's it, I hope it will be helful. > > To finish, the downsides of this setup are : > * You need to keep your manifest up-to-date on your build machine. > * There are no easy ways to track cuirass status but to do some sql on > cuirass database. > > Besides that, it's working fine. > > Happy building, > > Mathieu >