From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id EKMPAYxwsV7JTwAA0tVLHw (envelope-from ) for ; Tue, 05 May 2020 13:56:28 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id ANJ8JJZwsV7RPwAA1q6Kng (envelope-from ) for ; Tue, 05 May 2020 13:56:38 +0000 Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:470:142::17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 403CF941B06 for ; Tue, 5 May 2020 13:53:22 +0000 (UTC) Received: from localhost ([::1]:47662 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jVy0g-0000mE-Dl for larch@yhetil.org; Tue, 05 May 2020 09:53:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51976) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jVy0M-0000D0-HW for guix-patches@gnu.org; Tue, 05 May 2020 09:53:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51824) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jVy0M-0005Eb-5k for guix-patches@gnu.org; Tue, 05 May 2020 09:53:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jVy0M-0007Eq-4F for guix-patches@gnu.org; Tue, 05 May 2020 09:53:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#40955] [PATCH 4/5] image: Add a new API. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 05 May 2020 13:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 40955 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 40955@debbugs.gnu.org Received: via spool by 40955-submit@debbugs.gnu.org id=B40955.158868676027789 (code B ref 40955); Tue, 05 May 2020 13:53:02 +0000 Received: (at 40955) by debbugs.gnu.org; 5 May 2020 13:52:40 +0000 Received: from localhost ([127.0.0.1]:35137 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jVy00-0007E9-4J for submit@debbugs.gnu.org; Tue, 05 May 2020 09:52:40 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:45702) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jVxzy-0007Ds-5q for 40955@debbugs.gnu.org; Tue, 05 May 2020 09:52:38 -0400 Received: by mail-wr1-f67.google.com with SMTP id k1so2810211wro.12 for <40955@debbugs.gnu.org>; Tue, 05 May 2020 06:52:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=LB17OykEPC0yy+1KVfwN7q8CPjLfMYYoo2NT0mLwAt8=; b=Qvz1eIHKr9s8p4tG/4rsWmwQAX4jeKWwUk0p7Fd2rwU6vtG5A+DWjXmSo9/UKjrLSf YodOtZlijbx78+c+bsYANI6QNtLcr/gLXEt8HG3ByAwLpnIG864RSyEtXItBdZc+KBln yiaaTJC4y3mYZxDIxh6XRqMWQpkgrdnO4PhWQ+PJSwX9vzgMAVieIhT/T2xWR576fyEF GcHTkv0wT16PpeCFCbB96UlV9Qqfq2NI4uAIiC4WVabJhviCFKe2rT28anuGZGFR4Ed7 D4FZUeiZ2hggQDt+CAG4B8sM27I3/ZASplLv31PZyz/WqH8OZtKFvUCHDXpoB/HrkES6 yVcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=LB17OykEPC0yy+1KVfwN7q8CPjLfMYYoo2NT0mLwAt8=; b=OXBFZyl33IKfOeg+wb88Vjaxf47HZtvs2rBzDKNN6a3o0hRdk+E8cxjhSz+DlnAT2V JSK3lqNyPGMqx8O+f3F5Viadc34PFScWAa1wY6lHFu5vE1reTmUti+X8GALzdMjkyLyg CVaHUDlBTYDVQSBYhNm8u7gkyXpXXpG+WyvIRF2geHRs/Su+ldzS67zBD0iYLRYjX7Bh 0tviTakZpo/R9+zE2E/A4FekspdmkgZGNg+j8+ljWG1nIMJGTzFa1hziHIvyo2NspEZb VBFwvrpLl1SRWfwdg2JmGWI5QuT3OT+4yAv7+WxivwVFg/YsHr1mBpgLxoZreTclE8AN PN/g== X-Gm-Message-State: AGi0PuZhzb5/QyWPq2lxHK45fiCgPPYKPpdgZ3yaayKxeCaZR+FHnIVE Czf/5g3tzpaGR6dif2ikrD0Fbclv X-Google-Smtp-Source: APiQypJ1xi9/CNFXe/3kZu7Z2wR5SUOU2OjsvsAmahsSSH75M4ToYBnO4COMqgZ3e9nHskayMhsIXQ== X-Received: by 2002:a5d:658e:: with SMTP id q14mr4102921wru.92.1588686751924; Tue, 05 May 2020 06:52:31 -0700 (PDT) Received: from meru ([2a01:cb18:832e:5f00:7097:5e4f:8b60:ade7]) by smtp.gmail.com with ESMTPSA id d1sm3207541wrx.65.2020.05.05.06.52.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 06:52:31 -0700 (PDT) From: Mathieu Othacehe References: <20200429084756.25072-1-m.othacehe@gmail.com> <20200429084756.25072-4-m.othacehe@gmail.com> <87wo5u8prf.fsf@gnu.org> Date: Tue, 05 May 2020 15:52:30 +0200 In-Reply-To: <87wo5u8prf.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sat, 02 May 2020 14:50:28 +0200") Message-ID: <87pnbicwv5.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 X-Spam-Score: 0.09 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=Qvz1eIHK; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 2001:470:142::17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Scan-Result: default: False [0.09 / 13.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.49544282726439]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2001:470:142::/48:c]; R_DKIM_REJECT(1.00)[gmail.com:s=20161025]; DWL_DNSWL_FAIL(0.00)[2001:470:142::17:server fail]; FREEMAIL_FROM(0.00)[gmail.com]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.12), country: US(-0.00), ip: 2001:470:142::17(-0.50)]; DKIM_TRACE(0.00)[gmail.com:-]; RCPT_COUNT_TWO(0.00)[2]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; MAILLIST(-0.20)[mailman]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; RCVD_IN_DNSWL_FAIL(0.00)[2001:470:142::17:server fail]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:22989, ipnet:2001:470:142::/48, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[larch=yhetil.org]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mothacehe@gmail.com,guix-patches-bounces@gnu.org]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[40955@debbugs.gnu.org]; HAS_LIST_UNSUB(-0.01)[]; RCVD_COUNT_SEVEN(0.00)[9]; FORGED_SENDER_MAILLIST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : SPF not aligned (relaxed),none] X-TUID: HwWXP6kUD5Ij > Would it make sense to separate #:owner-uid and #:owner-gid? Yes, fixed. > > It does mean that we can only create images where all the files have the > same UID/GID. > > Looking at (gnu build install), there=E2=80=99s one case where it might be > problematic: the store=E2=80=99s GID is supposed to match the =E2=80=98gu= ixbuilder=E2=80=99 > group. But the good news is that the daemon does this: > > if (chown(chrootStoreDir.c_str(), 0, buildUser.getGID()) =3D=3D -1) > throw SysError(format("cannot change ownership of =E2=80=98%1%=E2= =80=99") % chrootStoreDir); > > So we can just remove the UID/GID from the directives that are in (gnu > build install). Done in a separate commit. > >> +(define* (genimage config target) >> + "Use genimage to generate in TARGET directory, the image described in= the >> +given CONFIG file." >> + ;; genimage needs a 'root' directory. >> + (mkdir "root") >> + (invoke "genimage" "--config" config >> + "--outputpath" target)) > > I had missed that bit, so we still need genimage in the end? genimage is used to assemble the disk-image together. It's a matter of doing some 'dd' with the right offset. Once this part is implemented in (gnu build image), we can get rid of genimage. >> +(define (register-bootcfg-root target bootcfg) >> + "On file system TARGET, register BOOTCFG as a GC root." >> + (let ((directory (string-append target "/var/guix/gcroots"))) >> + (mkdir-p directory) >> + (symlink bootcfg (string-append directory "/bootcfg")))) > > Maybe just =E2=80=98register-gc-root=E2=80=99? Turns out, I don't think this is useful anymore, so I removed it. > >> +(define* (register-closure prefix closure >> + #:key >> + (deduplicate? #t) (reset-timestamps? #t) >> + (schema (sql-schema))) >> + "Register CLOSURE in PREFIX, where PREFIX is the directory name of the >> +target store and CLOSURE is the name of a file containing a reference g= raph as >> +produced by #:references-graphs.. As a side effect, if RESET-TIMESTAMP= S? is >> +true, reset timestamps on store files and, if DEDUPLICATE? is true, >> +deduplicates files common to CLOSURE and the rest of PREFIX." >> + (let ((items (call-with-input-file closure read-reference-graph))) >> + (register-items items >> + #:prefix prefix >> + #:deduplicate? deduplicate? >> + #:reset-timestamps? reset-timestamps? >> + #:registration-time %epoch >> + #:schema schema))) > > This is duplicated from (guix build vm). Should we instead factorize it > in (guix build store-copy)? I tried it. The problem is that it introduces a dependency to guile-sqlite3, so a bunch of with-extensions needs to be added in various places. I kept the copy, as (gnu build vm) will be deprecated/removed anyways. > I=E2=80=99d suggest either =E2=80=98srfi-1:=E2=80=99 as the prefix or, be= tter, hide whichever > binding is causing a name clash. I need partition from srfi-1 and partition from (gnu image). That's not great :( I opted for the srfi-1 prefix as you suggested. > Can we use =E2=80=98computed-file=E2=80=99 as well instead of =E2=80=98ge= xp->derivation=E2=80=99? That > way, the API is entirely non-monadic and hopefully easier to use. Ok, so I used computed-file as suggested. However, the "system-image" procedure is calling the monadic system-disk-image-in-vm. So I had to keep this one monadic. The good news is once, system-disk-image-in-vm is no longer needed, this whole file will be entirely non-monadic. > That=E2=80=99s all! Thanks a lot for going though all of this. Mathieu