From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id OFEND1W+11/dQgAA0tVLHw (envelope-from ) for ; Mon, 14 Dec 2020 19:34:45 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 6Pr9ClW+11/vJQAAbx9fmQ (envelope-from ) for ; Mon, 14 Dec 2020 19:34:45 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C0318940396 for ; Mon, 14 Dec 2020 19:34:43 +0000 (UTC) Received: from localhost ([::1]:47996 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kotcH-0005Ve-EM for larch@yhetil.org; Mon, 14 Dec 2020 14:34:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:57958) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kotbw-0005VU-HM for guix-devel@gnu.org; Mon, 14 Dec 2020 14:34:20 -0500 Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]:53898) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kotbh-0008Pz-9X for guix-devel@gnu.org; Mon, 14 Dec 2020 14:34:19 -0500 Received: by mail-wm1-x32d.google.com with SMTP id k10so14832866wmi.3 for ; Mon, 14 Dec 2020 11:34:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greghogan-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Qks3srE2iTjJXDAYwQu07tVoavfuVhfOdQF+Spbcnpw=; b=MTfvQCaQ0JhuXESpNdzk8rCmuobYsudn4TNPjMkL2oUtJEP3uwqo96P2AIUoj+9jEJ sDx9GbszkwXVDplaQ5YmMgczlaa2x2r9BbimlFybk4vlBV/O4Cs7W5FXhyVp+lRFU45T fZq/2XHmSjroolbWoEBsSAUzWG3bf/CT9T2HTmcyKv3pI2TMo2yU1TxfTfe0/hi0PZAt fQYMiA6TYyZljBBFAFRjKklOXVKnomfg2Py/2tPFngP3kMsdpSoxqu5Na0ivgGUy5RAO D6AZ1YvEDSFBXepWBFDOXQirYuLn9b2jOGLj+rXqCG73pxuxev6nV4GNJbm1DYUtmfLV n9Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Qks3srE2iTjJXDAYwQu07tVoavfuVhfOdQF+Spbcnpw=; b=fJ7oVIE5VgN+jTSgMKQ8tiNoOlfEE8TXaxc0J06DqcI5Um7NCe/TbiVz32YexPkrmz w52TCGMs4tcJr/DoJmD1fhNoY4xnOzpeb/WCXMuuKToIQuZDrKjqCOgaqoJkafqIQR7c 1HPIeslxHD5FEvJW+olBLhAb91ESTIE+vICEtvL/lWO2UtIrxwhoJ06MdwEv1IY09Pgz a15u3gHJCG3PL1QO77qpeXtU2GQPOMWEn2+Ii3D+pjXM5drw4H4ddm1gOA9vItzRDW6L IvjVqyBytpDVo2NIPe6zV2F2qlZJH8iuXT8tC6MgqtmtcEfnjb8hMgcxi2PTUmfmTGEE yjNA== X-Gm-Message-State: AOAM5300+R4goQIxFSuIRpRFAyS2MjVmxTdg8e15B2UAd3ghiAvnu3lP ylxUM87QGaRcouBpkp4HZJsIwJHLEU5hOM1AiITf/mgvD+SguRog X-Google-Smtp-Source: ABdhPJzDmK1TUp7c1OcH+G5KiB1DRFmX/ZURh1g7UC0pYVmY3sQd+KT1Q5e1j/eLbNPi5bYrwtR39Qqu8dQ7MlXM0cA= X-Received: by 2002:a1c:3c09:: with SMTP id j9mr29556851wma.180.1607974441301; Mon, 14 Dec 2020 11:34:01 -0800 (PST) MIME-Version: 1.0 References: <87y2i4j9z4.fsf@nckx> In-Reply-To: <87y2i4j9z4.fsf@nckx> From: Greg Hogan Date: Mon, 14 Dec 2020 19:33:49 +0000 Message-ID: Subject: Re: Offline build failure To: Tobias Geerinckx-Rice Content-Type: multipart/alternative; boundary="00000000000061cfdb05b671b9a2" Received-SPF: none client-ip=2a00:1450:4864:20::32d; envelope-from=code@greghogan.com; helo=mail-wm1-x32d.google.com X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, PDS_BTC_ID=0.499, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.51 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=greghogan-com.20150623.gappssmtp.com header.s=20150623 header.b=MTfvQCaQ; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: C0318940396 X-Spam-Score: -2.51 X-Migadu-Scanner: scn0.migadu.com X-TUID: 7RyZ+gILoPd6 --00000000000061cfdb05b671b9a2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks, Tobias. I am now properly populating the store. I have switched to using 'guix graph --type=3Dderivation' to pull in what seems to be the full set of dependencies. I am seeing a strange issue when importing a package. I can typically duplicate and import a file and it is copied to the original location, for example: ---------------------------------------- $ cp -a /gnu/store/v2zls012iwxyw6058cir7n2b792lsvc9-perl-5.30.2.tar.gz perl-5.30.2.tar.gz $ guix download perl-5.30.2.tar.gz /gnu/store/v2zls012iwxyw6058cir7n2b792lsvc9-perl-5.30.2.tar.gz 128nfdxcvxfn5kq55qcfrx2851ys8hv794dcdxbyny8rm7w7vnv6 $ cp -a /gnu/store/17acz7ks1f7xn6yp1a2y4g7vc6bhr8wc-make-3.82.tar.gz make-3.82.tar.gz $ guix download make-3.82.tar.gz /gnu/store/17acz7ks1f7xn6yp1a2y4g7vc6bhr8wc-make-3.82.tar.gz 1rs2f9hmvy3q6zkl15jnlmnpgffm0bhw5ax0h5c7q604wqrip69x ---------------------------------------- For the Python-3.5.9 tarball the first version loads in the expected way. The second tarball is restored to a new location: ---------------------------------------- $ cp -a /gnu/store/f99fblkzb6ip268sg096shhs7wzjyp55-Python-3.5.9.tar.xz Python-3.5.9.tar.xz $ guix download Python-3.5.9.tar.xz /gnu/store/f99fblkzb6ip268sg096shhs7wzjyp55-Python-3.5.9.tar.xz 0jdh9pvx6m6lfz2liwvvhn7vks7qrysqgwn517fkpxb77b33fjn2 $ cp -a /gnu/store/nj79fxxl5wvnq7jpj2wgbx0591gkjw41-Python-3.5.9.tar.xz Python-3.5.9.tar.xz $ guix download Python-3.5.9.tar.xz /gnu/store/9sa83nyjlm5dyhwys4imm1wa40mjaw1x-Python-3.5.9.tar.xz 0rkn451qfz3gbni57la00a5fbgish9jmm5bmhmgmf223vxwya447 ---------------------------------------- Since the tarball is not restored to the original location the guix build command still attempts the download and fails the offline build. Greg Hogan On Fri, Dec 11, 2020 at 9:42 PM Tobias Geerinckx-Rice wrote: > Hullo Greg, > > Greg Hogan =E5=86=99=E9=81=93=EF=BC=9A > > If there is a better way to setup / configure / execute offline > > builds > > please let me know! > > ...yes :-) > > > I am attempting an offline build without success. I have a Guix > > 1.2.0 node > > with internet access on which I download sources with transitive > > dependencies: > > $ guix build --sources=3Dtransitive tzdata > ~/transfer > > OK. > > > I then copy the files as root to a Guix 1.2.0 node without > > internet access > > (only local network access): > > # cat /home//transfer | xargs -n 1 -I{} scp -p {} > > :{} > > Now you've basically reinvented =E2=80=98guix copy --to=3D=E2=80=99, = but in a > way that won't update the store database in /var/guix/db. I'm > afraid that won't work. > > Guix won't =E2=80=98see=E2=80=99 the files you copy to the remote store a= nd will > consider them G to be C'd next time you run =E2=80=98guix gc=E2=80=99. O= r in this > case: > > > Guix starts downloading and the transferred file is gone! > > Same thing. > > > I'm lost as to > > why a new download attempt is made as the file data and > > timestamps match > > the original server. > > If the file isn't registered in the database, the store item is > never considered valid. Guix doesn't (yet) care about the > data/timestamps at this point. > > If there's a reason you can't/won't use =E2=80=98guix copy=E2=80=99, you = might > work around that by copying each file in ~/transfer to, say, > :/tmp/staging (instead of :/gnu/store), then running =E2=80=98gui= x > download /tmp/staging/...=E2=80=99 on the remote host. > > Kind regards, > > T G-R > --00000000000061cfdb05b671b9a2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks, Tobias. I am now properly populating the stor= e. I have switched to using 'guix graph --type=3Dderivation' to pul= l in what seems to be the full set of dependencies. I am seeing a strange i= ssue when importing a package. I can typically duplicate and import a file = and it is copied to the original location, for example:

<= /div>
----------------------------------------
$ cp -a /gnu/s= tore/v2zls012iwxyw6058cir7n2b792lsvc9-perl-5.30.2.tar.gz perl-5.30.2.tar.gz=
$ guix download perl-5.30.2.tar.gz
/gnu/store/v2zls012iwxyw6058cir7= n2b792lsvc9-perl-5.30.2.tar.gz
128nfdxcvxfn5kq55qcfrx2851ys8hv794dcdxbyn= y8rm7w7vnv6

$ cp -a /gnu/store/17acz7ks1f7xn6yp1a2y4g7vc6bhr8wc-make= -3.82.tar.gz make-3.82.tar.gz
$ guix download make-3.82.tar.gz
/gnu/s= tore/17acz7ks1f7xn6yp1a2y4g7vc6bhr8wc-make-3.82.tar.gz
1rs2f9hmvy3q6zkl1= 5jnlmnpgffm0bhw5ax0h5c7q604wqrip69x
-------------------------= ---------------

For the Python-3.5.9 tarball the f= irst version loads in the expected way. The second tarball is restored to a= new location:

-------------------------------= ---------
$ cp -a /gnu/store/f99fblkzb6ip268sg096shhs7wzjyp55-Pyt= hon-3.5.9.tar.xz Python-3.5.9.tar.xz
$ guix download Python-3.5.9.tar.xz=
/gnu/store/f99fblkzb6ip268sg096shhs7wzjyp55-Python-3.5.9.tar.xz
0jdh= 9pvx6m6lfz2liwvvhn7vks7qrysqgwn517fkpxb77b33fjn2

$= cp -a /gnu/store/nj79fxxl5wvnq7jpj2wgbx0591gkjw41-Python-3.5.9.tar.xz Pyth= on-3.5.9.tar.xz
$ guix download Python-3.5.9.tar.xz
/gnu/store/9sa83n= yjlm5dyhwys4imm1wa40mjaw1x-Python-3.5.9.tar.xz
0rkn451qfz3gbni57la00a5fb= gish9jmm5bmhmgmf223vxwya447
---------------------------------= -------

Since the tarball is not restored to the o= riginal location the guix build command still attempts the download and fai= ls the offline build.

Greg Hogan
On Fri, = Dec 11, 2020 at 9:42 PM Tobias Geerinckx-Rice <me@tobias.gr> wrote:
Hullo Greg,

Greg Hogan =E5=86=99=E9=81=93=EF=BC=9A
> If there is a better way to setup / configure / execute offline
> builds
> please let me know!

...yes :-)

> I am attempting an offline build without success. I have a Guix
> 1.2.0 node
> with internet access on which I download sources with transitive
> dependencies:
>=C2=A0 =C2=A0$ guix build --sources=3Dtransitive tzdata > ~/transfer=

OK.

> I then copy the files as root to a Guix 1.2.0 node without
> internet access
> (only local network access):
>=C2=A0 =C2=A0# cat /home/<user>/transfer | xargs -n 1 -I{} scp -p= {}
>=C2=A0 =C2=A0<ip>:{}

Now you've basically reinvented =E2=80=98guix copy --to=3D<ip>=E2= =80=99, but in a
way that won't update the store database in /var/guix/db.=C2=A0 I'm=
afraid that won't work.

Guix won't =E2=80=98see=E2=80=99 the files you copy to the remote store= and will
consider them G to be C'd next time you run =E2=80=98guix gc=E2=80=99.= =C2=A0 Or in this
case:

> Guix starts downloading and the transferred file is gone!

Same thing.

> I'm lost as to
> why a new download attempt is made as the file data and
> timestamps match
> the original server.

If the file isn't registered in the database, the store item is
never considered valid.=C2=A0 Guix doesn't (yet) care about the
data/timestamps at this point.

If there's a reason you can't/won't use =E2=80=98guix copy=E2= =80=99, you might
work around that by copying each file in ~/transfer to, say,
<ip>:/tmp/staging (instead of <ip>:/gnu/store), then running = =E2=80=98guix
download /tmp/staging/<file>...=E2=80=99 on the remote host.

Kind regards,

T G-R
--00000000000061cfdb05b671b9a2--