From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id yG/mE5g91F5lCAAA0tVLHw (envelope-from ) for ; Sun, 31 May 2020 23:28:24 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id KEvDD5g91F4DNgAA1q6Kng (envelope-from ) for ; Sun, 31 May 2020 23:28:24 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 78AB8940415 for ; Sun, 31 May 2020 23:28:23 +0000 (UTC) Received: from localhost ([::1]:57064 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfXNM-00041h-L7 for larch@yhetil.org; Sun, 31 May 2020 19:28:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35164) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfXNF-00041a-AO for help-guix@gnu.org; Sun, 31 May 2020 19:28:13 -0400 Received: from mail-qv1-xf30.google.com ([2607:f8b0:4864:20::f30]:41735) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jfXNE-0000s2-69 for help-guix@gnu.org; Sun, 31 May 2020 19:28:12 -0400 Received: by mail-qv1-xf30.google.com with SMTP id er17so1002040qvb.8 for ; Sun, 31 May 2020 16:28:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=TxiGutdk+cw/vWDiEwFquSIT7Gs3WUrB1rF71nyyzFA=; b=IO33pTB+cIl8eaf5zotQw4tMIbbfMJmbZ7HyhG4SYyKSTHH+q/ZEtDiqONHV3DcSyq Y5mpreLHOuG0uZmuThja7TPL77sVEwKbYod3vFnBfd4bYekKo3/MH/ePefiPRwXXVHHm g13u6iVMdLhKINBpEHM/zT8Jh/6TN6Jekvt5EIl4qHCyHXRK/1+hdjrd4ISKhj+DlY/8 HBOdCgDDtB+jgWfWu9oHDL72BOzwl7NKBemnX7bgyN7Cux5z8rlbdBDbbrD3SWKl9Sxi jMBn8j6GYhFSa76YOTUjbHt1EDiCJ4egomze+IEgs+bA4WS0jZs9Gvjg/wv7Uapxbh36 TOsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TxiGutdk+cw/vWDiEwFquSIT7Gs3WUrB1rF71nyyzFA=; b=qvZg+HaeX+5nic8U9s9W5kkq8wEwbXEGyemJFiMcvYYJcI8Zms6ES1cF7Cb4P7H7UF jAB3jMmU+6nFmuIbWQMIuTOhdqygSwMPYQS3uV2hkhAERb7DnrL1ZCEFfVeN4HsIhFGO cEvpAVYJr3Lhu+1H2rEXxoFuaF/+L+QenBepe2/aKG2VeZYzVouAWKI10G3eFW0W1uOM QDLjJKWmww7cV6AeygLmDtz3d2i7wa4EJ4Q3+VjFxlV6om4PKk1/ar9mpsjHj23V4yek eKMGPxVSqN/emYFx1NNjvxUYipyFH+pndz2B0U58wPZ1iqFEQ3E8QxIuDVcWCf2ViHpl OO5w== X-Gm-Message-State: AOAM533NrNEuez8JSVc01oJj+Gn/gIBD2vWoBhHNPwc92CQ+5pBkYUq4 QPaMiSNAZAbRm1vt1ba8l4DLsXMLwTBn07V1vfs= X-Google-Smtp-Source: ABdhPJxXC3HSV6vz416ge4fqmJft40X9aZ8P6+6OA4iudRqIUfA/632/IKNoZXB90F8evpi0pnb8+mEsjeJadYO8a78= X-Received: by 2002:ad4:536a:: with SMTP id e10mr17659139qvv.246.1590967691074; Sun, 31 May 2020 16:28:11 -0700 (PDT) MIME-Version: 1.0 References: <87h7vyxqrz.fsf@gmail.com> In-Reply-To: From: zimoun Date: Mon, 1 Jun 2020 01:27:59 +0200 Message-ID: Subject: Re: Guix Docker image inflation To: Stephen Scheck Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::f30; envelope-from=zimon.toutoune@gmail.com; helo=mail-qv1-xf30.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: help-guix Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=IO33pTB+; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: 2yreTh+8QnUF Dear Stephen, On Sun, 31 May 2020 at 21:43, Stephen Scheck wrote: > On Sun, May 31, 2020 at 2:51 PM zimoun wrote: >> >> Maybe the explosion of size would be slower. If you do, please report >> here the number after say 12 generations; I am really interesting. ;-) > > Now I'm confused - in your reply to Vincent, it seemed that there were still problems > with the GC removing dead store items even after you did an export/re-import with the > entire image on a single Docker layer? Or did I misread it? The export/import trick cut by half the size of "your guix-bootstrap" image. So even I am not convince that it will fix the issue, I think that my proposal is the correct thing to do to delete dead items in the store. Basically, after the pull, you need to delete all the other generations of /root/.config/guix/current (expected one) by "guix pull -d", then to delete all the generations of /root/.guix-profile with "guix package -d" and last garbage collect. For sure, it will not delete the items coming from previous layer but it will delete all the dead items of the current session. And "docker export | docker import" could remove other items -- even if in the case of "install/remove hello" it is not work cleanly, some items are deleted. Well, it is just to be complete with your approach. >> All the question seems to be: >> - what is the purpose of such Docker image? Which usage? >> - what infrastructure do you have at hand to build the Docker images? > > Well, Guix is interesting, and there aren't ready-made containers for it out there like there are for > Ubuntu, Fedora, etc. if you have a need to do some task in that kind of environment, or just to play > around, or see how the system is evolving. Also, I have been playing around with Guile lately and > I thought Guix might be a better fit for that kind of work than other environments where Guile is > largely neglected (Guix is *written* in Guile, after all). And I happened to be learning GitLab CI/CD > around the same time, and it seemed like a good opportunity to experiment with both at once, > so I thought, why not? :-) Which infrastructure - well, GitLab CI/CD, with fixed compute limits :-) Yeah, "ready-made container" could be really cool! AFAIK, no one took the time to implement and document. There are various attempts but not always reported on help-guix or guix-devel. Well, the answer of these 2 questions implies different strategies. For example, I am running Guix on the top of Debian so I basically use only the package manager. And I use this infrastructure to produce Docker images containing apps running "guix pack -f docker -m manifest.scm". Because I am interested in Reproducible Science, I also use "guix time-machine -C channel.scm -- pack -f docker -m manifest.scm". However, the Docker images contains only applications (R or Python with bunch of packages) and the "user" cannot use these images to extend them running "guix install foo"; because I want to track reproducibility so the only way is to go by 'manifest.scm' and 'channel.scm' files. Another example is the Dockerfile way. Based on any image (Alpine or Debian), I build an image containing the Guix package manager -- roughly speaking as you are doing with your image 'guix-bootstrap'. Then I use this image in 2 different ways: with a Dockerfile or directly. In both cases, it always starts by "guix pull. And I never chain the images -- I mean only 3 "layers" at maximum: 0-debian, 1-guix-fresh and 2-guix-lastest. Well, I have never run "guix gc" inside a Docker image. Last, I have never played with "guix system docker-image". But in the context of GitlabCI, what I would try should be something like: CONTAINER=`docker run --detach --privileged $OLD` docker exec $CONTAINER guix pull docker exec $CONTAINER guix system docker-image --root=/image.tar stuff.scm docker cp $CONTAINER:$IMG $NEW with maybe instead of "guix pull" this bazooka: docker exec $CONTAINER guix install git docker exec $CONTAINER git clone docker exec $CONTAINER guix environment guix docker exec $CONTAINER ./bootstrap && ./configure --localstatedir=/var/ docker exec $CONTAINER make -j docker exec $CONTAINER ./pre-inst-env guix system docker-image Well, use "guix system docker-image" inside a Docker image already containing Guix; this avoid the layer issue, isn't it? But as I said elsewhere, I am not really familiar with Docker so my words are probable irrelevant. All the best, simon