From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id 0OsZNExtbGdPEQEAe85BDQ:P1 (envelope-from ) for ; Wed, 25 Dec 2024 20:38:37 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id 0OsZNExtbGdPEQEAe85BDQ (envelope-from ) for ; Wed, 25 Dec 2024 21:38:36 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers eddsa verify failed") header.d=josefsson.org header.s=ed2303 header.b=RK7O3aT0; dkim=fail ("headers rsa verify failed") header.d=josefsson.org header.s=rsa2303 header.b=gBtPOSQP; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1735159115; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature:openpgp:openpgp; bh=B5SQyq7uY/EFWoAzZhOtHr/Yy26uwPyyyp5o3p2WAtc=; b=NnCMJHUXV1cVa8a38+0XsqfJEmfx8omviYTnqUo5yeeh9AsWj9ptBYvJFOqyNIaRsOw0Ro a51HFu0F3eGbCxfkL5Mnn8/ITNttD1H1klxKjcCfTg2qLqmWrkHkdG+mmjAEKLzl31TySi aAKsCqIrDAkXkM18URUTkgduAmfnr/8VbncgnGzqE+jG0A+j23Iatt1G2bg5Nb7MvApNDa X8HKDjKXqWWRMPur76WRgWZqoQJUnYUcV0R98Ac3Kz7KLXXxSK33jFWLChsyy8IAwOO4Da 8GmYtDd3cxBvhD6IhEQkpX8juDCJU+d0kn3oJq7VOvkanydYIjukDE8LKijqrQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers eddsa verify failed") header.d=josefsson.org header.s=ed2303 header.b=RK7O3aT0; dkim=fail ("headers rsa verify failed") header.d=josefsson.org header.s=rsa2303 header.b=gBtPOSQP; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Seal: i=1; s=key1; d=yhetil.org; t=1735159115; a=rsa-sha256; cv=none; b=UBb5c5HE6qX46vcJRnFK1pMSDKMxLYPBNXBoxnX0zeEpyx7mp7jnnMuT24q52pT2o+I/u/ 2BawGzhTDYFNDfZPpQxipLm32gWEm6JcHAnMj8RCe8moREWArW4f6kaV9+06lZSilBfVBt Yacl+gR0c78E/kuHb4StA4Q2Ih5fv7NRYql7yd+WLcSs92hSb+fsvo4cbO9u3U82hHmJ5y Vl4p+o6hrQwzaNN7pANwlOcxe6Z9hpyRtoe5zGEJx6KmLtHZsHKxklPRwNF7ZHr4fSgtxV t1LVUuOTbmRHR/h3n0W9EOCKy54uC+RIRm1Hp7UqpvLsIRIHwmWUUQft2O6F7A== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6B9FA8464A for ; Wed, 25 Dec 2024 21:38:34 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tQY91-0004mK-JA; Wed, 25 Dec 2024 15:38:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tQY8y-0004ls-9X for help-guix@gnu.org; Wed, 25 Dec 2024 15:38:12 -0500 Received: from uggla.sjd.se ([2001:9b1:8633::107]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tQY8v-0004Tv-JN for help-guix@gnu.org; Wed, 25 Dec 2024 15:38:12 -0500 DKIM-Signature: v=1; a=ed25519-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=ed2303; h=Content-Type:MIME-Version:Message-ID:In-Reply-To :Date:References:Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding :Content-ID:Content-Description; bh=B5SQyq7uY/EFWoAzZhOtHr/Yy26uwPyyyp5o3p2WAtc=; t=1735159080; x=1736368680; b=RK7O3aT0S3VPQGrwHdCIiMDrPeUPHFgYRu4FT11jJZi1LSoX/iGsaWlOTNEiK6TCEr24+YnqFOA V7naHIOjQBA==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=rsa2303; h=Content-Type:MIME-Version:Message-ID: In-Reply-To:Date:References:Subject:To:From:Sender:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description; bh=B5SQyq7uY/EFWoAzZhOtHr/Yy26uwPyyyp5o3p2WAtc=; t=1735159080; x=1736368680; b=gBtPOSQPP0gru98Z26Wyxh+T05uc3VxDtEfsxQqy/tCyC6P91Fs1mRy40NPddV62O2wZ9rHDzIO DD19y6RL7/v0UEOiYXrdoXgz09+V7lsWpGTrcf9kyfwkvRUoMOtz6cwcsnkIWqYpzNztW8LyXdqPp DFexLE1mEnBFQ49smovjA5wQFHD19blz5B5ZhW4Zz5sdzWIHNHLRgPHSUqNFJxdBM2aC1WUrwn6Gv 0/kjqQjH8QIvPQ5rUlDPt4Hy2/+l4OKS3hmzSrQEeW4+bf2vLChLLgEX68WKCc3pIWeCqavByrOEQ p8S6EDMS4ry9nWR8ojiSb+okq7Sv1f9XKXtcEIn8pBYEwrbs7NS/vuEV4dd1oQL/jCwuyznVan6xF tqGfNtvv5IzqJy5X9VqkI2QeVQZjP7GAwd/xDX+xF3wwXTAAersWr52aMCwUGymKwmCdZJJ57; Received: from h-178-174-130-130.a498.priv.bahnhof.se ([178.174.130.130]:34862 helo=kaka) by uggla.sjd.se with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1tQY8j-007mod-Dp for help-guix@gnu.org; Wed, 25 Dec 2024 20:37:57 +0000 To: help-guix@gnu.org Subject: Re: Building a Docker image for GitLab-CI References: <87ttb4d5c8.fsf@inventati.org> <87a5cwd4bn.fsf@inventati.org> <87ed27oqn9.fsf@kaka.sjd.se> <87zfkurbja.fsf@inria.fr> <87zfkulolb.fsf@kaka.sjd.se> <87o71ar4j5.fsf@inria.fr> <87ed25lvol.fsf@kaka.sjd.se> <87h66xgie9.fsf@inria.fr> <87ldw7pp6d.fsf@kaka.sjd.se> OpenPGP: id=B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE; url=https://josefsson.org/key-20190320.txt X-Hashcash: 1:23:241225:help-guix@gnu.org::NJa5GF3o/ELsZO9v:2vMy X-Hashcash: 1:23:241225:ludo@gnu.org::6Bxj6MAGA6kUmRIU:LWjj X-Hashcash: 1:23:241225:suhail@bayesians.ca::pvWByMDBHfWJDLP3:QI8q X-Hashcash: 1:23:241225:csantosb@inventati.org::RCQhjo0AyM1RCf6Q:Ewg2 Date: Wed, 25 Dec 2024 21:38:14 +0100 In-Reply-To: <87ldw7pp6d.fsf@kaka.sjd.se> (Simon Josefsson via's message of "Sun, 22 Dec 2024 19:07:06 +0100") Message-ID: <87cyhfo5vt.fsf@kaka.sjd.se> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=2001:9b1:8633::107; envelope-from=simon@josefsson.org; helo=uggla.sjd.se X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Simon Josefsson From: Simon Josefsson via Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: help-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Queue-Id: 6B9FA8464A X-Migadu-Scanner: mx13.migadu.com X-Migadu-Spam-Score: -6.00 X-Spam-Score: -6.00 X-TUID: y+BfWpesq3F0 --=-=-= Content-Type: text/plain All, Here are some updates about Guix container images for GitLab pipelines or local podman usage. I'm declaring this v1.0. tl;dr: https://gitlab.com/debdistutils/guix/container Final images are built from a pure Guix container now. Everything is done on public shared GitLab runners in the pipeline, no container uploads. Stage0 creates Debian+Guix that builds a pure Guix stage1 which builds the final Stage2 images. The content of these images appears to be reproducible, but alas the docker images itself aren't: https://issues.guix.gnu.org/75090 No need for --disable-chroot in GitLab CI runs. Local podman usage ironically requires 'podman --privileged' if you want to avoid --disable-chroot. If someone can figure out which --cap-add are sufficient, that would be nicer over --privileged. Ultimately I think 'guix-daemon' should handle this, it is a desirable property to be able to use chroot building inside a container. I'm using small/medium GitLab runners. It seems whatever 'guix' is consuming resources for, it isn't helped by additional CPU nodes, disk, or RAM. Network bandwidth is improved by using guix from GitLab instead of Savannah. Maybe the bottleneck are the substitution servers? Or perhaps single-core CPU speed? For stage1 [1], 1m52s is spent on 'guix install skopeo' and 2m44s on 'guix pack'. For stage2 [2], 1m35s is spent on 'guix install nss-certs skopeo tar gzip' and 4m30s on 'guix pack'. Creating the stage0 debian+guix image is where the 'guix pull' happens [3], and it takes around 35 minutes (I recall seeing runtimes down to 25 minutes when I used larger nodes). The 'latest' image with gcc, automake etc as a development environment is around 400MB and the 'slim' image with minimal packages only is 183MB. Does anyone how to optimize 'guix pack' output sizes? Even the 'slim' image seems to have a lot of duplicated stuff [4]. There is a bunch of small nits, and if someone has ideas about improvements that would be great! See list of issues here: https://gitlab.com/debdistutils/guix/container#known-quirks Merry Christmas, /Simon [1] https://gitlab.com/debdistutils/guix/container/-/jobs/8723179887 [2] https://gitlab.com/debdistutils/guix/container/-/jobs/8723179903 [3] https://gitlab.com/debdistutils/guix/container/-/jobs/8723242065 [4] 'guix pack guix bash-minimal coreutils-minimal net-base' and doing cd /gnu/store; ls|sort -k1.33: gd3s60nav0qhp8lxjj21ffynivwibfl5-avahi-0.8 3jhfhxdf6v5ms10x5zmnl166dh3yhbr1-bash-minimal-5.1.16 x47i4yafqxdav838aykda9c2hhhn9sa4-bash-minimal-5.1.16 87z5k84hxbqs87plgwsl2v6a4j7m3k7h-bash-static-5.1.16 56aq6sdx35f7rsxq8jq9ypafk0dhd3p3-bzip2-1.0.8 59kd6jyvrq8prl9mbnh3g8d22rc1dbwv-bzip2-1.0.8 qy1769103d15zh8gg09wlywfsyblham4-coreutils-minimal-9.1 vdaspmq10c3zmqhp38lfqy812w6r4xg3-curl-8.6.0 af6rfyb76j51g2m981a4r0747pvg3j7c-dbus-1.15.8 dnjwcdxmwma6fl7fvvn3p4frib7f5chl-disarchive-0.6.0 vb1rs3dk181ariczl0zqcmfjncjkrv0f-emacs-subdirs faxgciaw9wxz8zyxk70f2pa3c5rr8al7-expat-2.5.0 zzpbp6rr43smwxzvzd4qd317z5j7qblj-gcc-11.4.0-lib hdb3jmxa67zkh4wj0l6w9ga3gj84k1yc-gdbm-1.23 9ri7c2haj2q3f5p6859z64kjvrjyy5n6-git-minimal-2.46.0 zgsphhmliwgmjjv1czmbyjql3gk7ynsx-glib-2.78.0 zvlp3n8iwa1svxmwv4q22pv1pb1c9pjq-glibc-2.39 pxnrbpc30m5qsr8jqx86a9m42mzn25ni-glibc-utf8-locales-2.39 kka705681m1hq98b9jz98vxk9s5qd4ld-gmp-6.3.0 9mkcil1rl450r84hn1hcbny5pi5js8ig-gnutls-3.8.3 7k8b93779dqpwcg2qjdvnf4nl43jv7hf-grep-3.11 mfkz7fvlfpv3ppwbkv0imb19nrf95akf-guile-3.0.9 003k1369b9b35b7vgfzjqrc1iha555i2-guile-avahi-0.4.1 1myi8hwa0a3lf9qw14dkqckhv9ljpzp1-guile-bytestructures-1.0.10 rf9xg52fa4zpn9ywd9w4kczhib4ggfsq-guile-bzip2-0.1.0 2bmrqh4w9pcgns0pi3wwqasrshpmv8hw-guile-gcrypt-0.4.0 kcvbb34cv4p19sg3rmi2rrld03wyvhpb-guile-git-0.9.0 pgjyl3fn4sflk6xy63qd5anrhqwylpgw-guile-gnutls-4.0.0 711y2zrpg0ygxaghy72v8hzwla7mjaqg-guile-json-4.7.3 p7qx1yhxlz61r1hpcgdvdhqq343cryyp-guile-lib-0.2.8 02i9pa0yj18riq7g90bzx0jaxmlxnax4-guile-lzlib-0.3.0 n2jz9qnxf7ainkzsdjyl3d4x078g15lw-guile-lzma-0.1.1 nj1051ag55p7llr1wc0ml6hg08gk1prs-guile-semver-0.1.1 yhzifwp225x81i9d056xa2r11g5w40kd-guile-sqlite3-0.1.3 vhby2mrlf25flwx571bmnllccigb49ml-guile-ssh-0.18.0 7h0khqsyzz3ic8dwyfmbbr5404qkmm98-guile-zlib-0.2.1 i0fm4jrkgz6rxpcscd1sazx62fwhqd58-guile-zstd-0.1.1 pzghsxxfx5dll69ikhckissq3b38542z-guix-1.4.0-29.3032221 0r2fx1lr1h2i3cl1x5fw4s4ly95qspya-gzip-1.13 w9zl48a95kylc7a91rwrrk27v70my968-gzip-1.13 96lahq0x84fiaj341vzx0fw5h18iyq9q-http-parser-2.9.4-1.ec8b5ee prf6y8cmysfdf6jys86ixcv1kdw4l2lf-info-dir 9vjs14mzxki1q857wc8jfhbfj06gvkcp-libcap-2.64 62xxxmgmpk6zhzdr1ciya6f572y75xkw-libdaemon-0.14 lqgg509yb3f85ck4k6l0qp7a70bz7daa-libevent-2.1.12 s6iqwc5sqjrk76kzslqc1n1wlcvfyqkw-libffi-3.4.4 pr73chdirm3jc2j7npc6hqzmcwjs7l8m-libgc-8.2.4 gfqifdfnfvnbksbm0w87fvq76138i8da-libgcrypt-1.10.1 ni0kk5ff3z8sdglksb3850c9w44a2zaj-libgit2-1.8.3 881qgylidmmx92jdv1wvkzjs858dw9cd-libgpg-error-1.47 7xizylh3gi6sj23nz19q6xhvx2d50wvr-libidn2-2.3.4 jcjm231n2g8mqs0w2pa85hv7l1nfi2qa-libpsl-0.21.1 085636515w3h03dp2fr7w3clsn3p2wj7-libssh-0.10.6 pr8xfc53m3fc6rx8jrfis1xz8jvbb53h-libssh2-1.10.0 b801mrqqcsnhbr34544mlfyanzg3skfx-libtasn1-4.19.0 zpaw3cp2k9jx36yhkpwra3jilfbb1mc7-libunistring-1.1 4775wjc2972kiwfsq710fv5pfzyc5laq-libx11-1.8.7 wxwv020jwxq9gr070vwy3fh8n028gwqg-libxau-1.0.10 y5a0l9a3z214yar8q7mznqqd4pnw0vvp-libxcb-1.15 q1vqb2hfclghbpl1vn094l1rzj12b6qb-libxcrypt-4.4.36 v712yc2mwkc10m1nzgjz3linnvl5i1dh-libxdmcp-1.1.3 40aa02d5xnxpi2w6dhlr4ldf1kir1wz2-lzlib-1.13 b9kfblvwd0xx5jr8zzvz4ypa0936jh6v-mit-krb5-1.20 7rsdf5kcqh0gl88av6nkgvgxg1ywvc5b-ncurses-6.2.20210619 bfp25w47fxn8z0fdwj45prx2609sx59j-net-base-5.3 al613p11xv5w1xmnqn7ykw0x6d4b0539-nettle-3.9.1 8i2kr43jfbqvhpv67hs8kgncj2kk19b6-nghttp2-1.58.0-lib xc98v8v485rs704wb26mipb0y5npdl1z-openssl-3.0.8 cmzi8a17f44fvb55s77jd7d4r678w093-p11-kit-0.24.1 gwn3p1r5ghlapv9yjad0mk2n23la7j8z-pcre2-10.42 a3lsdsalcmg5wnk67869af7wljprkbam-pkg-config-0.29.2 bwfrm3dmm33lfr69r1h5jy24hj51ii23-profile dl3665ynrp41ynyw2ay5kfqix93myj5d-readline-8.1.2 81wqxjgqfinrxxh473c89r1n7arxfv3s-sed-4.8 laj6a3z6gjza9f18kyxw1nz5211ghwfs-sqlite-3.39.3 j5zgzgsmbjgywr67r86h1n6s4qiabv5q-tar-1.34 2p8j6npwa2k59d8lbhlqzvffn0437x8l-util-linux-2.37.4-lib 70s4sq1hx1m5rmsg5bcnjxslwc8ppiag-xz-5.4.5 fbaw0sb21gv02qq7gs9wg5y5wlpdgzih-xz-5.4.5 1prv14v6jfnzzg7szm57690b7fr6sx33-zlib-1.3 m05g4pzw906bg2pydbl74vrnvkmi9rbj-zstd-1.5.2-lib --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZ2xtNhQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFosXrAP47NPL9o99IulyKtroRbl3SfyuFxlU+ 9sb21BkvVVk5NAEAnfrr/qR8dCjqym5kqMtfvOSBJak1oaOs0LI0ro9vZAA= =CS74 -----END PGP SIGNATURE----- --=-=-=--