From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id uFmFOyr8EmBcEAAA0tVLHw (envelope-from ) for ; Thu, 28 Jan 2021 18:02:18 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id GKg6Nyr8EmBEZwAAbx9fmQ (envelope-from ) for ; Thu, 28 Jan 2021 18:02:18 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 848A4940481 for ; Thu, 28 Jan 2021 18:02:18 +0000 (UTC) Received: from localhost ([::1]:36744 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l5BcX-0007P1-FK for larch@yhetil.org; Thu, 28 Jan 2021 13:02:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41778) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l5BYJ-0004iY-JA for guix-devel@gnu.org; Thu, 28 Jan 2021 12:57:55 -0500 Received: from imta-35.everyone.net ([216.200.145.35]:33410 helo=imta-38.everyone.net) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l5BYB-0005T7-AK; Thu, 28 Jan 2021 12:57:52 -0500 Received: from pps.filterd (omta001.sj2.proofpoint.com [127.0.0.1]) by imta-38.everyone.net (8.16.0.43/8.16.0.43) with SMTP id 10SHqSeI032512; Thu, 28 Jan 2021 09:57:38 -0800 X-Eon-Originating-Account: sZmo-_lEo0LYIuDnHC4zhDaDuN00SeUdHNK1KAl8zsI X-Eon-Dm: m0117124.ppops.net Received: by m0117124.mta.everyone.net (EON-AUTHRELAY2 - 5a81c1f0) id m0117124.6000aa74.26677f; Thu, 28 Jan 2021 09:57:36 -0800 X-Eon-Sig: AQMHrIJgEvsQjdJm9gIAAAAE,2478aea8f351adb52dbca534d6a0a273 X-Eip: Qyzh6u-AtIgBhEMWTGHCMBdWTebWEk7dnVOexLnLoOs Date: Thu, 28 Jan 2021 18:57:26 +0100 From: Bengt Richter To: zimoun Subject: Re: [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile Message-ID: <20210128175726.GA6094@LionPure> References: <20210116182957.31075-1-ludo@gnu.org> <87wnvyv7w6.fsf@gnu.org> <86bldahz42.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <86bldahz42.fsf@gmail.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-28_12:2021-01-28, 2021-01-28 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 phishscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 clxscore=1034 adultscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101280086 Received-SPF: pass client-ip=216.200.145.35; envelope-from=bokr@oz.net; helo=imta-38.everyone.net X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Bengt Richter Cc: guix-devel , 45919@debbugs.gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.85 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 848A4940481 X-Spam-Score: -1.85 X-Migadu-Scanner: scn1.migadu.com X-TUID: nObDtavok+r6 Hi zimoun, Thank you for your tutorial-style and resource-reference-footnoted posts! I don't know about others, but I save many on my hard disk as reference examples. I hope this provides positive feedback for your efforts. Thank you. On +2021-01-27 22:02:53 +0100, zimoun wrote: > Hi Ludo, > > On Wed, 27 Jan 2021 at 14:14, Ludovic Courtès wrote: > > Ludovic Courtès skribis: > > > >> Here’s a simple but oft-requested feature (I remember discussing > >> with Pierre and Simon a year ago at the Guix Days about ways to > >> implement it—time flies!): these patches provide ‘guix package > >> --export-manifest’ and ‘--export-channels’. These options spit > >> a manifest and a channel spec, respectively, with the goal of > >> helping users who wish to migrate to the declarative model. > > > > I’m rather happy with this patch set but since this is something we’ve > > discussed several times in the past, I think it’d be great if those > > interested could chime in and comment: > > > > https://issues.guix.gnu.org/45919 > > > > I’ll leave a few more days and then… push! > > You are looking at me? :-) > > Well, the feature is nice! At first, I thought that it could be a bit > smarter than using only one commit. But as you said, the aim is for > transitioning. Maybe a future improvement should to list somewhere in > the comments which commit provides which set of packages. It could > help… or not. :-) > > Using Docker, it works. Let describe how in case people are interested. > > > On machine A, I have: > > --8<---------------cut here---------------start------------->8--- > $ guix describe -f channels > (list (channel > (name 'guix) > (url "https://git.savannah.gnu.org/git/guix.git") > (commit > "cb68ae668af2ade4b0777d82f227e5462768e9e5") > (introduction > (make-channel-introduction > "9edb3f66fd807b096b48283debdcddccfea34bad" > (openpgp-fingerprint > "BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA"))))) > > $ cat ~/.config/guix/manifests/python.scm > (specifications->manifest > (append > '("python" > ) > (map > (lambda (pkg) > (string-append "python-" pkg)) > '("ipython" > "numpy" > "matplotlib" > "scipy" > "biopython" > )))) > --8<---------------cut here---------------end--------------->8--- > > And I generate a Docker pack with: > > --8<---------------cut here---------------start------------->8--- > $ guix pack -f docker --save-provenance -m ~/.config/guix/manifests/python.scm > /gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz > --8<---------------cut here---------------end--------------->8--- > > > Then on machine B, after fetching this tarball, I run: > > --8<---------------cut here---------------start------------->8--- > $ docker load < /tmp/img/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz > $ docker images > REPOSITORY TAG IMAGE ID CREATED SIZE > python-python-ipython-python-numpy latest 49ddfedf1e27 51 years ago 1.45GB > --8<---------------cut here---------------end--------------->8--- > > And it works as expected: > > --8<---------------cut here---------------start------------->8--- > $ docker run -ti python-python-ipython-python-numpy:latest python3 > Python 3.8.2 (default, Jan 1 1970, 00:00:01) > [GCC 7.5.0] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import this > The Zen of Python, by Tim Peters > > Beautiful is better than ugly. > Explicit is better than implicit. > Simple is better than complex. > Complex is better than complicated. > Flat is better than nested. > Sparse is better than dense. > Readability counts. > Special cases aren't special enough to break the rules. > Although practicality beats purity. > Errors should never pass silently. > Unless explicitly silenced. > In the face of ambiguity, refuse the temptation to guess. > There should be one-- and preferably only one --obvious way to do it. > Although that way may not be obvious at first unless you're Dutch. > Now is better than never. > Although never is often better than *right* now. > If the implementation is hard to explain, it's a bad idea. > If the implementation is easy to explain, it may be a good idea. > Namespaces are one honking great idea -- let's do more of those! > >>> import numpy as np > >>> A = np.array([[1,0,1],[0,1,0],[0,0,1]]) > >>> _, s, _ = np.linalg.svd(A); s; abs(s[0] - 1./s[2]) > array([1.61803399, 1. , 0.61803399]) > 0.0 > >>> > --8<---------------cut here---------------end--------------->8--- > > Neat! > > So far, so good. Well, let extract the ’manifest’ from this Docker > blob. > > --8<---------------cut here---------------start------------->8--- > $ docker export -o /tmp/img/re-pack.tar $(docker ps -a --format "{{.ID}}" | head -n1) > $ tar -xf /tmp/img/re-pack.tar $(tar -tf /tmp/img/re-pack.tar | grep 'profile/manifest') > $ cat gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile/manifest | grep -E "(\(\"python|cb68ae)" | head -n5 > (("python" > "cb68ae668af2ade4b0777d82f227e5462768e9e5") > ("python-ipython" > (("python-backcall" > ("python-pyzmq" > --8<---------------cut here---------------end--------------->8--- > > Now, a trick to get the channels and specifications: > > --8<---------------cut here---------------start------------->8--- > $ ./pre-inst-env guix package -p /tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-channels > ;; This channel file can be passed to 'guix pull -C' or to > ;; 'guix time-machine -C' to obtain the Guix revision that was > ;; used to populate this profile. > > (list > (channel > (name 'guix) > (url "https://git.savannah.gnu.org/git/guix.git") > (commit > "cb68ae668af2ade4b0777d82f227e5462768e9e5") > (introduction > (make-channel-introduction > "9edb3f66fd807b096b48283debdcddccfea34bad" > (openpgp-fingerprint > "BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA")))) > ) > > $ ./pre-inst-env guix package -p /tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-manifest > $ ./pre-inst-env guix package -p /tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-manifest > ;; This "manifest" file can be passed to 'guix package -m' to reproduce > ;; the content of your profile. This is "symbolic": it only specifies > ;; package names. To reproduce the exact same profile, you also need to > ;; capture the channels being used, as returned by "guix describe". > ;; See the "Replicating Guix" section in the manual. > > (specifications->manifest > (list "python" > "python-ipython" > "python-numpy" > "python-matplotlib" > "python-scipy" > "python-biopython")) > --8<---------------cut here---------------end--------------->8--- > > Awesome! > > > The unexpected is this channels and manifests files do not reproduce the > same Docker pack tarball: > > --8<---------------cut here---------------start------------->8--- > $ guix describe > Generation 99 Jan 05 2021 16:56:39 (current) > guix-past 829923f > repository URL: https://gitlab.inria.fr/guix-hpc/guix-past > branch: master > commit: 829923f01f894f1e687735627025ada26230832f > guix-bimsb a8b539d > repository URL: https://github.com/BIMSBbioinfo/guix-bimsb > branch: master > commit: a8b539d61a359060c35f3cb34c7edd1d9d14241d > bimsb-nonfree 4084e63 > repository URL: https://github.com/BIMSBbioinfo/guix-bimsb-nonfree.git > branch: master > commit: 4084e63c9c0d662780870aded9f5a6ca1b063780 > guix-science cf87b05 > repository URL: https://github.com/guix-science/guix-science.git > branch: master > commit: cf87b0501c4a38b96edf41025a27bf1cb91f521a > guix 957f0c4 > repository URL: https://git.savannah.gnu.org/git/guix.git > branch: master > commit: 957f0c40327ce00f53db22737e3775ce616ac258 > > $ guix time-machine -C /tmp/img/channels.scm -- pack -f docker --save-provenance -m /tmp/img/manifest.scm > Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'... > /gnu/store/xzk604g8gysv4azn7sf9nylr6iah97gl-docker-pack.tar.gz > --8<---------------cut here---------------end--------------->8--- > > To compare with > /gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz. > > On a third machine, I get: > /gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz > > Well, that’s another story and I have not inspected yet the > derivations and what could be wrong on the machine B. > > > Cheers, > simon > KUTGW ;-) -- Regards, Bengt Richter