From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id COCyD3Kyxl9XZQAA0tVLHw (envelope-from ) for ; Tue, 01 Dec 2020 21:15:30 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id SKpwC3Kyxl9INgAAbx9fmQ (envelope-from ) for ; Tue, 01 Dec 2020 21:15:30 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 92ADA9404C9 for ; Tue, 1 Dec 2020 21:15:29 +0000 (UTC) Received: from localhost ([::1]:34354 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kkCzg-0001F9-JY for larch@yhetil.org; Tue, 01 Dec 2020 16:15:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33552) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kkCNB-0005Q9-Eo for help-guix@gnu.org; Tue, 01 Dec 2020 15:35:41 -0500 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]:44366) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kkCN9-0004UG-89 for help-guix@gnu.org; Tue, 01 Dec 2020 15:35:41 -0500 Received: by mail-pg1-x530.google.com with SMTP id t3so1895824pgi.11 for ; Tue, 01 Dec 2020 12:35:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:in-reply-to:references:date:message-id:mime-version :content-transfer-encoding; bh=pLkP8ZsuYkCSbEd76Vi7iFAjMn4Fo6B53A2JMEScTrU=; b=aZIHAmAw0K7Dyu/VM3y33TFsrbxqPbIZd4ndslMLvcM+FihkYbIMzBJWX/QuivxAXn zsgn8izWEUOHIuAoY528wGJIo1lrZWTTa1kaawf5YomWqLbgo6ozwAaCoxfFC3Vwm8Mp UM9iLqHz9cA7LODyYXHNg1wlBhgt7eeS27YM7C5qQ3aPKWaOsyZ9WIBSLgPNGLPSx9lj 6udOGN4ganlC7tMvzyvjxyV9xm2p9azGIHrLXxvFXLfVtOTX2lCfnGoYvDDm/5onZ1H0 Lc2KCWJt1FPV5Fwfakkpkhhocxo8AHUdX/jg2D8bFqkcsEm6OSxjG6p3FBwL87IdWHVa 5xqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=pLkP8ZsuYkCSbEd76Vi7iFAjMn4Fo6B53A2JMEScTrU=; b=lQ1qapCTOi7IKhIWEGG5FfEgm3LMXUnefAVgderI57fdH+R0xQ/AfkwejAVHr6U0LG /g0gEx2Ea9vQT5kaStLiz6cTEWkKS1TkhuulUcsxD7BE1dyDXeahEp+GnJOKM8eWRoXE 3nuAWWOC309EZHj6D0++mhfv6wflCBCOLapyOnU9fykDqYX4xEVSjzfOAOmG+BR+jJ1i qivPN7ghRWqf3LQn0H0HMkmZFa50RlfxGDTysiyO98sCO0lUq0KAPczZdpCx7dzaSWIF 50FA6dxQ0Q3lHhSTGK5YPBZfInwqtN2MX0VK5O9JLDp6PqZ4+LrBUNXTuuMLtTaHfoxq 5pCg== X-Gm-Message-State: AOAM533cgLCC1NO3A67QWid6kKOHZ2xxxo4Xv3EpT4Gab/b4hfc+JCS+ tpEPwcERVoG/tAlw7mQtPAt5N4ObnZ4= X-Google-Smtp-Source: ABdhPJwP7LBZuxm5aAM5PIR7VNSJVnj00sl0x3efGfRMa1BoL6mYa3hOt0EZKb95k+SBGjEHLeUavA== X-Received: by 2002:a65:5501:: with SMTP id f1mr3900032pgr.210.1606854937642; Tue, 01 Dec 2020 12:35:37 -0800 (PST) Received: from branjam-ASM100 ([2601:601:1980:1a40:14e7:5c85:5ec7:6f5]) by smtp.gmail.com with ESMTPSA id l8sm536378pjq.22.2020.12.01.12.35.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Dec 2020 12:35:37 -0800 (PST) From: Brandon Ellington To: zimoun , help-guix@gnu.org Subject: Re: Reproducing a Python project environment (using guix inferiors) In-Reply-To: <86r1o9vj7a.fsf@gmail.com> References: <87czzw6hid.fsf@branjam-ASM100.i-did-not-set--mail-host-address--so-tickle-me> <86sg8rvwgr.fsf@gmail.com> <875z5m5v0q.fsf@branjam-ASM100.i-did-not-set--mail-host-address--so-tickle-me> <86r1o9vj7a.fsf@gmail.com> Date: Tue, 01 Dec 2020 12:35:36 -0800 Message-ID: <87360p5ko7.fsf@branjam-ASM100.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=branjam4@gmail.com; helo=mail-pg1-x530.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Tue, 01 Dec 2020 16:13:14 -0500 X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.49 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=aZIHAmAw; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Queue-Id: 92ADA9404C9 X-Spam-Score: -1.49 X-Migadu-Scanner: ns3122888.ip-94-23-21.eu X-TUID: L4NMApdHOsWP Thanks again for the reply simon! zimoun writes: > Hi, > > On Mon, 30 Nov 2020 at 14:39, Brandon Ellington wrot= e: > >>>> | package of interest | guix commit | status | >>>> |---------------------+--------------+--------| >>>> | python-matplotlib | "7e06086522" | bad | >>>> | python-pandas | ce2cfcabfc | bad | >>>> | python-networkx | 269f100330 | good | >>>> | python-numpy | 4d6ed794dd | bad | >>>> | python-scipy | 02ddafef55 | good | > > Which version of do you need? You can kind of get it from [2] in my initial email, but here is a cleaner table: | package of interest | version (rec. minimum) | |---------------------+------------------------| | python | 3.6.5 | do not rebuild |---------------------+------------------------| | python-matplotlib | 2.0.2 | used to exist in guix | python-pandas | 0.22.0 | | python-networkx | 2.1 | | python-numpy | 1.13.3 | | python-scipy | 1.0.1 | |---------------------+------------------------| | python-pygraphviz | 1.3 | bump from current guix | python-nltk | 3.2.5 | |---------------------+------------------------| | python-wikipedia | 1.4.0 | importing | python-toposort | 1.5 | | python-community | 0.10 | | python-louvain | 0.14 | |---------------------+------------------------| | (author's own) | (not applicable) | > Because, I think that =E2=80=9Cbad=E2=80=9D commits 7e06086522 (Jan 2018)= , ce2cfcabfc > (Feb 2018) and 4d6ed794dd (Jan 2018) simply pre-date the introduction of > =C2=ABInferior=C2=BB. Therefore, they are not reachable by this mechanis= m. > That=E2=80=99s one of the motivations for the channel =E2=80=9Cguix-past= =E2=80=9D. :-) >=20 > [...] >=20 > For commits before ~July 2018, the strategy is the one of =E2=80=98guix-p= ast=E2=80=99 > channel; which I roughly described above. > > 1: https://guix.gnu.org/en/blog/2018/multi-dimensional-transactions-and-r= ollbacks-oh-my/ Ah, lesson learned! I'll direct my attention toward guix-past then :) > For commits after the complete introduction of Inferiors mechanism > (~July 2018, please read [1]), it is ok. Be aware that each inferior > has a cost and if your package foo depends on X packages coming from > inferiors, then it means you are running under the hood X time: =E2=80=98= guix > pull=E2=80=99 then =E2=80=99guix build =E2=80=99 which could = require to also build > other dependencies, therefore installing foo could be long. Indeed I noticed this. Building the two good channels was not too bad, but I have not tried building networkx or scipy yet. As for your hypothetical... > Explicitly, imagine that python-foo depends on python, python-bar and > python-baz. Imagine that you use 2 inferiors, one for python-bar and > one for python-baz. Now imagine that python-bar and python-baz depends > both on the package python. Then, there is a high probability to build > 3 times tiny variants of the package python. And maybe the 3 packages > python-foo, python-bar and python-baz perfectly works with the same > variant of python. And it is worse because that applies to all the > implicit dependencies (compilers and so on). ...I can imagine it, and that would be terrible yes. I thought python wouldn't be a dependency of python packages in part for that reason though. Or maybe I read this for something else like emacs and extrapolated to python. Maybe I'm dreaming :P. In any case, the overall idea makes sense, especially the implicit dependencies thing. For me, if I went for full reproducibility including the older version of python, then all these packages would work on 3.6.5, and I'd still have a lot to learn about the implicit dependencies. > What I will do is: a) fix a Guix version b) create a channel containing > all the necessary variants (backporting dependencies if required, i.e., > copy/pasting old package definition and fix them if they does not build > anymore) to build my short list of . > > This is more or less the strategy used to feed the channel =E2=80=9Cguix-= past=E2=80=9C. > > And I will not use the inferior mechanism because it adds a lot of > complexity and will not solve your problem since old packages are not > reachable and/or your need to add old dependencies. Got it. The last part of the last sentence is surprising (because the mechanism must have been the right choice to someone/for something) but everything else adds up. > The time-machine is orthogonal with the way to distribute, IMHO. It is > simply a easy CLI to fix the Guix version. > > Usually, I do: > > guix describe -f channels > channels.scm > > edit my-manifest.scm # containing >=20=20=20 > [.. hack my custom variants ..] > [.. backport variant dependencies when I need for these variants ..] > guix time-machine -C channels.scm > -- build -L /path/to/variants/ -m my-manifest.scm > [.. loop hack until it works ..] > > and then I create what my audience expects, e.g., Docker image: > > guix time-machine -C channels.scm > -- pack -f docker -L /path/to/variants/ -m my-manifest.scm > > or relocatable tarball (pack -RR) or =E2=80=98system docker-image=E2=80= =99 or whatever. > > > The /path/to/variants is a Git repo and I add the files channels.scm and > manifest.scm. And now, it is easy to rebuild everything in the future. Okay, definitely worth a try! > Now, these questions and the other ones about =E2=80=99guix-past=E2=80=99= are answered, > right? Yes, I appreciate you helping me out here! Now I can go down the correct rabbit hole to build my project :) --Brandon