From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id wIk6A/0PhWODOgAAbAwnHQ (envelope-from ) for ; Mon, 28 Nov 2022 20:46:05 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id AN5MAv0PhWMYjAAAG6o9tA (envelope-from ) for ; Mon, 28 Nov 2022 20:46:05 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id F1ACDD1CE for ; Mon, 28 Nov 2022 20:46:03 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ozk4f-0001zc-PA; Mon, 28 Nov 2022 14:45:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ozk4a-0001zK-9B for guix-science@gnu.org; Mon, 28 Nov 2022 14:45:51 -0500 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ozk4W-0003mJ-H6 for guix-science@gnu.org; Mon, 28 Nov 2022 14:45:47 -0500 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id D18053200937; Mon, 28 Nov 2022 14:45:35 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 28 Nov 2022 14:45:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; t=1669664735; x=1669751135; bh=v /vo7nLLHh2BwIoPOZeb7AgAuRXlAFbRVBq8ixKlkCk=; b=DhjMMoZkCgf23HJq5 ahZnY4qwHTzAqw4dWFSJeCW5bAAeLQ8Pqa3xMDBDBXBsa+NEUv8DlyRNWfkld67U B2tj+PeWOeMbAysmqPtYI7Z3BjcF0ibuinBIWV2xBvSOL4ScC86wvM27W8AzfeEm r2tTBK9P3uubM2umfMRkC1A3ZhnWVCM3vUamZbLlNRxy/bL1XsApRaf6LANjR2uL v6XtYFFbRMoy3pgoYAtmKkW3s9fXcOPAupdk7U2lUfYFCCxeNG1HZijRUg9rn13E g1XwsVchStXl6+1irrW8TquDPGfimZIXBmEFBERRLu+XbjFL77O+htAXUxAGqDAr 77jZw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrjedvgddufeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufgjfhffkfggtgfgsehtqhertddttdejnecuhfhrohhmpefmohhnrhgr ugcujfhinhhsvghnuceokhhonhhrrggurdhhihhnshgvnhestghnrhhsrdhfrheqnecugg ftrfgrthhtvghrnhepgefhtdeiuedvkeeuhefgveelueehieelvdfgieevhfegieduveeh gfeffeehueefnecuffhomhgrihhnpegtnhhrshdqohhrlhgvrghnshdrfhhrpdhorhgtih gurdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho mhepkhhonhhrrggurdhhihhnshgvnhestghnrhhsrdhfrh X-ME-Proxy: Feedback-ID: if40146ab:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 28 Nov 2022 14:45:31 -0500 (EST) From: Konrad Hinsen To: Thibault Lestang , guix-science Subject: Re: Conda environments and reproducibility In-Reply-To: <87pmd7ar8k.fsf@imperial.ac.uk> References: <87pmd7ar8k.fsf@imperial.ac.uk> Date: Mon, 28 Nov 2022 20:45:30 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: softfail client-ip=64.147.123.25; envelope-from=konrad.hinsen@cnrs.fr; helo=wout2-smtp.messagingengine.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-science-bounces+larch=yhetil.org@gnu.org Sender: guix-science-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1669664764; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=v/vo7nLLHh2BwIoPOZeb7AgAuRXlAFbRVBq8ixKlkCk=; b=tx9vCwc03+7UVmf0Sv1EeBRV9xkgV0Uz2PszETEadPhLAkvR0hszUI29ifQaGBjzMm2pzv 01+U5QdM0vuxriYQPg3fSXKJQ/v4kMw0cjhGM9yfgDjsDkP/8x1mGeYvUvmkYU2kYQGEe+ /KuHG9Rwztj8JyPKUYdGqywf2vZPCpJwo26lrW2oierXYjx6Bo/rNRC01cpCQ9nYXQCDoW tKPaTUnY2jZH3kvnLpdL9LDesQa1CJjiMlGh4YUtzifD/W4sb07oU81s/plbbkaau5TR9E Az8Bm4Ku0JXGts/xXqV3JMS0aLpzVDu/amxnvoGbMYZVaiNLj4zPFvwGataWRw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1669664764; a=rsa-sha256; cv=none; b=gPyw/5BmeSKSjpF8GtYusP0zmVBK6iJbTvGZSgoeIk7C0IHlDlrERtYZviEBJTCB4NSWD/ ypVvtjfgiy0XU1sO9Xl6E9x/JT2JcY1GTBgliiTi/AvFYPFGpehhW7rufm7qENMRUNC6cS 237ELCwpJQf+t2U8YUPNdw9zv1vLJY+bMgymCRP6zhRKNdyBz5sVMhCEREw4KbSSpzS8w1 HmSEtOrWlsATaEe34YeRpEv7JZ1UNr1WHc1WgVMfmhwQFQDY2a3kFjgwWOUHbC33xyoStw E/LntGk/vnox3n3qJ1X2dFgpsVuWD9caFDeVMO4zNNhlYFv5wnog3y+NzRlFpg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=DhjMMoZk; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-science-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-science-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 0.34 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=DhjMMoZk; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-science-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-science-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: F1ACDD1CE X-Spam-Score: 0.34 X-Migadu-Scanner: scn0.migadu.com X-TUID: gs573Q9+3E+F Hi Thibault, > ----- > @luispedrocoelho > Me, 6 months ago: I am going to save this conda > environment with all the versions of all the packages so it can be > recreated later; this is Reproducible Science! > > conda, today: these versions don't work together, lol. > ----- > > I simply can't explain how such a behavior can happen. The error message is not exactly as cited. Conda doesn't claim that these versions don't work together, it claims that it cannot find a combination of package known to work together and available in the archive. One possible reason for this is an update in conda's build infrastructure. That's what happened to the software environment for the reproducible research MOOC on Fun (of which I am an author). We published the environment file, but a few months later conda could not reconstruct it any more. They had updated the compiler infrastructure, which requires a rebuild of all packages. But they didn't rebuild all the versions from the past, so most older environment files became unusable. The lesson is that packages are reproducible only if you can re-run the construction of the entire environment, from source code. Which is what Guix can do (though if you actually have to do this, it will be a very long process). There may be other causes for the conda problem cited, I don't claim to be an authority of conda! After the MOOC experience, I have never used conda again. > Is it the case that someone at Anaconda would modify some package, > keeping the same version tag and other identifiers used by conda, whilst > at the same time marking this package as incompatible with packages it > was previously compatible with? That's in a way what happened in my scenario: rebuilding with a new compilation infrastructure produces different packages that share version numbers and tags with the prior ones. Cheers, Konrad. --=20 --------------------------------------------------------------------- Konrad Hinsen Centre de Biophysique Mol=C3=A9culaire, CNRS Orl=C3=A9ans Synchrotron Soleil - Division Exp=C3=A9riences Saint Aubin - BP 48 91192 Gif sur Yvette Cedex, France Tel. +33-1 69 35 97 15 E-Mail: konrad DOT hinsen AT cnrs DOT fr http://dirac.cnrs-orleans.fr/~hinsen/ ORCID: https://orcid.org/0000-0003-0330-9428 Twitter: @khinsen ---------------------------------------------------------------------