From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id sONDMsj6hGNuSgAAbAwnHQ (envelope-from ) for ; Mon, 28 Nov 2022 19:15:36 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id gM8eMsj6hGNnKQAAauVa8A (envelope-from ) for ; Mon, 28 Nov 2022 19:15:36 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3F95B195BF for ; Mon, 28 Nov 2022 19:15:36 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ozif2-0000g4-Gv; Mon, 28 Nov 2022 13:15:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oziWT-0007ZS-Od for guix-science@gnu.org; Mon, 28 Nov 2022 13:06:29 -0500 Received: from smtphub2.cc.ic.ac.uk ([2a0c:5bc0:88:101::47]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oziWP-0004AN-Rg for guix-science@gnu.org; Mon, 28 Nov 2022 13:06:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=imperial.ac.uk; s=main01; h=MIME-Version:Content-Type:Message-ID:Date: Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=3/am0xlR1ltnlgcqsH8paVoHBEuD56g2Q8B94pGNaus=; b=E31WOXwOrPpDX2/BYipxqJFZQN O0NE7CZCMcvSHxhczp2jHlbDyFcJMT3j9fSlIPEeg8YsSDDG34dorsDVZYIeAsPDDZn6+SuJ+ct41 U6+XDgT9/pcsRaAE8+D95KWBxpM9XunA3A+glZdIp6TZro/dNQV53qYJv81z5YrXbbe0=; Received: from mail-lo2gbr01lp2052.outbound.protection.outlook.com ([104.47.21.52] helo=GBR01-LO2-obe.outbound.protection.outlook.com) by smtphub2.cc.ic.ac.uk with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1oziWK-0007eu-Sw for guix-science@gnu.org; Mon, 28 Nov 2022 18:06:21 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HP7nLUc6rsOKzhqPrXG7zYCi3Cf9SqeN7ZEMBkkuGvrNeCwKvhooZtB3njZv6re78KEdMYUJ3e6GSc73CM/BCssQvNYOqyHky607MLxpcfzb49+sbzUEC86gavVUkZ5rElrc2yK7KwtmYbnjMTarnm9uomlKaW2B4fuhLd68hiicKqlgzBwP361biWs44MScLZPIlv/3zmPNbCikJc576xEKI5/xRG4UmDeo+k7MvMU20UjlcdiXFLanqjVnfntkbCWgbtnLaoXfwjPaYG1kr5z1KAnPCoTKvN1ELP2Ckj2oboX9tIZZ/1ZZqQYcoqf9JJNly2B+oSyfm94Axchqkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3/am0xlR1ltnlgcqsH8paVoHBEuD56g2Q8B94pGNaus=; b=VlUKdz14V/OFFqeh1AabaviJ5Qg4kvVa7vh/uE9jG2rwZgfXVvPm3ejcSByftCCqF6m3LeNSzFt4pB468sMGX+YbwNyQuY6QT896kKbkQoEyrFyrrMc2QSrGXSFIjb/GvDXOLy1ewtCHX23JgWwHjRc3s1uI11EXcAL8PoN2egGc/6CfKFQ/TVf8wF3nGCIXuM0yq81aTdBoOuARmUEQBk2y+JYmtbZROgaOesyfb2+6flQEBEk80uYnWgmgiuh0IqRpjTdGUnRgBeYUvin6nM5chb9ekM86jWMyueKfgI7Afc5koUX/DpmXr9EuDNeRul40j/RLpWlORbuYZwn3hQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=imperial.ac.uk; dmarc=pass action=none header.from=imperial.ac.uk; dkim=pass header.d=imperial.ac.uk; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ImperialLondon.onmicrosoft.com; s=selector2-ImperialLondon-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3/am0xlR1ltnlgcqsH8paVoHBEuD56g2Q8B94pGNaus=; b=goCqQzHJBbpwx/3oTJbFpqcjgeWEQcIPZCxZrZNO8yK2xJeldzKZEe+vjOfjQ2gwaOVZ9YeIIXL/jRuKl39349mOE4eI6kz3w7Z1dmI5ESlNfqNncjCLhLcFwGhlFUADw51B2OtF9YyH+e1mvHBQy5ImPpdbxpwuCBroNbkdDrA= Received: from LO2P265MB0590.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:6a::19) by LO4P265MB6432.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2d4::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Mon, 28 Nov 2022 18:06:19 +0000 Received: from LO2P265MB0590.GBRP265.PROD.OUTLOOK.COM ([fe80::ba52:925:b6c5:8a14]) by LO2P265MB0590.GBRP265.PROD.OUTLOOK.COM ([fe80::ba52:925:b6c5:8a14%6]) with mapi id 15.20.5857.023; Mon, 28 Nov 2022 18:06:19 +0000 User-agent: mu4e 1.6.10; emacs 28.1 From: Thibault Lestang To: guix-science Subject: Conda environments and reproducibility Date: Mon, 28 Nov 2022 17:28:48 +0000 Message-ID: <87pmd7ar8k.fsf@imperial.ac.uk> Content-Type: text/plain X-ClientProxiedBy: LO2P265CA0326.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a4::26) To LO2P265MB0590.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:6a::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO2P265MB0590:EE_|LO4P265MB6432:EE_ X-MS-Office365-Filtering-Correlation-Id: 84ea7924-18b5-47ef-bdb9-08dad16b4015 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qVvtkKopsI945aGvTiSfJhs0sP7n7WiyHb18sjWGwOolgKd3S4jl9ZKatQG7p0ECFYukqIbUvVvD8rRvE1wXZT3odhYG5cLrG4AnqCXgaQcHjsyX+b/gWdTWeEnTUfFb6+S+UGIJVXLJ2Ffj22/89opgrOv34FzptdRc3JDiXUtAr+nsLJ8kN8KHeHoWLYduihcl2/yCMd0TKi/OgTWCOVp/djjGfwYnNa8fsF+MfAlZakZMsX5HQB3iy5uBTF/dq7AkCHVHIRv/TOzcQAsGOycpMjkimk9hxUhoL63v6IQ49DtWdN3Tqt714Q6ZY2tV4CO6Qf6MWmUQLZFpyVbZOJEfjplRAftudFrf/ajWAVNvFb5FbU2k+OGXOEzqyGzo7RHuYFyH2yDQB9C8oMpArPNh5g41n837P9H21HzwyYZt1sTQgyr6tu45V4DbQIK38pg5BbLY/qoFE1LHhdRjx7OCmywYcNzR0uNzjEZpZ40M2FTjnBKsqP+JY7wzkjZC/W1ISTVgbx9eUeUdpzSDVxVNDEkP71roR8Yy4labxX6L17srfbOMaNkBH5vskE4eX5WQWlc7YIMJptllcR3LTG4CrTTaKYcpQKNLehlGM7baQRoeZp6sivA1Fa8imcVOYW1y9daMoLmhkseC+WBav5aGm5PJA6Wchey99XnND9k= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:LO2P265MB0590.GBRP265.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(136003)(376002)(396003)(366004)(346002)(451199015)(41300700001)(8936002)(2906002)(5660300002)(316002)(786003)(6916009)(6486002)(66946007)(8676002)(66556008)(83380400001)(41320700001)(966005)(66476007)(478600001)(36756003)(86362001)(6666004)(6506007)(52116002)(2616005)(186003)(3480700007)(6512007)(38100700002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6pQW4G0zsqa4yFiFZR6G1SgnwCevCP955Eq8ifceAoOfULYhdEd4gSNDK5m3?= =?us-ascii?Q?AGM8Pw0BZbouOictHZ1syAhXZMptF+s/NPq7KRN5Bv/skz9ZDOTCBYPCj9rf?= =?us-ascii?Q?rLNvD/u6JC7bV7uURqEpU/jl2/8RyMA3mpD1CV40m+s0eLTCJwzh0t1VfI/g?= =?us-ascii?Q?5yZBoEQKFKyuCiztQ65xID3PNCj63FPxN4Dnph2FotWB/eM19M/fsW0DCi7s?= =?us-ascii?Q?Os9A0+mMjC3NCs0kbydyQM4lZ6usZ2y+CkYVeJk6WVSAZX0MPFRglAwHgw80?= =?us-ascii?Q?y7t0ELp79hAFQW7rQ/ygclEkH5kRXjplRyQ63n6laMYkpVgp9ZzSTLPy/CwV?= =?us-ascii?Q?6NQA7vs5+jvfrgMbMz+dSrgGQSpdM8oQ2BK8edBdpmCeltffwWqf25kXFrNV?= =?us-ascii?Q?rxpFZXd758iBsrkNbFbq8j+GxJPKTmHB42sdL9j0okyse5zQ127GwAP1SoQT?= =?us-ascii?Q?DTjVED/5qEzUwtNdFJkTv7KUH3wIKF6R9YYTmulqb3pjlfNd5GtAt8dKIJD8?= =?us-ascii?Q?a2wuuZ8gbV41NgBFdWFYNbE2+YXT7ue7MGFOv2Fwh48M3rEBlnme+AdWnWDj?= =?us-ascii?Q?X3bp5lSZUyJ2OVG6rggf84zvZdwhm/0Vhirj7MRLyNxjiKq8E1TkUVqJCv0F?= =?us-ascii?Q?rNxPckn8nb1ncVH/yV/3RVXe8wiariefxco1gaeK7ZjDH9NxizLEBGEhLB9p?= =?us-ascii?Q?fSn3hPwd71D80iTOuwUcE9AeFjrED+jBI5MF7+RZ9gCrCp2iHWOpqqWXZVQ6?= =?us-ascii?Q?ftmX5DlhLD7d7FaA15mZlLGwjk2EGid7KOeHNu6sHNzDARrzrxADB7A6+yIH?= =?us-ascii?Q?k48qmy1uBaMGzASf6LhJNEycGBoYgwxjJAw09nP6mUD8MooKJbsDgLIE8nV2?= =?us-ascii?Q?BGtsdEqhxXaomRY4jiGEHEvYLKUDuTFskwaJozQDp+qaH5zXSCgalC8FX9SF?= =?us-ascii?Q?Nm6bFyFpHJ/h4iOmiNY17RAYuLjqeyYtFJLUdUgn7dFqAn9qv0xZGrWxaSl/?= =?us-ascii?Q?3kRcnOMqDQ3AESOSXkl70O1EQf8Z7ri4G+3wp7ZfK0DHZoQ6o5EG1Elckt/6?= =?us-ascii?Q?Z/mGscgY82W4Ep9CFU9fQn+4lL+eGB5nXcJ36qL9pECdSQ1vKTQLF11ajMiG?= =?us-ascii?Q?t/zWSjkbm5w7qlkau896i3HyA6wt3R30cBMtoqtDv/o2l5oyTG3CknpEeiCI?= =?us-ascii?Q?ABXy9c56C384G39abf28Of52O3AS/Bb7DRsiIhRTdEwg1F3b7JD23k7sy0N3?= =?us-ascii?Q?zh+gZkmy77CSBpZxsKhBFOnCNLQNQmJURjW7j8P9Qwna5YYLbFs6u8iu1NNk?= =?us-ascii?Q?D5Jb2t6zcZDvC6EUXyZdXZgmA0uR75EUIYaxizrC69sW3hORPw2/XXCeQayW?= =?us-ascii?Q?cq7RFo8JsMvJx/KokizAo/TP+baiWZwDZWzixRejCNIbNYDljyFb6OZ6G+sO?= =?us-ascii?Q?2doyoHf060jj6kHv8Dhl+Wkbrz0PKInLJkb6FrdHoFBVE02ds/1NEXvD0xyq?= =?us-ascii?Q?Fh42oQRJVZwuoQAvT1K/MSgQFn3HrqLxQbgkDvKdGra+cmg57b5aNrMUAbm9?= =?us-ascii?Q?pV3N6eT4AslHn8re/+qShJyG8rYPhYhvXgvENE0DpviFnp7IQEd0yEa/WKOS?= =?us-ascii?Q?B66cDa6R/0XcfLM60H28G7YxrAYA+ML6KVblLiHwatC1?= X-OriginatorOrg: imperial.ac.uk X-MS-Exchange-CrossTenant-Network-Message-Id: 84ea7924-18b5-47ef-bdb9-08dad16b4015 X-MS-Exchange-CrossTenant-AuthSource: LO2P265MB0590.GBRP265.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Nov 2022 18:06:19.6343 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 2b897507-ee8c-4575-830b-4f8267c3d307 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ufG0n/l0bzf3Rn0rXghSTEABmWORwzWniwFDtti8VLTcSYTquzi3thdwEPmpxhZz/BerhYxSCgMZx3/soL+xYQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO4P265MB6432 X-IC-MsgID: 1oziWK-0007eu-Sw Received-SPF: pass client-ip=2a0c:5bc0:88:101::47; envelope-from=t.lestang@imperial.ac.uk; helo=smtphub2.cc.ic.ac.uk X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 28 Nov 2022 13:15:14 -0500 X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-science-bounces+larch=yhetil.org@gnu.org Sender: guix-science-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1669659336; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=3/am0xlR1ltnlgcqsH8paVoHBEuD56g2Q8B94pGNaus=; b=aNM5v3VXFPig0RK1V4oDaIg/gWA2e9XaE8iG+5LPRRKxZDFUbz07l+A9ioqbOmLPnzCKDs lykYjcKABafKL+7W7spWzlz8mdwOaAEuhr7+I4Nn6Aajr/Y8InFmHDnQkSTaOJbWylDdnz Pj3EarKN1ob0yjjRjjwRxPUDUfN9jp963FrIN59jgNiBrlsmKTbLjNoIcnXE1ZodR1oSTZ UxN7mLHTXqM363iKesFJalb2YPtWS+GG45QUy4QjJlE73d1GIt47tPNeJC/NCl3TM1FSKH wLKYnFeWbntEDSoF97o4Evr3OI6Ru68UX4PKteJMfqRdNx7CmplWj6YpakY9Bw== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1669659336; a=rsa-sha256; cv=pass; b=mXABF6n3Lnzd3d7vNVxSIwU+Y2WSOLDmxIU6sWAwlRg2xS+t6GbwnvxdKvhmO2zIcP+k4/ vFGxA/sUtP8UJvMfQAwHfbbdnANY8eGqKoVLKCnSIVR6O6jm9X/R3vT8Nan4jFWaEQgTBz Ykq0aC5BM/pnZNHqsgJ1CPd9uINWdhcP5V9eSHZR1/LMc/GQx1PvW0neULkBYhZygEUYxs rjocFAIiC+dzftZG3meYZ5ht1qFIyDjOO4+Rfrm8lzlO5GhaWUKlBNPdYDiiulpH1r+0kV pPUWcmOOtLqnbhLJHlWXh9h2RI8q/yub+r01+mVJ0TpEsCLRZVWH9JFaRK12lg== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=imperial.ac.uk header.s=main01 header.b=E31WOXwO; dkim=pass header.d=ImperialLondon.onmicrosoft.com header.s=selector2-ImperialLondon-onmicrosoft-com header.b=goCqQzHJ; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=imperial.ac.uk (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-science-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-science-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -5.56 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=imperial.ac.uk header.s=main01 header.b=E31WOXwO; dkim=pass header.d=ImperialLondon.onmicrosoft.com header.s=selector2-ImperialLondon-onmicrosoft-com header.b=goCqQzHJ; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=imperial.ac.uk (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-science-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-science-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3F95B195BF X-Spam-Score: -5.56 X-Migadu-Scanner: scn0.migadu.com X-TUID: YHgT1cRix4hA Hi there, I'm new to the list so apologies if this has already been discussed before. I've been ruminating on reproducible builds ever since I attended the 10 years birthday event a few months ago. For me, putting /python/ and /reproduciblity/ together in the same sentence used to invariablity lead to /virtualenvs/ or /conda/ being featured in the next - suffice it to say that part of my world view was shattered a bit (that's okay). Things progressively start to make sense, but when talking about Guix with a colleague earlier today it became apparent that my understanding isn't exacly rock solid yet. Particularly, looking at this tweet https://twitter.com/luispedrocoelho/status/1087685131144495104 referred to in Ludovic's article "Toward reproducible Jupyter notebooks" (https://hpc.guix.info/blog/2019/10/towards-reproducible-jupyter-notebooks/). The tweet says (22 Jan 2019) ----- @luispedrocoelho Me, 6 months ago: I am going to save this conda environment with all the versions of all the packages so it can be recreated later; this is Reproducible Science! conda, today: these versions don't work together, lol. ----- I simply can't explain how such a behavior can happen. I understand that conda ships pre-compiled binaries. I see how that's bad for reproducibility and provenance tracking since it's not straightforward to know how these binaries and dependencies were compiled. I'm assuming that, when conda saves an environment, it records version tags and "everything else required" to pull the same binaries later. Okay - I see how binaries could /technically/ be modified at a later stage whilst maintaning the same version tag (provenance tracking issue). Is it the case that someone at Anaconda would modify some package, keeping the same version tag and other identifiers used by conda, whilst at the same time marking this package as incompatible with packages it was previously compatible with? Thanks for reading! Thibault -- Dr Thibault Lestang Senior Research Software Engineer Department of Aeronautics, Imperial College London