From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id ANZRMgm1C2JXOAEAgWs5BA (envelope-from ) for ; Tue, 15 Feb 2022 15:13:29 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id aIykLwm1C2LplgAA9RJhRA (envelope-from ) for ; Tue, 15 Feb 2022 15:13:29 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A369F30828 for ; Tue, 15 Feb 2022 15:12:53 +0100 (CET) Received: from localhost ([::1]:58172 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nJyZY-0000Ko-Pp for larch@yhetil.org; Tue, 15 Feb 2022 09:12:52 -0500 Received: from eggs.gnu.org ([209.51.188.92]:41536) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nJyXs-0006wZ-8w for guix-devel@gnu.org; Tue, 15 Feb 2022 09:11:08 -0500 Received: from imta-37.everyone.net ([216.200.145.37]:34390 helo=imta-38.everyone.net) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nJyXm-0004Yo-Sw; Tue, 15 Feb 2022 09:11:06 -0500 Received: from pps.filterd (localhost.localdomain [127.0.0.1]) by imta-38.everyone.net (8.16.0.43/8.16.0.43) with SMTP id 21FE26r8005074; Tue, 15 Feb 2022 06:10:42 -0800 X-Eon-Originating-Account: emrm3cz8aY1690qJ9avIhMu9AtQnCUDE2AA4EMARKL4 X-Eon-Dm: m0116953.ppops.net Received: by m0116953.mta.everyone.net (EON-AUTHRELAY2 - 53b92129) id m0116953.61e9ed53.28581f; Tue, 15 Feb 2022 06:10:41 -0800 X-Eon-Sig: AQMHrIJiC7RhVxUj8gIAAAAE,528c1a39fb3ce72aba2564a695d20fb0 X-Eip: R6qM5yax6cpl4b87DsivwtDrf2q8ibdKMQkk0CPOb3s Date: Tue, 15 Feb 2022 15:10:32 +0100 From: Bengt Richter To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: Investigating a reproducibility failure Message-ID: <20220215141031.GA13837@LionPure> References: <871r0l9fd1.fsf@gmail.com> <87tuddh06r.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <87tuddh06r.fsf@gnu.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Proofpoint-GUID: -iG43yPhMGS4ZbCabC0CBrbOdk3GyDaV X-Proofpoint-ORIG-GUID: -iG43yPhMGS4ZbCabC0CBrbOdk3GyDaV X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.816 definitions=2022-02-15_04:2022-02-14, 2022-02-15 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 clxscore=1034 mlxscore=0 impostorscore=0 priorityscore=1501 mlxlogscore=376 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202150082 Received-SPF: pass client-ip=216.200.145.37; envelope-from=bokr@oz.net; helo=imta-38.everyone.net X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Bengt Richter Cc: Guix Devel , zimoun Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1644934373; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=N9sEGgePPQaSvlwTRxNr7DIyIlJL0YiG39FMFUqA+eQ=; b=eLa14pAHeEEjysVlAWnzSErOQkM2M1hZBcFU/Ni50i9DCABkWa/SeMRkOpUKmTlUf3UDa7 okuvEAlwRo2POU/UlDyryoR88yX8HOxwmB7Ku1dOrRgXQhdE++7FF9wT9VKcqlHzqBXkED cEGiuqF1+yHcxIG8aXmtxFktW0o2DOTn5qOofmrRGBzqom3eYilwrHmncreUlpz6qyngZg XmACKwsp1PWpe2fzqavh0lk5mTbRlJbPSwPSL95M/n+cexZKJscxvf1EmzbvgqA5D798uw vozF+mQap89cMvOzPAsmYn9Q8r7V9vLcxYtUZO4GeSpqHikF+pPfkXORMzq1+A== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1644934373; a=rsa-sha256; cv=none; b=IfcyzHTcAOT7YYdHnzT1xE5nEDQWQZFZtQWijq0lrUMO8QD1NcCTdhVY/2+KDcQtKLNGsD dor9AoB53xN3AyzCFGdttnjjcUNr9Albh9XJLsh7EscfW1g6APsriSJeIJJ6qXQ9NbV2LG B/wR0vyQYEEHBW+Uqx+ksP9G4+2Ck3YEAli8wpwrKeMl1Igpmi0wzoPBm69fSJnDwRoKM6 yTDYaRiK9q75xiQDzOlIkao+ykCPtun/FkvbeKMusxLQJhZ60bBr4ZSZAvtgmekm+l4YCu nZ7yjO8y+WYgKolTnjo1heW3rGpSWpeGKlUHwSvtx3axIXxPvZi2cxMv3D21Rw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.63 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: A369F30828 X-Spam-Score: -2.63 X-Migadu-Scanner: scn1.migadu.com X-TUID: H0P42MUlrnYx Hi, On +2022-02-05 15:12:28 +0100, Ludovic Courtès wrote: > Konrad Hinsen skribis: > > > There is obviously a trade-off between reproducibility and performance > > here. > I suspect what you really want to reproduce is not verbatim code, but the abstract computation that it implements, typically a digitally simulated experiment? Thus far, "show me the code" is the usual way to ask someone what they did, and guix makes is possible to answer in great detail. But what is really relevant if you are helping a colleague reproduce e.g. a monte-carlo simulation experiment computing pi by throwing random darts at a square, to draw a graph showing convergence of statistically-computed pi on y-axis vs number of darts thrown on x-axis? (IIRC pi should be hits within inscribed circle / hits in 1x1 square) Well, ISTM you can reproduce this experiment in any language and method that does the abtract job. The details of Fortran version or Julia/Clang or guile pedigree only really come into play for forensics looking for where the abstract was implemented differently. E.g., if results were different, were the x and y random numbers displacing the darts within the square really uniform and independent, and seeded with constants to ensure bit-for-bit equivalent computations? How fast the computations happened is not relevant, though of course nice for getting work done :) > I tried hard to dispel that belief: you do not have to trade one for the other. > > Yes, in some cases scientific software might lack the engineering work > that allows for portable performance; but in those cases, there’s > ‘--tune’. > > https://hpc.guix.info/blog/2022/01/tuning-packages-for-a-cpu-micro-architecture/ > > We should keep repeating that message: reproducibility and performance > are not antithetic. And I really mean it, otherwise fellow HPC > practitioners will keep producing unverifiable results on the grounds > that they cannot possibly compromise on performance! > Maybe the above pi computation could be a start on some kind of abstract model validation test? It's simple, but it pulls on a lot of simulation tool chains. WDYT? > Thanks, > Ludo’. > -- Regards, Bengt Richter