From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?B?QmrDtnJuIEjDtmZsaW5n?= Subject: Re: Treating tests as special case Date: Thu, 5 Apr 2018 14:19:47 +0200 Message-ID: <20180405141947.6dfeb927@alma-ubu> References: <20180405052439.GA30291@thebird.nl> <20180405082115.60e604a6@alma-ubu> <87fu4aj6xu.fsf@elephly.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/CM_.5MuY61/.LWyNQ=u2+XC"; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:49178) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f43ru-0007wg-NM for guix-devel@gnu.org; Thu, 05 Apr 2018 08:19:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f43rr-0005NW-GG for guix-devel@gnu.org; Thu, 05 Apr 2018 08:19:54 -0400 Received: from m4s11.vlinux.de ([83.151.27.109]:51794 helo=bjoernhoefling.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f43rr-0005Ml-7n for guix-devel@gnu.org; Thu, 05 Apr 2018 08:19:51 -0400 In-Reply-To: <87fu4aj6xu.fsf@elephly.net> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus Cc: guix-devel@gnu.org --Sig_/CM_.5MuY61/.LWyNQ=u2+XC Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Thu, 05 Apr 2018 12:14:53 +0200 Ricardo Wurmus wrote: > Bj=C3=B6rn H=C3=B6fling writes: >=20 > > And you mentioned different environment conditions like machine and > > kernel. We still have "only" 70-90% reproducibility. =20 >=20 > Where does that number come from? In my tests for a non-trivial set > of bioinfo pipelines I got to 97.7% reproducibility (or 95.2% if you > include very minor problems) for 355 direct inputs. >=20 > I rebuilt on three different machines. I have no own numbers but checked Ludivic's blog post from October 2017: https://www.gnu.org/software/guix/blog/2017/reproducible-builds-a-status-up= date/ "We=E2=80=99re somewhere between 78% and 91%=E2=80=94not as good as Debian = yet, [..]". So if your numbers are valid for the whole repository, that is good news and would mean we are now better than Debian [1], and that would be worth a new blog post. Bj=C3=B6rn [1] https://isdebianreproducibleyet.com/ --Sig_/CM_.5MuY61/.LWyNQ=u2+XC Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlrGFGQACgkQvyhstlk+X/0aLQCfdYYm4oeaMxDVR/i5tKBFCSLi uAgAn3l/mZ6mlBL+qShWMBErzUdCNVGz =1QCj -----END PGP SIGNATURE----- --Sig_/CM_.5MuY61/.LWyNQ=u2+XC--