From mboxrd@z Thu Jan 1 00:00:00 1970 From: zimoun Subject: bug#21803: guitarix non-reproducible hard to reproduce Date: Mon, 18 Nov 2019 19:18:54 +0100 Message-ID: References: <87sgmropu2.fsf@gnu.org> <87v9rmmah9.fsf@gnu.org> <87y2wfx0kd.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:37551) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iWld5-0000Wg-O1 for bug-guix@gnu.org; Mon, 18 Nov 2019 13:20:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iWld4-0006xi-CV for bug-guix@gnu.org; Mon, 18 Nov 2019 13:20:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:37394) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iWld4-0006xe-9E for bug-guix@gnu.org; Mon, 18 Nov 2019 13:20:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1iWld4-0008OH-3t for bug-guix@gnu.org; Mon, 18 Nov 2019 13:20:02 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87y2wfx0kd.fsf@gnu.org> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 21803@debbugs.gnu.org Hi Ludo, On Sat, 16 Nov 2019 at 17:12, Ludovic Court=C3=A8s wrote: > zimoun skribis: > > > My intuition is: the non-determinism comes from the WAF configuration > > files (./waf or wscript or wafadmin/). > > > > The process conditionally enters in functions such as "do_install" or > > "symlink_as", so maybe something is hidden by Python module "os" or > > similar. Moreover, note that wscript:l.990 "add_group()" is used after > > "add_subdirs('libgxw/gxw')". > > My intuition :-) is that waf traverses files using directly > opendir/readdir, which returns files in an order that=E2=80=99s file > system-dependent. That, in turn, leads it to make .so a symlink or not > in a non-deterministic fashion. Yes, it should come from the function 'os.listdir' (readdir). The documentation [1] says: "The list is in arbitrary order." [1] https://docs.python.org/2/library/os.html#os.listdir The function 'os.listdir' is called some times ;-) --8<---------------cut here---------------end--------------->8--- $ egrep -r "os\.listdir\(" wafadmin3/py3kfixes.py: for x in os.listdir(os.path.join(dir,y))= : wafadmin3/Scripting.py: lst=3Dos.listdir(cwd) wafadmin3/Scripting.py: if WSCRIPT_FILE in os.listdir(calldir): wafadmin3/Scripting.py: dirlst=3Dos.listdir(cwd) wafadmin3/Scripting.py: names=3Dos.listdir(src) wafadmin3/Scripting.py: lst=3Dos.listdir('.') wafadmin3/Tools/qt4.py: lst=3Dos.listdir('/usr/local/Trolltech/'= ) wafadmin3/Tools/javaw.py: lst=3Dos.listdir(path) wafadmin3/Utils.py: return os.listdir(s) tools/check_rpc: for f in os.listdir(basedir): src/gx_head/builder/make:for f in os.listdir(__dir__): wafadmin/py3kfixes.py: for x in os.listdir(os.path.join(dir,y)): wafadmin/Scripting.py: lst=3Dos.listdir(cwd) wafadmin/Scripting.py: if WSCRIPT_FILE in os.listdir(calldir): wafadmin/Scripting.py: dirlst=3Dos.listdir(cwd) wafadmin/Scripting.py: names=3Dos.listdir(src) wafadmin/Scripting.py: lst=3Dos.listdir('.') wafadmin/Tools/qt4.py: lst=3Dos.listdir('/usr/local/Trolltech/') wafadmin/Tools/javaw.py: lst=3Dos.listdir(path) wafadmin/Utils.py: return os.listdir(s) --8<---------------cut here---------------end--------------->8--- And do not talk about the function 'walk' which internally use 'listdir'. := -) I have no idea how to address this issue. > It would also be worth checking what others involved in the Reproducible > Builds effort have done (Debian, openSuSE, etc.). Lot of sun for Debian [2] ;-) I mean from what I understand, they do not find any reproducibility issue and they apply only this patch [3]. [2] https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/guit= arix.html [3] https://sources.debian.org/patches/guitarix/0.36.1-1/02-libdl.patch/ Do not know about openSuSE. Cheers, simon