bug#21803: guitarix non-reproducible hard to reproduce

[zimoun <zimon.toutoune@gmail.com>]

Hi Ludo,

On Sat, 16 Nov 2019 at 17:12, Ludovic Courtès <ludo@gnu.org> wrote:

> zimoun <zimon.toutoune@gmail.com> skribis:
>
> > My intuition is: the non-determinism comes from the WAF configuration
> > files (./waf or wscript or wafadmin/).
> >
> > The process conditionally enters in functions such as "do_install" or
> > "symlink_as", so maybe something is hidden by Python module "os" or
> > similar. Moreover, note that wscript:l.990 "add_group()" is used after
> > "add_subdirs('libgxw/gxw')".
>
> My intuition :-) is that waf traverses files using directly
> opendir/readdir, which returns files in an order that’s file
> system-dependent.  That, in turn, leads it to make .so a symlink or not
> in a non-deterministic fashion.

Yes, it should come from the function 'os.listdir' (readdir). The
documentation [1] says: "The list is in arbitrary order."

[1] https://docs.python.org/2/library/os.html#os.listdir


The function 'os.listdir' is called some times ;-)

--8<---------------cut here---------------end--------------->8---
$ egrep -r "os\.listdir\("
wafadmin3/py3kfixes.py:            for x in os.listdir(os.path.join(dir,y)):
wafadmin3/Scripting.py:    lst=os.listdir(cwd)
wafadmin3/Scripting.py:        if WSCRIPT_FILE in os.listdir(calldir):
wafadmin3/Scripting.py:        dirlst=os.listdir(cwd)
wafadmin3/Scripting.py:    names=os.listdir(src)
wafadmin3/Scripting.py:    lst=os.listdir('.')
wafadmin3/Tools/qt4.py:            lst=os.listdir('/usr/local/Trolltech/')
wafadmin3/Tools/javaw.py:        lst=os.listdir(path)
wafadmin3/Utils.py:        return os.listdir(s)
tools/check_rpc:    for f in os.listdir(basedir):
src/gx_head/builder/make:for f in os.listdir(__dir__):
wafadmin/py3kfixes.py:            for x in os.listdir(os.path.join(dir,y)):
wafadmin/Scripting.py:    lst=os.listdir(cwd)
wafadmin/Scripting.py:        if WSCRIPT_FILE in os.listdir(calldir):
wafadmin/Scripting.py:        dirlst=os.listdir(cwd)
wafadmin/Scripting.py:    names=os.listdir(src)
wafadmin/Scripting.py:    lst=os.listdir('.')
wafadmin/Tools/qt4.py:            lst=os.listdir('/usr/local/Trolltech/')
wafadmin/Tools/javaw.py:        lst=os.listdir(path)
wafadmin/Utils.py:        return os.listdir(s)
--8<---------------cut here---------------end--------------->8---

And do not talk about the function 'walk' which internally use 'listdir'. :-)


I have no idea how to address this issue.


> It would also be worth checking what others involved in the Reproducible
> Builds effort have done (Debian, openSuSE, etc.).

Lot of sun for Debian [2] ;-)
I mean from what I understand, they do not find any reproducibility
issue and they apply only this patch [3].

[2] https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/guitarix.html
[3] https://sources.debian.org/patches/guitarix/0.36.1-1/02-libdl.patch/


Do not know about openSuSE.


Cheers,
simon