From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Subject: bug#29654: GDBM output is not deterministic Date: Sun, 17 Dec 2017 01:26:17 +0100 Message-ID: <87d13eci8m.fsf_-_@gnu.org> References: <87mv2jfzmi.fsf@gnu.org> <87po7fo6gv.fsf@elephly.net> <87wp1mcjty.fsf@gnu.org> <87shcacj8p.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:52217) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eQMnJ-0004Gs-Tm for bug-guix@gnu.org; Sat, 16 Dec 2017 19:27:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eQMnG-0007VO-OM for bug-guix@gnu.org; Sat, 16 Dec 2017 19:27:05 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:57417) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eQMnG-0007Uy-Kl for bug-guix@gnu.org; Sat, 16 Dec 2017 19:27:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1eQMnG-00082K-9P for bug-guix@gnu.org; Sat, 16 Dec 2017 19:27:02 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87shcacj8p.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sun, 17 Dec 2017 01:04:38 +0100") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: bug-gdbm@gnu.org Cc: Ruud van Asseldonk , 29654@debbugs.gnu.org Hello Sergey & co., While investigating , we realized that databases produced by GDBM are not bit-for-bit reproducible. This can be seen with this program: --8<---------------cut here---------------start------------->8--- #include #include int main () { unlink ("/tmp/t.db"); gdbm_open ("/tmp/t.db", 0, GDBM_WRCREAT, 0644, NULL); } --8<---------------cut here---------------end--------------->8--- Under Valgrind, we get: --8<---------------cut here---------------start------------->8--- =3D=3D15981=3D=3D Syscall param write(buf) points to uninitialised byte(s) =3D=3D15981=3D=3D at 0x5335080: __write_nocancel (in /gnu/store/3h31zsqx= jjg52da5gp3qmhkh4x8klhah-glibc-2.25/lib/libc-2.25.so) =3D=3D15981=3D=3D by 0x4E3E44D: _gdbm_full_write (in /gnu/store/kg8ffb14= msfnc9aivxj6djrl51g9b3zz-gdbm-1.13/lib/libgdbm.so.4.0.0) =3D=3D15981=3D=3D by 0x4E3B6AD: gdbm_fd_open (in /gnu/store/kg8ffb14msfn= c9aivxj6djrl51g9b3zz-gdbm-1.13/lib/libgdbm.so.4.0.0) =3D=3D15981=3D=3D by 0x400743: main (in /home/ludo/src/guix/a.out) =3D=3D15981=3D=3D Address 0x55fb204 is 4 bytes inside a block of size 4,09= 6 alloc'd =3D=3D15981=3D=3D at 0x4C2AAD6: malloc (in /gnu/store/p2b1rzqlpdqbhn42g7= 6xzgykbivwc063-valgrind-3.12.0/lib/valgrind/vgpreload_memcheck-amd64-linux.= so) =3D=3D15981=3D=3D by 0x4E3B5E6: gdbm_fd_open (in /gnu/store/kg8ffb14msfn= c9aivxj6djrl51g9b3zz-gdbm-1.13/lib/libgdbm.so.4.0.0) =3D=3D15981=3D=3D by 0x400743: main (in /home/ludo/src/guix/a.out) --8<---------------cut here---------------end--------------->8--- =E2=80=A6 which suggests that an uninitialized block is being written strai= ghtly to disk. If we tell libc to initialize =E2=80=98malloc=E2=80=99 areas that were othe= rwise left uninitialized, we can confirm the problem (again with the program above): --8<---------------cut here---------------start------------->8--- $ for i in `seq 0 20` ; do MALLOC_PERTURB_=3D$i ./a.out ; sha256sum /tmp/t.= db ; done 0483a04a43d0a08a5a0aa71d09b262b62eb2504e85c5fe7104b783a5f0f7ae15 /tmp/t.db 29122b2994face5b5a716c8cd0e6beda7f05c3441441447872c2f15e7e2db308 /tmp/t.db 34a130820ab6d5e638adae55e83cb784fec036c2da4727a113d3d1191331bd7c /tmp/t.db 9604ce76ea1f1b37968bd037798c0eb212a1e654f060b7c3d66011620b249212 /tmp/t.db a1c275c1bf1b5980166431bd7245d2d649499a8cad233059abf2a6cbf2dec19b /tmp/t.db 1cbe43f146a158ac4c21064feac1371e7a6bb4ac9d0013b15acfe7fec754486f /tmp/t.db 83778759475790f1377adadd77b9d2eee7db7e5e43e006ef04e805f4b4309590 /tmp/t.db 6b80d00119fd806d05f8863056919ea0a6b66a0771d16439b072e2cc2eac6b58 /tmp/t.db f1f65b95e39f74cff12c3e830eea77a9841c8793f4441c6a8bfa9ad92d765d8d /tmp/t.db 3dd2479f44c0be15e02f0040fcef3d765a306dbd2a47b3557cb19fb1282b0b05 /tmp/t.db fed4cc15d84f676737954e80ab594034e47d2f2b7cbcd12dad722dddff6c5afb /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db 3d9bccf127fd76b56617435c412aeec0c8fc95ab6b9d5f9e3f24060398899c26 /tmp/t.db 743808bd91a42e2c149a30bbdfacadbaa4124f82e01ea43323d5c1fd10de5499 /tmp/t.db 86a92606713ca03cab09f1cc97463184ed0c4d2051d7512dd98b136bc7ca1465 /tmp/t.db 93f5428a4fde073cab1737c1efd64e2554731db1ab51d77aec95c81592b014a0 /tmp/t.db b93fd6692eeebfc5b49b64857de7647a88e541be7d4fd4e62fb6771cb4b00618 /tmp/t.db ba08b6acee5812c3b3770f842aca7435729acc1082cddf8bfe26024f9ea8f54e /tmp/t.db 30dc2e3d1ea0150e4a200435b1f609de726680ebfc8661a9d121afbd4cf0b0a8 /tmp/t.db 0de2cc18e445ff0a2697cd2783f379fa202d57020db9a73c55c4f7b8fd5cac74 /tmp/t.db dd938784d3799c5a575abc3fe24191f3e1ac593643719a9694f9fed32ed272da /tmp/t.db $ for i in `seq 0 20` ; do MALLOC_PERTURB_=3D11 ./a.out ; sha256sum /tmp/t.= db ; done a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db a43fb60e3d94093c11bd7bb2cf5253ccb5c3e772eca82ba9c9aac64204e15bfc /tmp/t.db --8<---------------cut here---------------end--------------->8--- I guess the fix is to use =E2=80=98calloc=E2=80=99 instead of =E2=80=98mall= oc=E2=80=99 for this specific allocation. WDYT? Ludo=E2=80=99.