From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark H Weaver Subject: bug#30820: Chunked store references in compiled code break grafting (again) Date: Mon, 19 Mar 2018 20:52:37 -0400 Message-ID: <87in9rbmay.fsf@netris.org> References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> <87370vg0e9.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:48786) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ey5XS-0005uT-DZ for bug-guix@gnu.org; Mon, 19 Mar 2018 20:54:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ey5XO-00007p-8O for bug-guix@gnu.org; Mon, 19 Mar 2018 20:54:06 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:34511) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ey5XO-00007g-4S for bug-guix@gnu.org; Mon, 19 Mar 2018 20:54:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87370vg0e9.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 19 Mar 2018 23:34:38 +0100") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 30820@debbugs.gnu.org ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Mark H Weaver skribis: > >> ludo@gnu.org (Ludovic Court=C3=A8s) writes: >> >>> The recently added glibc grafts triggered issues that, in the end, show >>> the return of (=E2=80=9CStore references in= 8-byte >>> chunks in compiled code=E2=80=9D). >> >> I think that we should generalize our reference scanning and grafting >> code to support store references broken into pieces, as long as each >> piece containing part of the hash is at least 8 bytes long. >> >> Here's my preliminary proposal: >> >> (1) The reference scanner should recognize any 8-byte substring of a >> hash as a valid reference to that hash. >> >> (2) To enable reliable grafting of chunked references, we should impose >> the following new restrictions: (a) the store prefix must be at >> least 6 bytes, (b) grafting can change only the hash, not the >> readable part of the store name, and (c) the readable part of the >> store name must be at least 6 bytes. >> >> (3) The grafter should recognize and replace any 8-byte subsequence of >> the absolute store file name. > > I=E2=80=99m quite reluctant because it would add complexity, it will prob= ably > slow things down, and yet it may not handle all the cases, as Danny > suggests. > > Mind you, the GCC patches are not perfect either, but they=E2=80=99re rel= atively > easy to deal with (well, so far at least). In theory we would need > similar patches for LLVM and maybe a couple other native compilers, > though, which is obviously a downside, although we haven=E2=80=99t had any > problems so far. We would also need to find a solution to the problem described in the thread "broken references in jar manifests" on guix-devel started by Ricardo, which still has not found a satifactory solution. https://lists.gnu.org/archive/html/guix-devel/2018-03/msg00006.html My opinion is that I consider Guix's current expectations for how software must store its data on disk to be far too onerous, in cases where that data might include a store reference. I don't see sufficient justification for imposing such an onerous requirement on the software in Guix. Ultimately, I would prefer to see the scanning and grafting operations completely generalized, so that in general each package can specify how to scan and graft that particular package, making use of libraries in (guix build ...) to cover the usual cases. In most cases, that code would be within build-systems. Mark