From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id +FItKBju1WGiHgAAgWs5BA (envelope-from ) for ; Wed, 05 Jan 2022 20:14:32 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id EPvPJBju1WHKXgAAauVa8A (envelope-from ) for ; Wed, 05 Jan 2022 20:14:32 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 25C762E832 for ; Wed, 5 Jan 2022 20:14:32 +0100 (CET) Received: from localhost ([::1]:51398 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n5Bjz-0001hz-7N for larch@yhetil.org; Wed, 05 Jan 2022 14:14:31 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n5Bdl-000513-IA for guix-patches@gnu.org; Wed, 05 Jan 2022 14:08:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58513) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n5Bdi-0001ah-Tr for guix-patches@gnu.org; Wed, 05 Jan 2022 14:08:05 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1n5Bdi-0002sc-Jd for guix-patches@gnu.org; Wed, 05 Jan 2022 14:08:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52940] [PATCH] gremlin: Mimic ld.so NEEDED deduplication behavior. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 05 Jan 2022 19:08:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52940 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Chris Marusich Cc: 52940@debbugs.gnu.org Received: via spool by 52940-submit@debbugs.gnu.org id=B52940.164140964610995 (code B ref 52940); Wed, 05 Jan 2022 19:08:02 +0000 Received: (at 52940) by debbugs.gnu.org; 5 Jan 2022 19:07:26 +0000 Received: from localhost ([127.0.0.1]:41823 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n5Bd7-0002rH-QH for submit@debbugs.gnu.org; Wed, 05 Jan 2022 14:07:26 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:42106) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n5Bd5-0002r0-08 for 52940@debbugs.gnu.org; Wed, 05 Jan 2022 14:07:24 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 6C1461B7; Wed, 5 Jan 2022 20:07:16 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 24007vY1uIAk; Wed, 5 Jan 2022 20:07:15 +0100 (CET) Received: from ribbon (91-160-117-201.subs.proxad.net [91.160.117.201]) by hera.aquilenet.fr (Postfix) with ESMTPSA id DDEC8112; Wed, 5 Jan 2022 20:07:14 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87zgofaw2g.fsf@gmail.com> Date: Wed, 05 Jan 2022 20:07:14 +0100 In-Reply-To: <87zgofaw2g.fsf@gmail.com> (Chris Marusich's message of "Sat, 01 Jan 2022 15:13:43 -0800") Message-ID: <87a6gaknml.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / X-Rspamd-Server: hera X-Rspamd-Queue-Id: 6C1461B7 X-Spamd-Result: default: False [-0.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_ENVRCPT(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1641410072; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=nQ+fKLd5bNNVdYEM7JqWA4Ic1vq4VTarZhaAIEkkfRg=; b=Ra5QbAuFwj63D9HuMC6kEQz2bboJZX76IJSz9qezeR/yj+ZQrFLs4EwmVOrD8vgxn8vWT0 pRZ9hdswqaNA3lHPmxQd4WDvYvAPjHCyf7wJzey3zGf1xd6FQ3sHk8Tv/0PFdHynCUpEAf Q1jZgidIwT1GaUvF5jFwnMYoIkzdDnTHVt+RlsmcSFVF9digmObsqtP8vlPYWi5NduS5Sv X4ZbuDEDn3dyZ5BAVp1/q2vk4zuwev/ZcfrKzQPUGmQggU/4dsNaeDAPNyBZgelZYzDfQ/ plaN+T6aRCV8tulan/b1KKAEGjeaURHkP5C7T9ZPpozd0JvxwsGgIae24i7SMQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1641410072; a=rsa-sha256; cv=none; b=N8mUlld0Tg0wuz6GeXXRSfXRcWDL3aUjfryf07GZKIcPFwfd/mY8AjtO8B62QcS821jrGR JgCaBxs6HhDVBL4ZfM1EFahxDnVU7l7q5TNfhYpYGWiF/mmIrYQxc3gc7rvK9KAdcWhJDl ef4Y12f9yQ44TlLLwMKGLUmsKtlBdUw5h3S3dTPhh+oA03xul2gQazgnatJubu8SGATAIQ xuUsPSgEVm+B5/Iv5y7YIvSBYXu80+5Jn1vvCGUwQgSl35yjGTcM4UlVgX4PZ0LciwY4ho fYfVj8nLdjEi14hQTap77BdqCMq5/IJFBLJ7CCgldm9ZneHrz+xmlhNviWMPLg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.10 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 25C762E832 X-Spam-Score: -2.10 X-Migadu-Scanner: scn1.migadu.com X-TUID: PVRoAR6r8IWQ Hi Chris, Chris Marusich skribis: > Here is the failing test output, including the test source: > > ;;; (truth ("linux-vdso64.so.1" "/gnu/store/gahs2sx5snbfkr9vlcjj5c2kvnlhr= 0zs-guile-3.0.7/lib/libguile-3.0.so.1" "/gnu/store/7x2cjqbmpgwrgmnb234gsxkm= sqs5pj09-libgc-8.0.4/lib/libgc.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybj= a1dvqb-glibc-2.31/lib/libpthread.so.0" "/gnu/store/521riv2sgv0b0s4j0kzz6i52= rf9rarh8-libffi-3.3/lib/../lib/libffi.so.7" "/gnu/store/xj20v8lk2wal0z1rla0= yx3bjkasbx6mq-libunistring-0.9.10/lib/libunistring.so.2" "/gnu/store/sipyfs= 2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libcrypt.so.1" "/gnu/store/sipyfs= 2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libdl.so.2" "/gnu/store/sipyfs254= 0b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libm.so.6" "/gnu/store/ys7b4gr5nbq8s= fnff9ry5blb4bhpx6mq-gcc-7.5.0-lib/lib/libgcc_s.so.1" "/gnu/store/sipyfs2540= b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libc.so.6" "/gnu/store/sipyfs2540b48b= 2sb9j8ypmybja1dvqb-glibc-2.31/lib/ld64.so.2")) > > ;;; (needed ("/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/= libc.so.6" "/gnu/store/ys7b4gr5nbq8sfnff9ry5blb4bhpx6mq-gcc-7.5.0-lib/lib/l= ibgcc_s.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/l= ibm.so.6" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libdl= .so.2" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libcrypt= .so.1" "/gnu/store/xj20v8lk2wal0z1rla0yx3bjkasbx6mq-libunistring-0.9.10/lib= /libunistring.so.2" "/gnu/store/521riv2sgv0b0s4j0kzz6i52rf9rarh8-libffi-3.3= /lib/../lib/libffi.so.7" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc= -2.31/lib/libpthread.so.0" "/gnu/store/7x2cjqbmpgwrgmnb234gsxkmsqs5pj09-lib= gc-8.0.4/lib/libgc.so.1" "/gnu/store/gahs2sx5snbfkr9vlcjj5c2kvnlhr0zs-guile= -3.0.7/lib/libguile-3.0.so.1" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-= glibc-2.31/lib/ld64.so.2" "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glib= c-2.31/lib/../lib/libc.so.6")) [...] > Note that the RUNPATH above contains an entry for > "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib" followed > later by > "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib". It = seems > that ld.so's tracing mechanism is smart enough to avoid printing the > second entry. > > So, the test fails because the "needed" list is not set-equivalent to > the "truth" list. There are two reasons why they are not > set-equivalent: > > A) "truth" contains "linux-vdso64.so.1", but "needed" does not. > > B) "needed" contains > "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/../lib/libc.s= o.6", > but "truth" does not. However, both contain > "/gnu/store/sipyfs2540b48b2sb9j8ypmybja1dvqb-glibc-2.31/lib/libc.so.6", > which refers to the same file. [...] > What is a good solution for (B)? I can think of the following potential > solutions: > > 1) Try to avoid introducing multiple entries referring to the same thing > in the first place. Somehow, somewhere, something is adding the second > entry to the dynamic section of Guile's ELF file. It happens on > powerpc64le-linux but not on x86_64-linux. What code or tool is doing > this? I don't know, but I guess I would start by looking at the > gnu-build-system code. I'm not sure if it's a really problem, though, > so I'm not eager to jump down this rabbit hole just yet. > > 2) Change the test so that it passes even if file-needed/recursive > returns multiple entries referring to the same file. In other words, > accept that the current behavior is OK, even if it means that the > results returned by file-needed/recursive are not always exactly the > same as the results returned by ld.so. > > 3) Try to change file-needed/recursive so that it does not return > multiple entries referring to the same file. In other words, make it > behave more like ld.so. > > I can't think of a reason why the current behavior of > file-needed/recursive is bad, but it was simple enough to make it > deduplicate entries similarly to ld.so. So, my patch implements > solution (3). Hopefully it's good enough! Good catch! We could go fancy and have =E2=80=98loop=E2=80=99 in =E2=80=98file-needed/recursive=E2=80=99 thread a map of device/inode number= pairs to file names; when calling =E2=80=98search-path=E2=80=99, we=E2=80=99d check = whether the file we found already is in the set, possibly under a different name, and we=E2=80= =99d use that name instead of introducing a new one. That=E2=80=99d be more efficient that calling =E2=80=98canonicalize-path=E2=80=99, especially O(n= =C2=B3) times roughly. But=E2=80=A6 given that this is a corner case, that modifying (guix build gremlin) entails a full rebuild, and that there just should be that =E2=80=9C/lib/../lib=E2=80=9D entry in the first place, I=E2=80=99d lean to= wards leaving gremlin.scm unchanged. WDYT? However=E2=80=A6 > diff --git a/tests/gremlin.scm b/tests/gremlin.scm > index 9af899c89a..86757e62b4 100644 > --- a/tests/gremlin.scm > +++ b/tests/gremlin.scm > @@ -1,5 +1,6 @@ > ;;; GNU Guix --- Functional package management for GNU > ;;; Copyright =C2=A9 2015, 2018, 2020 Ludovic Court=C3=A8s > +;;; Copyright =C2=A9 2022 Chris Marusich > ;;; > ;;; This file is part of GNU Guix. > ;;; > @@ -92,7 +93,9 @@ > (loop result)))))) >=20=20 > (define ground-truth > - (remove (cut string-prefix? "linux-vdso.so" <>) > + (remove (lambda (entry) > + (or (string-prefix? "linux-vdso.so" entry) > + (string-prefix? "linux-vdso64.so" entry))) > (read-ldd-output pipe))) >=20=20 > (and (zero? (close-pipe pipe)) =E2=80=A6 I think this part should definitely be committed (=E2=80=98master= =E2=80=99 is fine). Thanks, Ludo=E2=80=99.