From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id oM7pAf9P2mFzLwEAgWs5BA (envelope-from ) for ; Sun, 09 Jan 2022 04:01:19 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id CFrqOv5P2mGp3AAA9RJhRA (envelope-from ) for ; Sun, 09 Jan 2022 04:01:18 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A064D1359C for ; Sun, 9 Jan 2022 04:01:18 +0100 (CET) Received: from localhost ([::1]:42296 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n6OSL-0003bi-MF for larch@yhetil.org; Sat, 08 Jan 2022 22:01:17 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n6OSA-0003ai-Vf for guix-devel@gnu.org; Sat, 08 Jan 2022 22:01:07 -0500 Received: from [2607:f8b0:4864:20::b29] (port=37780 helo=mail-yb1-xb29.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n6OS6-0007xH-3E for guix-devel@gnu.org; Sat, 08 Jan 2022 22:01:06 -0500 Received: by mail-yb1-xb29.google.com with SMTP id 127so12354559ybb.4 for ; Sat, 08 Jan 2022 19:01:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google-2018; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=PztX9ow8/HEW/hZfWxEUs6GoqmrvEvFXjBfNBMJeChE=; b=d+8IhQ2lOxmto9OAvkK1YivbwCiGR7IbOYLPQcLyLICICxdOICPGRDEz8CBi+gyrhV ZYKHhsVN8f/sHBGxojGPHDZjJeSwF3RmRd5J5sHbT0xHfovzQ70Q2UxLIwj+DxYs9BFD BJLNCzr4w3xsWa60IlVJDEqwxIofDGKFhgDwWIMmA8BwiKvLfKOdC3rmweaSYcI1e1wa vpmC/k6X95Dh2VhhdvD3A8/9M7x1lOcArXwDgl6JJp+NEH1dRszSePCv/1WpzfrndxAH eeuaMK1NT4qJacpdZLvx7sXID0OOBUqyMpa/ICbvfHcT/gA7v+5oTG5Pp5mrWTcHRE5d fi1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=PztX9ow8/HEW/hZfWxEUs6GoqmrvEvFXjBfNBMJeChE=; b=nAtxE4nd7YHGplJr0xMGwBmT+8A34CtGjF80s/taiH0NE1XCBP+KI/ccJ3tIMy40Ur 1oNaih6TWM1y/rPG3IrZzagJLTTc50pJcDZSJIorxrsORkppe5CDiaQv8APtqmLMbQxA RlDkG9dP2OIJi3M1D/g+OGf5F8ZKwbvIjPAelZzV48knoBgFsgUW0bBgdG/afuWQtRrV wkWpILQZLzI8KIBTXa1gYILFILVjCrYkn+SR1Si4yJHcE+p+BBoOyAqd98FVP955eRyq bw/5LqSHbdYkMpTkK/NRMg5nYdXMXCer+e8bg8GA2xuALQaXb4ZYGR1cnSx4DPAolodL 1NWg== X-Gm-Message-State: AOAM533W/BfCBECVm+jfVqNm6QIWHZKrPdh+NUzT57a7N94UIy3fXZam b1g/AKLTY0i0GHpHRoD+k49btvum2OyabzlpAW9QWA== X-Google-Smtp-Source: ABdhPJyFkQLwocOXnSaqfgppbQOYiilnIcJVDuuU4kpwepCDWLUWbldSkqQodFOmef2DoRxGVuYc8/2Tx9MbqMkGKjg= X-Received: by 2002:a05:6902:1147:: with SMTP id p7mr63564425ybu.184.1641697260167; Sat, 08 Jan 2022 19:01:00 -0800 (PST) MIME-Version: 1.0 References: <878rvp53eg.fsf@gnu.org> In-Reply-To: <878rvp53eg.fsf@gnu.org> From: Farid Zakaria Date: Sat, 8 Jan 2022 19:00:49 -0800 Message-ID: Subject: Re: Alternative solution to stat storm problem To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Cc: guix-devel@gnu.org, "Scogland, Tom" , Carlos Maltzahn Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::b29 (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::b29; envelope-from=fmzakari@ucsc.edu; helo=mail-yb1-xb29.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1641697278; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=PztX9ow8/HEW/hZfWxEUs6GoqmrvEvFXjBfNBMJeChE=; b=UNG7L1ZZmtx2exRyoPt8zQZqArb62P/8rbcKMSb0UC7giBIeCuN1wemvYTCwh12g+VBe6D lnXKwK09hLux5SHLkmMd4RIPvUFEYmPHUNUmwzHyJt8m49qRLe5P97JSjaX3gVxhE9O7gT sASFP2+bA7u/nGoqbaBNNqF0hXEdtRPL2kHvUziEKFKb5wnfgJFm8rPZOeDf7Ua6QD86qh efNxLRZqwkP2OKM/fY6hcjFrTPObXEYx4lfufUBIVEYMrcd4TJTUwAwYcN34H5/JYPe/VV IKzwnJXWv7JH0m2oxHlURCxSNaTGaneiWC7bJZQUl6YY8r8D+N+cSjFQ14k88A== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1641697278; a=rsa-sha256; cv=none; b=lyFvAKKMkVz16M7L+i7tvUK1CFfXvh7bVu4jg3Kf6ObOHL+zoGJ3Kw4PZVN8PHPQSodPiZ SCCef/NA9iW69LrFn/l86/Z8CTWQrOxMk89CEeANMlfo1jjqWwnH3kTqS46ESBbqlPZ12Y pqE47yLhCtiqyLIp0vHcquazsWvnIJ6PN8NJbiOW5hQycz7Lc5Xp+BhSKv02AMaN6qrkYl lTQE8sX78hYC3ZoTpi+qTf71mQkhXl4NT32VXcUzM8CE9RVJZHb2jCYneQDMmA8Nof6j91 m45SFSV/vtn/1eQWUelMoUOo39EfuUp4rZcpEwtdF/3pA7EiFr9I4jx9YUdj6Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=ucsc.edu header.s=ucsc-google-2018 header.b=d+8IhQ2l; dmarc=pass (policy=quarantine) header.from=ucsc.edu; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -6.31 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=ucsc.edu header.s=ucsc-google-2018 header.b=d+8IhQ2l; dmarc=pass (policy=quarantine) header.from=ucsc.edu; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: A064D1359C X-Spam-Score: -6.31 X-Migadu-Scanner: scn1.migadu.com X-TUID: WC6NJfzPs5aB Hi Ludovic, On Sat, Jan 8, 2022 at 1:22 PM Ludovic Court=C3=A8s wrote: > > Hi Farid, > > Farid Zakaria skribis: > > > I have written a tool _shrinkwrap_ [2] that takes all transitive > > dynamic shared object dependencies (only those listed in DT_NEEDED) > > and turns them into an absolute path. > > > > This has the same result as caching the entries and avoids the > > unnecessary failed attempts at trying each RUNPATH entry. > > > > Using the same demo application _emacs_ shows as much as well: > > Nice! I think that=E2=80=99s another interesting way to address the prob= lem. > > I guess the advantage is that you don=E2=80=99t need the ld.so patch. Th= e > downside is that PatchELF needs to be able to write longer NEEDED > strings in the dynamic section, which it may not always be successful at > (I think?). I can't claim to be a ELF specification guru but I have not encountered that longer NEEDED strings to be a cause for failure. The emacs example is a pretty good test case because the transitive closure of all NEEDED libraries is quite large, which all seem to be added successfully to the ELF header. The benefit to me seems: 1 - does not need a glibc patch for functionality (although for other libc such as musl it might in this case https://www.openwall.com/lists/musl/2021/12/21/1) 2 - understanding the dependencies of an application become simpler 3 - there are esoteric cases where in fact libraries might link to the wrong libraries (although they are correct at build time) given a RUNPATH/RPATH since there are subtleties with the inheritance model. I'm actually researching ways to improve (3) as well through mentorship with Tom Scogland by researching alternative ways to do linking: - RUNPATH per NEEDED - the ability to specify whether a RUNPATH should be inherited or not to downstream dependencies > Also, I wonder if the absolute file names in NEEDED interfere with uses > of $LD_LIBRARY_PATH (making it impossible to force use of another > libxyz.so than the one that would be found in RUNPATH.) Correct. For a system with reproducibility in mind this can perhaps be a desired feature. It is the current limitation of the proposal. In fact, Carlos brought up a great philosophical question: "Is linking to libraries through a content-addressable value allowed for LGPL software?" What if the linked address also forced the content-address by having it resolve to something on IPFS ? > Thoughts? > > Thanks for sharing! > > Ludo=E2=80=99.