From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id KKiwGipR2mHhMwEAgWs5BA (envelope-from ) for ; Sun, 09 Jan 2022 04:06:18 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id sHJmFypR2mE8VwEAauVa8A (envelope-from ) for ; Sun, 09 Jan 2022 04:06:18 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 244DD218F0 for ; Sun, 9 Jan 2022 04:06:18 +0100 (CET) Received: from localhost ([::1]:45286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n6OXB-0005oV-5B for larch@yhetil.org; Sat, 08 Jan 2022 22:06:17 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48168) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n6OWx-0005mm-NH for guix-devel@gnu.org; Sat, 08 Jan 2022 22:06:03 -0500 Received: from [2607:f8b0:4864:20::b2d] (port=43856 helo=mail-yb1-xb2d.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n6OWv-0000G6-At for guix-devel@gnu.org; Sat, 08 Jan 2022 22:06:02 -0500 Received: by mail-yb1-xb2d.google.com with SMTP id g81so1337325ybg.10 for ; Sat, 08 Jan 2022 19:06:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google-2018; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=BhMkZLifr5M4O5Mci+t0B9syGQkFS6knxRpKeHxqCtQ=; b=DO6x2xcIgCPqWLqS1cDttZqbbTPKaeQbrG9TC9LkOS7rPDqtC0Y3ZfbkDZtkCAoFpm cUNKcCUo5en92BcvI0UdHsWpV7JPIw/8Ruzj4c30BqQQ6ppevCZxQTYHsUcop14R0TLF 7BaeahUZ6GwOG+i1lKBCiQYXcVaTQCs8X4FTQ4V/CGNQyGyyF3xFnU9shGe5c+hs+kTD KlwcIxEfkVZVtpuTpWjwDTN1OSJeRxOf+tZvZp+wkiZKEMnz4HDc5uAMY4SFw1h3u/ZS FEhqU05V2LT7eUAfGnI8z7DAZTsjMf8DY7AOwiqpyc5D/oQp6/0w2km8m3Shsfu3IzdT Tn/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=BhMkZLifr5M4O5Mci+t0B9syGQkFS6knxRpKeHxqCtQ=; b=H0XWam48pDywrqsF1DsQ+OHvtBeyxGHpnCtvLDUNmHVTITU3YzABm62XSiWaH9NOIA K1Yh4iy0R6YZ8sJ5DrlFiux7B7l0Sxfc5syeUUOkgSFkCQJIZ8uaT44p+uViDQNhIFSv 4qt74kel+6TCguRsqwOXPRrd4DMAGhWFxc6iNWcrGWClaxxa3dDRIL4XgXJY3WI+oFL5 A8+EaTvMrd+mCLM8m8dg58sDAZEwjn68htVvnXYYjPP7fBnNlOPcyJuMT24kQsw+VIwk Q3qZnuWm+XYtnPSTo5+VOlX/DirlDJJoEh9b75kP0r9JXCsydslgstdZXn9OLG0E5YZV 0VEA== X-Gm-Message-State: AOAM530ixC3GBBh1dvO7e76S+/aCmbti/bgfMdBydXHkpckYKp45G5wC ahSLvZBPwoxai73WndYjOa1VHglOBuP95q6kA1xh2w== X-Google-Smtp-Source: ABdhPJxAJNXO6SPW4IkA7QEA+kDEvwiMam/uD9A3EKvAuyZ0lm78nRK0PT2uTPW37VSNe5JCyBooyH81m4MhXrbLBhA= X-Received: by 2002:a25:260d:: with SMTP id m13mr74253848ybm.29.1641697560113; Sat, 08 Jan 2022 19:06:00 -0800 (PST) MIME-Version: 1.0 References: <878rvp53eg.fsf@gnu.org> In-Reply-To: From: Farid Zakaria Date: Sat, 8 Jan 2022 19:05:49 -0800 Message-ID: Subject: Re: Alternative solution to stat storm problem To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Cc: guix-devel@gnu.org, "Scogland, Tom" , Carlos Maltzahn Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::b2d (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::b2d; envelope-from=fmzakari@ucsc.edu; helo=mail-yb1-xb2d.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1641697578; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=BhMkZLifr5M4O5Mci+t0B9syGQkFS6knxRpKeHxqCtQ=; b=bIgYJdGJDe4xM1DLNeb9B+eRx9H0XtViKpStZO9VIZHwahqAg5ezPWLNP4vq7OIWmSznip o8erq6cuyu3LSzIVrp5epfgDbuDsLh88ZbBtCaoWQxvLat/qoWtU4KUqbRUgbqoTonhk1L RkHhTjkpWYzlBUL9QWnT2Wcpp36CmFDJsDjZWSXlRCGYR2IANeSxNA+jb+4a9xp5uk9jU3 4/sVnzznXssjhN2vjaodg0JBr4MGQbnJqrWdtN0zhZ87mbe1RlJbG6dfSsv4MoAbjhAcR0 VeS2pr2/3BJziJoi/oraS9WGZX6vQ2tI8J4q4/dY+pQpLmE+z8pRnZ2tHl9GWA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1641697578; a=rsa-sha256; cv=none; b=AgpJ2wxKmCYSPizUMTS7jKY0URp63njfCV2ZFPyFL+DtrhhNkAlpWHoWI5276PPEZqOj6m 1Qp7RKqai+CKwbY24mJK/9uRSW3efKNvB+iajiWm/9tNChqjgowjZQfatGssY+6076t/B0 SfnG3DjwGhZi55mH5VvitDVNBOJPzZkn5SeG0ZcW19z1NExfg1bgIbxvneSHA+KIiRCavt vh6cyH0E8a0x+sSd7uIt60Y2OZb/hu500nhScg9scFkydSMJSsbplSBbtpRxFCKYm/YYSA VmkpdAqjpD2kfl4Qovo/BmZnHm38qHFtivsOdm1M366SrxSZn0JvaaR+qMgYPw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=ucsc.edu header.s=ucsc-google-2018 header.b=DO6x2xcI; dmarc=pass (policy=quarantine) header.from=ucsc.edu; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -6.31 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=ucsc.edu header.s=ucsc-google-2018 header.b=DO6x2xcI; dmarc=pass (policy=quarantine) header.from=ucsc.edu; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 244DD218F0 X-Spam-Score: -6.31 X-Migadu-Scanner: scn1.migadu.com X-TUID: 1NNFaXN0lfLG I did forget to mention the point of LD_LIBRARY_PATH, that you can still make use of LD_PRELOAD and I am also thinking about maybe using something like dlopen-resolver[1] to further expand the NEEDED section. [1] https://github.com/Mic92/dlopen-resolver On Sat, Jan 8, 2022 at 7:00 PM Farid Zakaria wrote: > > Hi Ludovic, > > On Sat, Jan 8, 2022 at 1:22 PM Ludovic Court=C3=A8s wrote: > > > > Hi Farid, > > > > Farid Zakaria skribis: > > > > > I have written a tool _shrinkwrap_ [2] that takes all transitive > > > dynamic shared object dependencies (only those listed in DT_NEEDED) > > > and turns them into an absolute path. > > > > > > This has the same result as caching the entries and avoids the > > > unnecessary failed attempts at trying each RUNPATH entry. > > > > > > Using the same demo application _emacs_ shows as much as well: > > > > Nice! I think that=E2=80=99s another interesting way to address the pr= oblem. > > > > I guess the advantage is that you don=E2=80=99t need the ld.so patch. = The > > downside is that PatchELF needs to be able to write longer NEEDED > > strings in the dynamic section, which it may not always be successful a= t > > (I think?). > > I can't claim to be a ELF specification guru but I have not > encountered that longer NEEDED strings to be a cause for failure. > The emacs example is a pretty good test case because the transitive > closure of all NEEDED libraries is quite large, which all seem to be > added successfully to the ELF header. > > The benefit to me seems: > 1 - does not need a glibc patch for functionality (although for other > libc such as musl it might in this case > https://www.openwall.com/lists/musl/2021/12/21/1) > 2 - understanding the dependencies of an application become simpler > 3 - there are esoteric cases where in fact libraries might link to the > wrong libraries (although they are correct at build time) given a > RUNPATH/RPATH since there are subtleties with the inheritance model. > > I'm actually researching ways to improve (3) as well through > mentorship with Tom Scogland by researching alternative ways to do > linking: > - RUNPATH per NEEDED > - the ability to specify whether a RUNPATH should be inherited or not > to downstream dependencies > > > Also, I wonder if the absolute file names in NEEDED interfere with uses > > of $LD_LIBRARY_PATH (making it impossible to force use of another > > libxyz.so than the one that would be found in RUNPATH.) > > Correct. For a system with reproducibility in mind this can perhaps be > a desired feature. > It is the current limitation of the proposal. > > In fact, Carlos brought up a great philosophical question: > "Is linking to libraries through a content-addressable value allowed > for LGPL software?" > What if the linked address also forced the content-address by having > it resolve to something on IPFS ? > > > Thoughts? > > > > Thanks for sharing! > > > > Ludo=E2=80=99.