From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id CIwaAaFL02IGjQAAbAwnHQ (envelope-from ) for ; Sun, 17 Jul 2022 01:37:05 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id kIR4O6BL02JVXwEAG6o9tA (envelope-from ) for ; Sun, 17 Jul 2022 01:37:04 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3FBB88196 for ; Sun, 17 Jul 2022 01:37:04 +0200 (CEST) Received: from localhost ([::1]:56272 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oCrLL-0003QO-DU for larch@yhetil.org; Sat, 16 Jul 2022 19:37:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40834) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oCrL5-0003Pq-6G for help-guix@gnu.org; Sat, 16 Jul 2022 19:36:47 -0400 Received: from mx.kolabnow.com ([212.103.80.153]:46596) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oCrL2-0004ZF-5n for help-guix@gnu.org; Sat, 16 Jul 2022 19:36:46 -0400 Received: from localhost (unknown [127.0.0.1]) by mx.kolabnow.com (Postfix) with ESMTP id 48A8B417AF; Sun, 17 Jul 2022 01:36:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kolabnow.com; h= content-transfer-encoding:content-type:content-type:mime-version :message-id:in-reply-to:date:date:subject:subject:from:from :references:received:received:received; s=dkim20160331; t= 1658014594; x=1659828995; bh=YAQv00mPlgDg5BScRvxOTGqcK0knbUxQbdy uvGuAvts=; b=qySDMbU29aCwkVLLaeY6/fr2xx5hqixD9mV1UHxGJ+3tUdTH+xT 9WkOfXyCo9S1pMdfiqGPDnItzO2e9np9VeiLg5A4zaYdGxFYlJ8Gr12OjvzPAWN1 ZaNiRqTYvGts7EmJW0M3EBRLKf5tWY8EfAvzccDWL2OQt1x0t0ieRHoqdPscqxJc 3+QsN7g5w5rlopGNHsiQmpDo6yI3CdIEIbYih0aG2JB+1EQxOm+98fONEUoKop2T 4aJwmaKrUgL6mdlLhFr/i1dDC3Qj3h7hTfzjr3/Mb/teasJwnfex0ivGCe/M4Gmf i5bS5slXEpzDwPkxjUrA3tBLm/SmbupJvYowq7mnkomPEtMEIUGS/7pWSdG/42XX aDE+ZEK1kq4zuPMYPTMjPNiCufEVJlLZ/JOCQaYTEGqp94x1xXZkH+vckaV9Ly4H gEsIXCKOdJqnGvZSeurflHIpovdKMsVhHyWxA/ubjHRO7EbB4C0FlYUIwaK+O2Xw xOVPuLQ2SYWuN9HE+4kSdDmjqlceQNWKuHLaFGU0f8y/qefy9tnKXPq/1xp4cydx oxHwp/T7Gbna/hd2lW64y4MUSQvS4Ndol3+4KNLhxWmKerWUE8HEe8+8VZ+7yexF t7H7p4+sdrJM9QXhEbIBNY23e9DS8Oq6hje6fTPpNkWsQ2Wm1aKPX3VA= X-Virus-Scanned: amavisd-new at mykolab.com Received: from mx.kolabnow.com ([127.0.0.1]) by localhost (ext-mx-out003.mykolab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nsQjipiAbF4J; Sun, 17 Jul 2022 01:36:34 +0200 (CEST) Received: from int-mx001.mykolab.com (unknown [10.9.13.1]) by mx.kolabnow.com (Postfix) with ESMTPS id 50C074096E; Sun, 17 Jul 2022 01:36:32 +0200 (CEST) Received: from ext-subm001.mykolab.com (unknown [10.9.6.1]) by int-mx001.mykolab.com (Postfix) with ESMTPS id 7295611643; Sun, 17 Jul 2022 01:36:31 +0200 (CEST) References: <8EA69D34-498D-4172-B0E4-C90E59009F68@polidoro.io> <24DD9713-A088-4C6A-8D8B-5DCF60E623AF@lepiller.eu> <86bktshp6m.fsf@polidoro.io> <4D18100C-EA74-494D-8014-12116892E5AE@lepiller.eu> <867d4ghltf.fsf@polidoro.io> <86cze77bm2.fsf@polidoro.io> From: Thiago Jung Bauermann To: Peter Polidoro Cc: Julien Lepiller , help-guix@gnu.org Subject: Re: Finding Dependencies at Run Time Date: Sat, 16 Jul 2022 18:53:18 -0300 In-reply-to: <86cze77bm2.fsf@polidoro.io> Message-ID: <87cze4eikm.fsf@kolabnow.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=212.103.80.153; envelope-from=bauermann@kolabnow.com; helo=mx.kolabnow.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1658014624; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=YAQv00mPlgDg5BScRvxOTGqcK0knbUxQbdyuvGuAvts=; b=bJrHzrE6K4aJ9wkuhtkUmEh1lglTb0Wkxz92HC7FPGVSG+kPzUSZk1JkIsmCmAbs2XShoB JRmZ27JO+4sgUZXuutTRGscTvNBij3TK3etdixW7hzredSHVbVIP/UHUFY7Kkr9O33tXwD 9AIJzSWU4f3yUsclfNPTGRqW/bJknNd9ajeE2/eH/TRysdD0EBWJdXugEgfGs8iuDvk6w3 Rap8/rzJn+EQIMxRc/EV42QBJI4v/Q/sQBMrZ8pw9ZDGfIdRNS8b+yp4EVa1YjmcUAkApb YzNiLTbZZcuP/xf5K2KkxduGd7Mo7Etx6f9GAHSANfQ1JgInxP8DeF4xcZM1fw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1658014624; a=rsa-sha256; cv=none; b=qebbFBRwb6CXGPPgOywbHsr2bdx6gQRA7zCJ6UNbUphH1GiB4yB8Ah6vE+E0h6bJwVvQ0o imN24nkoiEI5fKrWTywhU8kToKXG+dNEqDkz5mKm2sNLu8x7iIQBpLcQb65uDluo2xNvGp jTwMK4O7bg48YMDxJmXBrT5SSsZIWTzbsRBWDg2IqRzJm5vZnd+eYj815CKf9/8gXsQ2kG OVNlG0Mgi2nDoX47zpuRzScTitYb4kvsCk1jfTqMN6bs6iaojSZ60q7IcLmcPLn9vm5iiQ 3KUkq1NpW89ktfcZTNT1uy2sAeAaug6ZqOWJxqxUYKQ7xMKTBIziC2zSBy/l7A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=kolabnow.com header.s=dkim20160331 header.b=qySDMbU2; dmarc=pass (policy=quarantine) header.from=kolabnow.com; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.94 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=kolabnow.com header.s=dkim20160331 header.b=qySDMbU2; dmarc=pass (policy=quarantine) header.from=kolabnow.com; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3FBB88196 X-Spam-Score: -3.94 X-Migadu-Scanner: scn1.migadu.com X-TUID: pJzx6zqtKbLr Hello Peter, Peter Polidoro writes: >> Remember the difference between inputs and propagated inputs: they're >> the same, but when you create a profile, inputs are not part of the >> profile (so they need a direct store reference, such as RPATH or a >> wrapper), whereas propagated inputs are part of the profile, so an >> environment variable allows to find them. > > Thank you for patiently explaining, I think I am starting to > understand now, but please correct me if I am still wrong. I think there are still some misunderstandings, or perhaps at least to me it's not clear what your question is. I think if you could provide a concrete example of what you are trying to understand it would be helpful. > So there is package build-time, profile build-time, and run-time. > > Wrappers should be used to set environment variables when the value > can be determined at package build-time, such as absolute paths to > inputs. No. Ideally wrappers shouldn't be necessary and wouldn't be used, but for some software they are a necessary/useful workaround. At least that's my understanding. In the ideal situation, a Guix-package program uses absolute paths to find the resources (libraries, scripts, other files) that it needs. For example: how does a compiled program finds the library it was linked with? It has the full path to glibc hard-coded into the ELF's library runpath: popigai: readelf --dynamic ~/.guix-profile/bin/htop | grep runpath 0x000000000000001d (RUNPATH) Library runpath: [/gnu/store/9rrnm= 5hdjw7cy96a2a9rfgh6y08wsbmf-ncurses-6.2.20210619/lib:/gnu/store/5h2w4qi9hk1= qzzgi1w83220ydslinr4s-glibc-2.33/lib:/gnu/store/094bbaq6glba86h1d4cj16xhdi6= fk2jl-gcc-10.3.0-lib/lib:/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10= .3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/10.3.0/../../..] Or if it's a shell script, it has the full path to its interpreter: popigai: head -n1 ~/.guix-profile/bin/vcsh #!/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/sh Sometimes the original program doesn't use absolute paths, but we can patch it so that it does. This is the case for example in public-inbox, which invokes =E2=80=9Cxapian-compact=E2=80=9D and expects it to be found i= n PATH, and we change it to use the full path (this is in guix/gnu/packages/mail.scm, line 4111 in the revision of Guix I'm looking at): --8<---------------cut here---------------start------------->8--- ;; Use absolute paths for 'xapian-compact'. (substitute* "lib/PublicInbox/Xapcmd.pm" (("'xapian-compact'") (format #f "'~a'" (search-input-file inputs "/bin/xapian-compact")))) --8<---------------cut here---------------end--------------->8--- But sometimes such changes aren't possible so in that case we do use wrappers to configure environment variables that will cause the program to find the resource. This also happens in public-inbox, which around line 4160 has a =E2=80=98wrap-programs=E2=80=99 phase that creates a wrappe= r so that it can find Perl modules and also the git and curl executables. One thing that is implicit from the above is that at runtime, Guix doesn't do anything to help a program find its dependencies. It has already done its job at build time by hard-coding paths so that the program can find them on its own or setting up a wrapper, and at profile build-time, by setting up any relevant environment variables. > Search paths should be used to set environment variables when the > value will be determined at profile build-time, such as relative paths > to either propagated-inputs or packages that also installed in the > same profile, but not listed explicitly as propagated-inputs to the > package declaring the search paths. I'd say that search paths should be used when the set of resources that a program will need can't be determined at build time. For example, the Python interpreter can load a huge number of libraries. It doesn't make sense to hard-code the absolute paths of all of them in the Python interpreter. Rather, it should look for the ones that are installed in the profile. Another use case is when a program's dependency is optional and it can work well enough without the dependency being available. That way, the user can install the dependencies they want and the program will use them. > I am still a little unclear about the difference between search-paths > and native-search-paths, though. If not cross-compiling then they act > the same? Yes, that's correct. > If you are cross-compiling, though, then it only finds packages that > are listed as native-inputs to the package declaring the search paths? I don't know about cross-compilation profiles, but hopefully it's similar to the build environment set up by Guix when cross-compiling a package. Because of that, I'll answer some of your questions in this section of the email but not all of them. When Guix cross-compiles a package, native inputs are the ones that will be used during the process of building the package (and not at runtime), so they should be built for the architecture of the build machine. For example, the compiler itself, the make tool, etc. The regular inputs will be used by the package at runtime and thus will be built for the target architecture. > Native-inputs are not normally installed in a profile, though, > correct? In the case of a non-cross build, the native inputs will also be installed but will be of the same architecture as the normal inputs. In this case, it doesn't make any difference whether a dependency is listed in the native-inputs or inputs field of a package. I hope I haven't added to the confusion. :-) --=20 Thanks Thiago