From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Installing guix packages without root permissions (in HPC environments) Date: Wed, 18 Jan 2017 22:42:09 +0100 Message-ID: <87eg006o5q.fsf@gnu.org> References: <20170117091544.GA8189@mail.thebird.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:36862) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cTxzl-0005tT-SO for guix-devel@gnu.org; Wed, 18 Jan 2017 16:42:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cTxzh-000533-UJ for guix-devel@gnu.org; Wed, 18 Jan 2017 16:42:17 -0500 In-Reply-To: <20170117091544.GA8189@mail.thebird.nl> (Pjotr Prins's message of "Tue, 17 Jan 2017 09:15:44 +0000") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Pjotr Prins Cc: guix-devel@gnu.org Hi Pjotr! Pjotr Prins skribis: > But, I thought the easy way is to patch a path with something the has > the exact same size(!). This has the advantage that it will always > work. Trying this second strategy I wrote a new tool which replaces > the old path with a new one that takes the prefix and truncates the > rest of the path so a prefix /usr/local/bin/hello overwrites > > Found @512: /gnu/store/qv7bk62c22ms9i11dhfl71hnivyc82k2-glibc-2.22/... > Replace with /usr/local/bin/hello/glibc-2.22-qv7bk62c22ms9i11dhfl71/... > > You can see we swap the hash position and start 'eating' the path from > the end. Ah ha, clever and fun! :-) > This should work across almost all files, unless the path is scrambled > in some way. The downside of the fixed strategy is that a prefix can > not grow beyond the size of the one in the store. Also every store > path may look a bit different between installs. The latter is the price to pay, I guess. > You can try it out by downloading and unpacking > > http://biogems.info/contrib/genenetwork/guix-build-hello-2.10-x86_64.= tgz > > Run the contained installer with > > ./install.sh prefix > > Use --help, -v and -d for more output. And run the installed tool > with > > env LC_ALL=3Dfr_FR prefix/profile-hfmsjsvx1p68wbx0fli/bin/hello > Bonjour, le monde! > > French (if the locale resolves ;). On non-guix machines I have > already deployed some complex packages this way, including Ruby, > sambamba and the ldc compiler (which includes LLVM). Woow, good to know that it works well in practice. > To create such an installable and relocatable tarball start from a > package with its dependencies, such as found in a Guix archive or > created with > > guix environment --container --ad-hoc mypackage tar gzip -- tar cvzf = mypackage.tgz /gnu/store > > Unpack that tar ball and copy the installer files that sit in the > hello example. That is all. > > The source code for the guix-relocator is at > https://github.com/pjotrp/guix-relocate/blob/master/src/main.d and the > supporting scripts are at https://github.com/pjotrp/gnu-install-bin (I > may have to change that name). > > To fix the prefix restriction there are two routes, one is building > Guix itself on a large(r) path, which is the easy route, or write a > relocate patcher that can handle the non-zero terminated paths. I > think the latter is feasible too. Even at this point the fixed length > strategy is useful for most environments - the prefix can be some 40 > characters long. I think the fixed-length restriction is not that bad. The two workarounds above look hacky and super tricky, respectively. ;-) Also, if we look at the big picture of non-root usage, this solution addresses the most hostile environments: no user namespaces, no container-spawning facility, no root guix-daemon, etc. Granted, these hostile environments are still commonplace in HPC, so that=E2=80=99s good. = :-) > What does this all mean? In short, if the prefix is not too long we > can run practically all Guix binary software on a non-guix host with > normal user permissions. Something we need for HPC and supercomputing > and is the subject of my FOSDEM talk on the HPC track: > > https://fosdem.org/2017/schedule/event/hpc_deployment_guix/ > > There is also scope for creating a one-step installer. A (trusted) > server could distribute binary software with its dependencies to any > Linux machine. I am sure software developers are interested in that. Awesome! Ludo=E2=80=99.