From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Marusich Subject: Re: How should I install non-guix software? Date: Sat, 04 Aug 2018 21:48:41 -0700 Message-ID: <871sbde7va.fsf@gmail.com> References: <20180804214540.497af879@gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:42001) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmAyK-0004FO-Rk for help-guix@gnu.org; Sun, 05 Aug 2018 00:48:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmAyJ-0003qP-6m for help-guix@gnu.org; Sun, 05 Aug 2018 00:48:52 -0400 Received: from mail-pl0-x233.google.com ([2607:f8b0:400e:c01::233]:38744) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmAyI-0003pp-SS for help-guix@gnu.org; Sun, 05 Aug 2018 00:48:51 -0400 Received: by mail-pl0-x233.google.com with SMTP id u11-v6so4242156plq.5 for ; Sat, 04 Aug 2018 21:48:49 -0700 (PDT) In-Reply-To: <20180804214540.497af879@gmail.com> (Luther Thompson's message of "Sat, 4 Aug 2018 21:45:40 -0400") List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+gcggh-help-guix=m.gmane.org@gnu.org Sender: "Help-Guix" To: Luther Thompson Cc: help-guix@gnu.org --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi Luther, Those are really good questions. I'll try to explain what I know, and hopefully it'll be useful to you. Luther Thompson writes: > I don't know if this has been discussed before, but is there a best > practice for installing software from outside the Guix package system? For custom packages, you can define your own via the GUIX_PACKAGE_PATH. See "Package Modules" in the Guix manual for how to do that [1]. For software installed via other package management tools like npm, Maven, or pip, it's trickier. Those package managers ought to work out of the box. I use Maven (installed via Guix) frequently on a GuixSD system, and it's always worked well for me. However, because (1) those package managers don't provide the same kind of strong guarantees about dependencies that Guix does, and (2) because GuixSD doesn't follow the FHS (for good reasons), there is definitely the risk that some software you install via those other package managers won't work correctly out of the box on GuixSD. > If I have a source tarball for a program that doesn't have a Guix > package, or if I write my own program, there just isn't any good place > to put the installed files. Scripts have the additional problem of not > being able to use their shebang line. Yes, the hard-coded references are a common problem. It's always been a problem when software assumes that the components they require will exist at certain absolute paths, since even among systems that don't use Guix these paths can vary. Software that has been designed from the start to be portable will sometimes mitigate this problem by providing a way for users to tell the program where its dependencies live (e.g., via an option to the ./configure script). However, this is still a common problem. I know of at least a few ways to deal with this problem. They are: 1) Work with upstream to provide a way for users to specify the location. For example, Pierre is doing this for magit right now [2]. 2) Replace every hard-coded occurrence of the path with a correct path. We do this all the time. For just one example, refer to the use of the substitute* procedure in the package definition for pies (run "guix edit pies"). 3) Run the software in an environment where its expectations are met. For (3), like you suggested, one way to do it is to create symlinks in your system. That's fine for a quick, local solution, but it isn't appropriate for software that will be included in Guix proper. I've sometimes created "/bin/bash" and "/usr/bin/env" on my system for reasons like this. When using GuixSD, you can define these kinds of files in your OS configuration using the special-files-service-type. See "Base Services" in the manual for details [3]. > As a more unique example, I was recently using a React tutorial. IIRC, > I ran the command `npm start`. It tried to run another installed script. > I had to manually change the shebang line of that other script to point > to /home/luther/.guix-profile/bin/node to get it to run. The build systems of Guix (e.g., gnu-build-system) generally have a build phase that automatically finds and patches shebang lines, replacing them with absolute paths to the corresponding interpreters in the store. Unfortunately, if other package managers like npm, Maven, pip, etc. do not provide a way to change hard-coded paths - and I don't think any of them do - this will always be a problem. In my opinion, if you have to use these kinds of tools to install software on GuixSD, the best compromise is probably to create symlinks in your system as above. As far as I know, there isn't a good, holistic way in GuixSD to deal with hard-coded paths in software that has been installed via other package managers like npm. If someone knows better, I'd love to hear otherwise! > 1. Write a Guix package for every little piece of software we want to > run. I have yet to successfully do this, but I might decide to learn > someday. I already know Scheme. AFAIK, this won't work for node > packages, but it should work for everything else. This solution would > be a lot of work for users. In some ways, this is the best approach. However, I know it's easier said than done. If you're ever feeling stuck, you can ask for help here as long as the software you're trying to package is free software. It's also helpful to look at the thousands of existing examples in the Guix source tree, which are located in the gnu/packages directory. > 2. Put ~/bin in our $PATH and install all our executables there. This > wouldn't work for the majority of software that installs more than just > executable files. I'm not sure what you mean here. How would putting ~/bin in your $PATH enable software that hard-codes a path of "/bin/bash" to find the bash program? > 3. Instead of installing the software, run it directly from whatever > directory we unpacked/compiled it to. Wouldn't this also fail to address the problem of hard-coded paths? > 1. Make a soft link at /usr/bin/env in the root filesystem pointing > to /run/current-system/profile/bin/env. This seems like the simplest > solution, but I figure there must be some reason the devs haven't > already done this. Yeah, I think creating symlinks like this is probably the easiest way to get "foreign" software working on a GuixSD system. It doesn't always work, but when it does, it's an easy hack. However, like I said, that solution isn't appropriate for software that will be included in Guix proper. This is because it just isn't necessary: if the software is packaged in Guix proper, then all of its dependencies are guaranteed to exist (at exactly the intended versions) in the store. In fact, if we added symlinks like "/bin/bash" to GuixSD by default, it would mask certain kinds of packaging problems. If I packaged a piece of software that uses the hard-coded path /bin/bash at runtime, it would run fine on my system because I have a "/bin/bash" symlink, which points to some (indeterminate) version of bash. When you install the same package, your /bin/bash symlink might point to a different version of bash, so the software might behave differently for you than it does for me. In addition, if for example you make a bundle of the software using "guix pack" and ship it to a system that doesn't have /bin/bash, the software might break entirely. My understanding is that one of the reasons why Guix doesn't follow the FHS is to make it difficult to accidentally package software that will break in these sorts of ways. If /bin/bash is missing from my system when I package software that assumes it will exist at that path, I will be more likely to notice that the software depends on something outside the store. I can then fix it by replacing the path with an exact version of bash that is managed by Guix. In this way, the problems mentioned above will not happen, since whenever somebody installs the software via Guix, it is guaranteed that the installed software will use the exact same version of bash that I specified in the package definition (assuming that the user isn't using a different version of Guix) In this way, GuixSD is designed to be a "pure" system that tries hard to prevent undeclared dependencies from creeping into your environment. Undeclared dependencies are bad because they can interfere with or break software. The usual FHS runs counter to this goal, since it encourages software to make specific assumptions about the environment. With Guix (and Nix), it is no longer necessary to make those kinds of assumptions in the first place, since all the dependencies are always known exactly. > 2. Know which interpreter we need for each program and invoke that > interpreter explicitly. Yes, this is also an option, but as you might imagine, it doesn't scale past a handful of scripts/programs. > 3. Manually change the shebang line for every script to point into > our .guix-profile. I don't like this idea, because it would mean > hard-coding the name of our home directory into every script. We can't > really do that with programs that we want to hack on and publish. Yeah, I agree that isn't a great solution, either. But it will work in a pinch, too. > Currently, I invoke binaries from their own directories and for > scripts, I invoke the interpreters. I've only been using GuixSD for a > short time, so I doubt that this is a fool-proof solution. How do > you guys handle this? In sum, I guess the most common things that I do are (in no particular order): * Create symlinks in my system when needed. * Manually patch specific files (e.g., replacing their shebang lines, or using patchelf to replace the ld interpreter path). * Create package definitions either in my GUIX_PACKAGE_PATH or in a special guix.scm file (which you can install into a custom profile using "guix package -f my-file.scm"). Ultimately, if it works out, I upstream my packages to Guix proper so everyone can benefit. Finally, an alternative of last resort is to run the problematic software in a VM using a more traditional GNU/Linux distribution. However, I didn't mention that until now, since it's basically equivalent to giving up on trying to run it in GuixSD itself. Anyway, I hope that helps! Thanks for bearing with my very long reply. Footnotes:=20 [1] https://www.gnu.org/software/guix/manual/en/html_node/Package-Modules.= html#Package-Modules [2] https://lists.gnu.org/archive/html/help-guix/2018-08/msg00032.html [3] https://www.gnu.org/software/guix/manual/en/html_node/Base-Services.ht= ml#index-special_002dfiles_002dservice_002dtype =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAltmgakACgkQ3UCaFdgi Rp1lqhAAnUp3K3cXEIoTIQGt0eeprq7RgI1A5iKlVR3PDPc6rMZxa7jYil8kxjWM OCkFjJ2Z9c9HXxm6B8Z0jbq5K3AzTXXRAVZTAqQ+NVJsBpHL6YTefYTB+IzegLqv NdttukWLuDxV6lagg4aYlKU7cxGb2Gl3yh/fALFQ+IOYQm9R5634C79+w7RyoVB5 Ny/2Do1S2eCpXUWNLFIceJMLrC4F9yl6DRuFK+5i3w7BylsfjzfzA6Qn7SralLxk /5IUtoudYuaKpf0fhVIMOYmSak1gtvb5vSn1i9ITHnOXzAM6KHzLOLlwOjk0v+cm IWYy69xdAEas9bsfLCntTZQhvMfQaUNBd73mCWE6WonaIj6YU8y3BRrC5IYu2oWn qK1mvyYtjzoWlk87v6HquSTH9kRy+SVqu3M0YOY0F/io+lzDmkARM3cMRBdMJkHi h8sCuZCQW/JpSvNPMB8NlOjmLJLmUjQrnf6oT+UiCAHHEqRD4YaaGKOXvh/9BIen zNV7xrKBXHlbhgEsLpguwUXUg9x9uFIqYXWRGcOBaOSDFXvD869p+RB1dzjuefCQ 5FqApd3RIduYudBmGK9vZGHV48DH/kzbdPuGVm7Fa5eDuu39TdIttPYFlkaAh7G8 CtRzEoHkbAMQAz3631MB/8wxQ46tJcMT58hLn1HV5tyMcsN7CN4= =9IlW -----END PGP SIGNATURE----- --=-=-=--