* thesis: guixsd should provide /usr/bin/env @ 2015-11-16 9:19 Andy Wingo 2015-11-17 6:48 ` Ricardo Wurmus 2015-11-19 16:07 ` Ludovic Courtès 0 siblings, 2 replies; 14+ messages in thread From: Andy Wingo @ 2015-11-16 9:19 UTC (permalink / raw) To: guix-devel Hi! I think GuixSD should follow NixOS's lead and provide /usr/bin/env. The reason is to support scripts that can run either on GuixSD or on some other system. For example instead of: #!/bin/bash we can do #!/usr/bin/env bash So we just need /usr/bin/env in the root and nothing else. Alternately, I am not sure if this would work but we could make a form of "guix environment" which populates a profile that is mounted at /usr in a container. That would allow many more non-Guix tools to run. WDYT? Andy ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2015-11-16 9:19 thesis: guixsd should provide /usr/bin/env Andy Wingo @ 2015-11-17 6:48 ` Ricardo Wurmus 2015-11-19 16:07 ` Ludovic Courtès 1 sibling, 0 replies; 14+ messages in thread From: Ricardo Wurmus @ 2015-11-17 6:48 UTC (permalink / raw) To: Andy Wingo; +Cc: guix-devel Andy Wingo <wingo@igalia.com> writes: > I think GuixSD should follow NixOS's lead and provide /usr/bin/env. The > reason is to support scripts that can run either on GuixSD or on some > other system. For example instead of: > > #!/bin/bash > > we can do > > #!/usr/bin/env bash > > So we just need /usr/bin/env in the root and nothing else. > > Alternately, I am not sure if this would work but we could make a form > of "guix environment" which populates a profile that is mounted at /usr > in a container. That would allow many more non-Guix tools to run. > > WDYT? I just did this on GuixSD: sudo -E guix package -p /usr -i bash coreutils Now I have /usr/bin/env and /usr/bin/bash. I don’t think I needed it before, though. “guix environment” is probably not the appropriate tool for this. I think it’s better to do this explicitly with profiles than to treat /usr/bin/env somehow specially. Providing /usr/bin/env doesn’t necessarily help — one would still have to have an interpreter installed in an active profile. ~~ Ricardo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2015-11-16 9:19 thesis: guixsd should provide /usr/bin/env Andy Wingo 2015-11-17 6:48 ` Ricardo Wurmus @ 2015-11-19 16:07 ` Ludovic Courtès 2015-11-19 17:35 ` Andy Wingo 1 sibling, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2015-11-19 16:07 UTC (permalink / raw) To: Andy Wingo; +Cc: guix-devel Andy Wingo <wingo@igalia.com> skribis: > I think GuixSD should follow NixOS's lead and provide /usr/bin/env. The > reason is to support scripts that can run either on GuixSD or on some > other system. For example instead of: > > #!/bin/bash > > we can do > > #!/usr/bin/env bash > > So we just need /usr/bin/env in the root and nothing else. In ~8 years of NixOS and then GuixSD, I’ve lived without /usr/bin/env. I use #!/bin/sh in scripts I write, which is known to be Bash on GuixSD. But I don’t write so many throwaway scripts, and when I do, it’s either Guile or Bash. On IRC, Mark pointed out that, if /usr/bin/env were available by default, then scripts or generated scripts may more often end up relying on /usr/bin/env rather that precisely specify what interpreter they need, which would partly defeat the point of all this. My current inclination would be to not provide /usr/bin/env by default, and instead let users add it if they want to, either using the sledgehammer Ricardo suggests ;-), or simply with: ln -s /run/current-system/profile/bin/env /usr/bin/env We could document it, and/or even add a switch in ‘operating-system’ to do that. How does that sound? > Alternately, I am not sure if this would work but we could make a form > of "guix environment" which populates a profile that is mounted at /usr > in a container. That would allow many more non-Guix tools to run. Technically ‘guix environment --container’ could create /usr, just like it creates /bin/sh. Not sure if it’s a good idea, though. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2015-11-19 16:07 ` Ludovic Courtès @ 2015-11-19 17:35 ` Andy Wingo 2015-11-20 14:00 ` Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Andy Wingo @ 2015-11-19 17:35 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel On Thu 19 Nov 2015 16:07, ludo@gnu.org (Ludovic Courtès) writes: > My current inclination would be to not provide /usr/bin/env by default, > and instead let users add it if they want to, either using the > sledgehammer Ricardo suggests ;-), or simply with: > > ln -s /run/current-system/profile/bin/env /usr/bin/env > > We could document it, and/or even add a switch in ‘operating-system’ to > do that. > > How does that sound? Fine with me. My use case is when working on Chromium, which has a bunch of scripts that are part of its development environment. Most of them start with /usr/bin/env, just a couple don't and I think those are bugs. In practice though that's something of a hack and what I really should go for is the --container thing... >> Alternately, I am not sure if this would work but we could make a form >> of "guix environment" which populates a profile that is mounted at /usr >> in a container. That would allow many more non-Guix tools to run. > > Technically ‘guix environment --container’ could create /usr, just like > it creates /bin/sh. Not sure if it’s a good idea, though. I think it is definitely interesting. The reason being, you might hack on something or have to deploy something and it's not part of Guix -- you don't want to rewrite the shebang lines for files in git that aren't build products. Being able to make a just-FHS-enough environment inside a container sounds to me like a useful tool to have for shimming Guix and the outside world, while also benefitting from Guix's reproducible environments, rollbacks, isolation, and so on. Andy ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2015-11-19 17:35 ` Andy Wingo @ 2015-11-20 14:00 ` Ludovic Courtès 2015-11-20 15:58 ` Andy Wingo 2016-02-29 18:11 ` Christopher Allan Webber 0 siblings, 2 replies; 14+ messages in thread From: Ludovic Courtès @ 2015-11-20 14:00 UTC (permalink / raw) To: Andy Wingo; +Cc: guix-devel Andy Wingo <wingo@igalia.com> skribis: > On Thu 19 Nov 2015 16:07, ludo@gnu.org (Ludovic Courtès) writes: [...] >>> Alternately, I am not sure if this would work but we could make a form >>> of "guix environment" which populates a profile that is mounted at /usr >>> in a container. That would allow many more non-Guix tools to run. >> >> Technically ‘guix environment --container’ could create /usr, just like >> it creates /bin/sh. Not sure if it’s a good idea, though. > > I think it is definitely interesting. The reason being, you might hack > on something or have to deploy something and it's not part of Guix -- > you don't want to rewrite the shebang lines for files in git that aren't > build products. Being able to make a just-FHS-enough environment inside > a container sounds to me like a useful tool to have for shimming Guix > and the outside world, while also benefitting from Guix's reproducible > environments, rollbacks, isolation, and so on. Yeah, makes sense. One can already do: guix environment --container --ad-hoc coreutils \ --expose=$(which env)=/usr/bin/env Should we add a --fhs or --/usr/bin/env option to simplify this? Thanks, Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2015-11-20 14:00 ` Ludovic Courtès @ 2015-11-20 15:58 ` Andy Wingo 2016-02-17 7:31 ` 宋文武 2016-02-29 18:11 ` Christopher Allan Webber 1 sibling, 1 reply; 14+ messages in thread From: Andy Wingo @ 2015-11-20 15:58 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel On Fri 20 Nov 2015 14:00, ludo@gnu.org (Ludovic Courtès) writes: > Should we add a --fhs or --/usr/bin/env option to simplify this? Oh my goodness this would be really nice! Exposing the profile in /usr would be quite interesting. Andy ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2015-11-20 15:58 ` Andy Wingo @ 2016-02-17 7:31 ` 宋文武 2016-02-17 7:52 ` Jookia 2016-02-29 9:46 ` Ludovic Courtès 0 siblings, 2 replies; 14+ messages in thread From: 宋文武 @ 2016-02-17 7:31 UTC (permalink / raw) To: guix-devel Someone (sorry, I forget who) mentioned in #guix that we can use binfmt_misc to handle the shebang. I tried coding it in C, and it seems to working fine. Here it is: <https://github.com/iyzsong/duang>. Try it with care: --8<---------------cut here---------------start------------->8--- $ sudo mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc $ git clone https://github.com/iyzsong/duang.git $ cd duang $ make $ sudo make install $ sudo make register --8<---------------cut here---------------end--------------->8--- This will install the handler into `/libexec/@', and register it in `/proc/sys/fs/binfmt_misc/duang'. With it activated, intepreter in the shebang doesn't need to be absolute path, and can handle multiple arguments, eg: `#!emacs -Q --script'. And when the interpreter in absolute path is not exist, it will fallback to be searched in the PATH, so `#!/usr/bin/perl -w' equals to `#!perl -w'. On GuixSD, this means if we remove the `/bin/sh' symlink, the sh-scripts will fallbacked to use `sh' in PATH, so we can remove /bin. I dunno if it's a good idea though :-) ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2016-02-17 7:31 ` 宋文武 @ 2016-02-17 7:52 ` Jookia 2016-02-29 9:41 ` Ludovic Courtès 2016-02-29 9:46 ` Ludovic Courtès 1 sibling, 1 reply; 14+ messages in thread From: Jookia @ 2016-02-17 7:52 UTC (permalink / raw) To: 宋文武; +Cc: guix-devel On Wed, Feb 17, 2016 at 03:31:42PM +0800, 宋文武 wrote: > Someone (sorry, I forget who) mentioned in #guix that we can use > binfmt_misc to handle the shebang. I might have been me since I thought this would be a neat idea. > I tried coding it in C, and it seems to working fine. > Here it is: <https://github.com/iyzsong/duang>. > > Try it with care: > --8<---------------cut here---------------start------------->8--- > $ sudo mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc > $ git clone https://github.com/iyzsong/duang.git > $ cd duang > $ make > $ sudo make install > $ sudo make register > --8<---------------cut here---------------end--------------->8--- > > This will install the handler into `/libexec/@', and register it in > `/proc/sys/fs/binfmt_misc/duang'. With it activated, intepreter in > the shebang doesn't need to be absolute path, and can handle multiple > arguments, eg: `#!emacs -Q --script'. And when the interpreter in > absolute path is not exist, it will fallback to be searched in the PATH, > so `#!/usr/bin/perl -w' equals to `#!perl -w'. On GuixSD, this means > if we remove the `/bin/sh' symlink, the sh-scripts will fallbacked to > use `sh' in PATH, so we can remove /bin. > > I dunno if it's a good idea though :-) Does this handle /usr/bin/env though? ;) That said, I think it's the wrong approach since it's automated, invisible and hides the issues of /usr/bin/env. Instead we need to do two things: Write some scripts to ease migration and documentation on how to use Autotools or other tools to fix shebangs. This is portable and will work on all systems. Basically, find all shebangs, replace them with a macro that expands to the binary, move the file from 'name.sh' to 'name.in.sh' and then use './configure' or 'make' to build the 'name.sh' scripts with fixed shebangs. Write a script that patches shebangs in a directory tree and has support for unpatching them. This is useful for developing or running unpatched third party trees where you want to change the source code and submit patches without actually fixing the problem. Both of these solutions can help destroy /usr/bin/env upstream and downstream. Jookia. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2016-02-17 7:52 ` Jookia @ 2016-02-29 9:41 ` Ludovic Courtès 0 siblings, 0 replies; 14+ messages in thread From: Ludovic Courtès @ 2016-02-29 9:41 UTC (permalink / raw) To: Jookia; +Cc: guix-devel Jookia <166291@gmail.com> skribis: > That said, I think it's the wrong approach since it's automated, invisible and > hides the issues of /usr/bin/env. Instead we need to do two things: > > Write some scripts to ease migration and documentation on how to use Autotools > or other tools to fix shebangs. This is portable and will work on all systems. > Basically, find all shebangs, replace them with a macro that expands to the > binary, move the file from 'name.sh' to 'name.in.sh' and then use './configure' > or 'make' to build the 'name.sh' scripts with fixed shebangs. I think projects already do that (or at least, They Should Be Doing That.) That is, either any Bourne shell is fine, and they use #!/bin/sh, or they require Bash specifically, and they use AC_PATH_PROG and #!@BASH@ (same for Python, Perl, etc.) Now, reality is that there’s probably a lot of typically small packages that use hard-coded shebangs. > Write a script that patches shebangs in a directory tree and has support for > unpatching them. This is useful for developing or running unpatched third party > trees where you want to change the source code and submit patches without > actually fixing the problem. I think patching is perfect in our build environment (the ‘patch-shebangs’ and related phases), but probably inconvenient when working on another project where it may force you to carry a patch forever. ISTR that Andy mentioned working on WebKit or Chromium, which has hard-coded shebangs, and where patching things is annoying at best. The ‘binfmt_misc’ hack (or other hacks to that effect) can be helpful in those cases. Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2016-02-17 7:31 ` 宋文武 2016-02-17 7:52 ` Jookia @ 2016-02-29 9:46 ` Ludovic Courtès 1 sibling, 0 replies; 14+ messages in thread From: Ludovic Courtès @ 2016-02-29 9:46 UTC (permalink / raw) To: 宋文武; +Cc: guix-devel iyzsong@member.fsf.org (宋文武) skribis: > Someone (sorry, I forget who) mentioned in #guix that we can use > binfmt_misc to handle the shebang. > > I tried coding it in C, and it seems to working fine. > Here it is: <https://github.com/iyzsong/duang>. > > Try it with care: > $ sudo mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc > $ git clone https://github.com/iyzsong/duang.git > $ cd duang > $ make > $ sudo make install > $ sudo make register > > This will install the handler into `/libexec/@', and register it in > `/proc/sys/fs/binfmt_misc/duang'. With it activated, intepreter in > the shebang doesn't need to be absolute path, and can handle multiple > arguments, eg: `#!emacs -Q --script'. And when the interpreter in > absolute path is not exist, it will fallback to be searched in the PATH, > so `#!/usr/bin/perl -w' equals to `#!perl -w'. On GuixSD, this means > if we remove the `/bin/sh' symlink, the sh-scripts will fallbacked to > use `sh' in PATH, so we can remove /bin. > > I dunno if it's a good idea though :-) Nice hack! :-) The downside is that it requires scripts to use this feature, and requires users to have Duang installed and registered. I think what would be nice to handle the use case that Andy raised is a binfmt_misc handler that simply handles /usr/bin/env (and possibly /bin/sh), and to add an ‘operating-system’ knob to enable it. That way, third-party scripts would work automagically. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2015-11-20 14:00 ` Ludovic Courtès 2015-11-20 15:58 ` Andy Wingo @ 2016-02-29 18:11 ` Christopher Allan Webber 2016-03-01 8:09 ` Andy Wingo 2016-03-03 17:00 ` Ludovic Courtès 1 sibling, 2 replies; 14+ messages in thread From: Christopher Allan Webber @ 2016-02-29 18:11 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès writes: > Andy Wingo <wingo@igalia.com> skribis: > >> On Thu 19 Nov 2015 16:07, ludo@gnu.org (Ludovic Courtès) writes: > > [...] > >>>> Alternately, I am not sure if this would work but we could make a form >>>> of "guix environment" which populates a profile that is mounted at /usr >>>> in a container. That would allow many more non-Guix tools to run. >>> >>> Technically ‘guix environment --container’ could create /usr, just like >>> it creates /bin/sh. Not sure if it’s a good idea, though. >> >> I think it is definitely interesting. The reason being, you might hack >> on something or have to deploy something and it's not part of Guix -- >> you don't want to rewrite the shebang lines for files in git that aren't >> build products. Being able to make a just-FHS-enough environment inside >> a container sounds to me like a useful tool to have for shimming Guix >> and the outside world, while also benefitting from Guix's reproducible >> environments, rollbacks, isolation, and so on. > > Yeah, makes sense. > > One can already do: > > guix environment --container --ad-hoc coreutils \ > --expose=$(which env)=/usr/bin/env > > Should we add a --fhs or --/usr/bin/env option to simplify this? > > Thanks, > Ludo’. Would this possibly screw us up on Hurd, where container support I don't think exists? ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2016-02-29 18:11 ` Christopher Allan Webber @ 2016-03-01 8:09 ` Andy Wingo 2016-03-01 8:11 ` Jookia 2016-03-03 17:00 ` Ludovic Courtès 1 sibling, 1 reply; 14+ messages in thread From: Andy Wingo @ 2016-03-01 8:09 UTC (permalink / raw) To: Christopher Allan Webber; +Cc: guix-devel Hi :) > Ludovic Courtès writes: > >> Should we add a --fhs or --/usr/bin/env option to simplify this? I think /usr/bin/env is probably going to be insufficient -- lazy people like me are always going to be asking for more things :) If we can manage to expose a profile at /usr, that would be better, I think. On Mon 29 Feb 2016 19:11, Christopher Allan Webber <cwebber@dustycloud.org> writes: > Would this possibly screw us up on Hurd, where container support I don't > think exists? In general, I think we can have the Hurd inform but not gate Guix features. In this case, exposing a custom view of the file system to a sub-process is something that the Hurd should be able to do anyway :) Andy ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2016-03-01 8:09 ` Andy Wingo @ 2016-03-01 8:11 ` Jookia 0 siblings, 0 replies; 14+ messages in thread From: Jookia @ 2016-03-01 8:11 UTC (permalink / raw) To: Andy Wingo; +Cc: guix-devel On Tue, Mar 01, 2016 at 09:09:45AM +0100, Andy Wingo wrote: > Hi :) > > > Ludovic Courtès writes: > > > >> Should we add a --fhs or --/usr/bin/env option to simplify this? > > I think /usr/bin/env is probably going to be insufficient -- lazy people > like me are always going to be asking for more things :) If we can > manage to expose a profile at /usr, that would be better, I think. > > On Mon 29 Feb 2016 19:11, Christopher Allan Webber <cwebber@dustycloud.org> writes: > > > Would this possibly screw us up on Hurd, where container support I don't > > think exists? > > In general, I think we can have the Hurd inform but not gate Guix > features. > > In this case, exposing a custom view of the file system to a sub-process > is something that the Hurd should be able to do anyway :) We can somewhat do this with containers already, perhaps it'd be best to have hacks that run in containers? > Andy ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: thesis: guixsd should provide /usr/bin/env 2016-02-29 18:11 ` Christopher Allan Webber 2016-03-01 8:09 ` Andy Wingo @ 2016-03-03 17:00 ` Ludovic Courtès 1 sibling, 0 replies; 14+ messages in thread From: Ludovic Courtès @ 2016-03-03 17:00 UTC (permalink / raw) To: Christopher Allan Webber; +Cc: guix-devel Christopher Allan Webber <cwebber@dustycloud.org> skribis: > Ludovic Courtès writes: [...] >> One can already do: >> >> guix environment --container --ad-hoc coreutils \ >> --expose=$(which env)=/usr/bin/env >> >> Should we add a --fhs or --/usr/bin/env option to simplify this? >> >> Thanks, >> Ludo’. > > Would this possibly screw us up on Hurd, where container support I don't > think exists? ‘call-with-container’ currently doesn’t work on GNU/Hurd. So whatever we do, there’s no regression. :-) But I agree with Andy: I think it’s OK to add optional features that require the Linux kernel. Besides, ‘call-with-container’ can definitely be ported to GNU/Hurd, only using different APIs. After all, the Hurd had “containers” (lightweight virtualization) right from the beginning. Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2016-03-03 17:00 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-11-16 9:19 thesis: guixsd should provide /usr/bin/env Andy Wingo 2015-11-17 6:48 ` Ricardo Wurmus 2015-11-19 16:07 ` Ludovic Courtès 2015-11-19 17:35 ` Andy Wingo 2015-11-20 14:00 ` Ludovic Courtès 2015-11-20 15:58 ` Andy Wingo 2016-02-17 7:31 ` 宋文武 2016-02-17 7:52 ` Jookia 2016-02-29 9:41 ` Ludovic Courtès 2016-02-29 9:46 ` Ludovic Courtès 2016-02-29 18:11 ` Christopher Allan Webber 2016-03-01 8:09 ` Andy Wingo 2016-03-01 8:11 ` Jookia 2016-03-03 17:00 ` Ludovic Courtès
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.