Re: 01/01: services: Add ‘/usr/bin/env’ special file.

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]


guix-commits@gnu.org writes:

> nckx pushed a commit to branch master
> in repository guix.
>
> commit 3b38bf141a464e1bb370af7d2b2651d1efb29781
> Author: Tobias Geerinckx-Rice <me@tobias.gr>
> Date:   Fri Sep 6 12:23:57 2019 +0200
>
>     services: Add ‘/usr/bin/env’ special file.
>
>     * gnu/services/base.scm (%base-services): Add ‘/usr/bin/env‘ to
>     special-files-service-type.
> ---

This seems to me like quite a big change, and I'd be interested in
knowing what your motivation was [1]?

1: And ideally this would be in the commit message.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[pelzflorian (Florian Pelz)]

On Fri, Sep 06, 2019 at 12:36:33PM +0200, Christopher Baines wrote:
> 
> guix-commits@gnu.org writes:
> 
> > nckx pushed a commit to branch master
> > in repository guix.
> >
> > commit 3b38bf141a464e1bb370af7d2b2651d1efb29781
> > Author: Tobias Geerinckx-Rice <me@tobias.gr>
> > Date:   Fri Sep 6 12:23:57 2019 +0200
> >
> >     services: Add ‘/usr/bin/env’ special file.
> >
> >     * gnu/services/base.scm (%base-services): Add ‘/usr/bin/env‘ to
> >     special-files-service-type.
> > ---
> 
> This seems to me like quite a big change, and I'd be interested in
> knowing what your motivation was [1]?
> 
> 1: And ideally this would be in the commit message.


This break building the guix website, possibly also guix pull.

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[pelzflorian (Florian Pelz)]

On Fri, Sep 06, 2019 at 12:44:53PM +0200, pelzflorian (Florian Pelz) wrote:
> On Fri, Sep 06, 2019 at 12:36:33PM +0200, Christopher Baines wrote:
> > 
> > guix-commits@gnu.org writes:
> > 
> > > nckx pushed a commit to branch master
> > > in repository guix.
> > >
> > > commit 3b38bf141a464e1bb370af7d2b2651d1efb29781
> > > Author: Tobias Geerinckx-Rice <me@tobias.gr>
> > > Date:   Fri Sep 6 12:23:57 2019 +0200
> > >
> > >     services: Add ‘/usr/bin/env’ special file.
> > >
> > >     * gnu/services/base.scm (%base-services): Add ‘/usr/bin/env‘ to
> > >     special-files-service-type.
> > > ---
> > 
> > This seems to me like quite a big change, and I'd be interested in
> > knowing what your motivation was [1]?
> > 
> > 1: And ideally this would be in the commit message.
> 
> 
> This break building the guix website, possibly also guix pull.
> 

When building the website:

gnu/services/base.scm:2431:35: unquote: expression not valid outside of quasiquote in form (unquote (file-append (canonical-package coreutils) "/usr/bin/env"))

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 654 bytes --]

Christopher,

Christopher Baines 写道：
> This seems to me like quite a big change, and I'd be interested 
> in
> knowing what your motivation was [1]?

It's not, really.  It's equivalent to the impure /bin/sh that Guix 
Systems already provide, but actually useful: ‘use 
#!/usr/bin/env, not #!/bin/sh!’ was already a mantra when I wrote 
my first shell script — and I'm not that young.

There's no good argument for not being able to run the vast 
majority of well-written scripts, out of the box, on Guix Systems.

I hope that's sufficient, but I don't think it needed to go in the 
commit message.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Mark H Weaver]

Hi Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Christopher,
>
> Christopher Baines 写道：
>> This seems to me like quite a big change, and I'd be interested in
>> knowing what your motivation was [1]?
>
> It's not, really.  It's equivalent to the impure /bin/sh that Guix
> Systems already provide, but actually useful: ‘use #!/usr/bin/env, not
> #!/bin/sh!’ was already a mantra when I wrote my first shell script —
> and I'm not that young.
>
> There's no good argument for not being able to run the vast majority
> of well-written scripts, out of the box, on Guix Systems.
>
> I hope that's sufficient, but I don't think it needed to go in the
> commit message.

This should have been discussed more widely before pushing to 'master'.
It is not a new idea.  We talked about it long ago, and decided that it
should *not* be included.  I don't have the energy right now to restate
the arguments or find the old discussions.

       Mark

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Mark H Weaver]

Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Christopher Baines 写道：
>> This seems to me like quite a big change, and I'd be interested in
>> knowing what your motivation was [1]?
>
> It's not, really.  It's equivalent to the impure /bin/sh that Guix
> Systems already provide, but actually useful: ‘use #!/usr/bin/env, not
> #!/bin/sh!’ was already a mantra when I wrote my first shell script —
> and I'm not that young.
>
> There's no good argument for not being able to run the vast majority
> of well-written scripts, out of the box, on Guix Systems.

It shows some hubris for you to declare that "there's no good argument".
How did you come to that conclusion?  Did you assume that there must not
be any good argument against it because you couldn't think of one?

We discussed this long ago and decided against including /usr/bin/env.
Therefore, I reverted your commit for now.  If you like, we can start
another discussion about it, and I'll restate my arguments against it
when I find the energy.

      Thanks,
        Mark

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Jesse Gibbons]

On Fri, 2019-09-06 at 19:21 -0400, Mark H Weaver wrote:
> Hi Tobias,
> 
> Tobias Geerinckx-Rice <me@tobias.gr> writes:
> 
> > Christopher,
> > 
> > Christopher Baines 写道：
> > > This seems to me like quite a big change, and I'd be interested
> > > in
> > > knowing what your motivation was [1]?
> > 
> > It's not, really.  It's equivalent to the impure /bin/sh that Guix
> > Systems already provide, but actually useful: ‘use #!/usr/bin/env,
> > not
> > #!/bin/sh!’ was already a mantra when I wrote my first shell script
> > —
> > and I'm not that young.
> > 
> > There's no good argument for not being able to run the vast
> > majority
> > of well-written scripts, out of the box, on Guix Systems.
> > 
> > I hope that's sufficient, but I don't think it needed to go in the
> > commit message.
> 
> This should have been discussed more widely before pushing to
> 'master'.
> It is not a new idea.  We talked about it long ago, and decided that
> it
> should *not* be included.  I don't have the energy right now to
> restate
> the arguments or find the old discussions.
> 
>        Mark
> 
Here's a post with what I think is a good argument against adding
/usr/bin/env. I think the standard patch-shebang phase does a good job
at preventing the potential issue, but the argument still applies to
scripts generated by make, as I learned while attempting to port
pysolfc. I recommend you read the thread too.

https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00505.html

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Tobias Geerinckx-Rice via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 1030 bytes --]

Jesse,

Thanks!  It was linked from another thread[0] Ludo' pasted to my 
patch, though.  I've read both.

Jesse Gibbons 写道：
> Here's a post with what I think is a good argument against 
> adding
> /usr/bin/env. I think the standard patch-shebang phase does a 
> good job
> at preventing the potential issue, but the argument still 
> applies to

So is the argument here that Guix packages would generate scripts 
at run-time that refer to /usr/bin/env?  Or just that it's 
currently easy to catch packages that install #!/usr/bin/env 
scripts?  Or something else?

Neither /bin/sh nor /usr/bin/env are available in the build 
environment.  Relying on the *likely* run-time non-existence of 
/usr/bin/env in *user* environments to catch *packaging* bugs does 
not sound acceptable to me.

> scripts generated by make, as I learned while attempting to port
> pysolfc.

If you have the time, could you elaborate?

Kind regards,

T G-R

[0]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35910

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 1083 bytes --]

Mark,

I'm going to disregard the ad-hom; energy is indeed precious for 
all of us.  I am disappointed that you cho(o)se to respond in this 
manner, and after the fact.  Promptly reverting patches you 
disagree with is a privilege that few can afford.  An equally 
forceful response in May would have been much more appreciated.

Mark H Weaver 写道：
> How did you come to that conclusion?  Did you assume that there 
> must not
> be any good argument against it because you couldn't think of 
> one?

:-/  What do you think?

> We discussed this long ago and decided against including 
> /usr/bin/env.

Could you link to this decision?  My duckduckfoo is lacking, but 
please remember that I read the mailing lists too.

I am aware only of previous discussions[0][1] in which you didn't 
participate (there is a reference to a 2015 IRC comment but that's 
long since gc'd).

Kind regards,

T G-R

[0]: 
https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00414.html
[1]: 
https://lists.gnu.org/archive/html/guix-devel/2017-02/msg00205.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 696 bytes --]

Mark,

Mark H Weaver 写道：
> This should have been discussed more widely before pushing to 
> 'master'.

It should certainly have been discussed more widely before 
reverting like you did.  There was plenty of opportunity for you 
to respond before that[0].

I have the impression this is not going anywhere productive, so 
shall we just drop this meta-discussion?

Despite what you may think, I *am* interested in your (or any) 
reasoning to include /bin/sh and not /usr/bin/sh, but understand 
the energy involved in reiterating.  I'd rather see them both 
gone; this was my compromise.

Kind regards,

T G-R

[0]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35910

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1141 bytes --]

Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Christopher,
>
> Christopher Baines 写道：
>> This seems to me like quite a big change, and I'd be interested 
>> in
>> knowing what your motivation was [1]?
>
> It's not, really.  It's equivalent to the impure /bin/sh that Guix 
> Systems already provide, but actually useful: ‘use 
> #!/usr/bin/env, not #!/bin/sh!’ was already a mantra when I wrote 
> my first shell script — and I'm not that young.
>
> There's no good argument for not being able to run the vast 
> majority of well-written scripts, out of the box, on Guix Systems.

If you are on Guix System and find that you need /usr/bin/env, you are
already a "power user": you are venturing outside of what is provided by
Guix alone.

At that point you are expected to read the manual.  Grepping for
/usr/bin/env there will show you exactly what you need to do.  Or
perhaps you decide that it would be better to create a G-exp or package
for the script.

For running arbitrary scripts we have 'guix environment': perhaps it
would make sense to add /usr/bin/env to `guix environment --container`?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Jesse Gibbons]

On Sat, 2019-09-07 at 12:06 +0200, Tobias Geerinckx-Rice wrote:
> Mark,
> 
> Mark H Weaver 写道：
> > This should have been discussed more widely before pushing to 
> > 'master'.
> 
> It should certainly have been discussed more widely before 
> reverting like you did.  There was plenty of opportunity for you 
> to respond before that[0].
> 
> I have the impression this is not going anywhere productive, so 
> shall we just drop this meta-discussion?
> 
> Despite what you may think, I *am* interested in your (or any) 
> reasoning to include /bin/sh and not /usr/bin/sh, but understand 
> the energy involved in reiterating.  I'd rather see them both 
> gone; this was my compromise.
> 
> Kind regards,
> 
> T G-R
> 
> [0]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35910

If I might chip in here to try to make this discussion a little more
productive, a user suggested /usr/bin/env should be added by default[0]
to solve a problem[1]. In summary, the user wanted to have a standard
for scripting in guile and other common GNU distros. If including
/usr/bin/env by default is not the best solution to the problem,
perhaps we can find a better solution.

I suggest we add the "guix shebang" command, which takes a script and
returns a script with a shebang pointing to the proper source, like
what 'patch-shebangs build phase does to all the scripts in the build
source. It could replace the input script (perhaps when given the --
replace option) or it could put the resulting script in the store and
accept the --root= option.

Thoughts?

[0]: https://lists.gnu.org/archive/html/help-guix/2019-09/msg00037.html
[1]: https://lists.gnu.org/archive/html/help-guix/2019-09/msg00024.html

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Jesse Gibbons]

[-- Attachment #1: Type: text/plain, Size: 1671 bytes --]

Hi T-G-R,
On Sat, 2019-09-07 at 09:52 +0200, Tobias Geerinckx-Rice wrote:
> Jesse,
> 
> Thanks!  It was linked from another thread[0] Ludo' pasted to my 
> patch, though.  I've read both.
> 
> Jesse Gibbons 写道：
> > Here's a post with what I think is a good argument against 
> > adding
> > /usr/bin/env. I think the standard patch-shebang phase does a 
> > good job
> > at preventing the potential issue, but the argument still 
> > applies to
> 
> So is the argument here that Guix packages would generate scripts 
> at run-time that refer to /usr/bin/env?  Or just that it's 
> currently easy to catch packages that install #!/usr/bin/env 
> scripts?  Or something else?
> 
> Neither /bin/sh nor /usr/bin/env are available in the build 
> environment.  Relying on the *likely* run-time non-existence of 
> /usr/bin/env in *user* environments to catch *packaging* bugs does 
> not sound acceptable to me.
> 
> > scripts generated by make, as I learned while attempting to port
> > pysolfc.
> 
> If you have the time, could you elaborate?
It's a bit complicated.
Pysolfc has a horrible build system. It doesn't build the required i18n
unless you call "make test". Howevver, "make test" runs a script
generates python test scripts that use #!/usr/bin/env to find both
python and python2. Since they are generated by a shell script, patch-
shebangs does not catch them, and it fails to find /usr/bin/env. It's a
mess that I don't know how to solve.

I've attached my pysolfc.scm so you can observe what I mean. Sorry it's
a horrible mess of commented broken code.
> 
> Kind regards,
> 
> T G-R
> 
> [0]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35910

-- 
-Jesse

[-- Attachment #2: pysolfc.scm --]
[-- Type: text/x-scheme, Size: 5889 bytes --]

;;; Broken Guix
;;; Guix packages that are in-progress, broken, nonfree, or otherwise will
;;; not build and run to my satisfaction.
;;;
;;; Copyright (C) 2019 Jesse Gibbons
;;;
;;; This program is free software: you can redistribute it and/or modify
;;; it under the terms of the GNU General Public License as published by
;;; the Free Software Foundation, either version 3 of the License, or
;;; (at your option) any later version.
;;;
;;; This program is distributed in the hope that it will be useful,
;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
(define-module (broken-packages pysolfc)
  #:use-module (guix packages)
  #:use-module (guix download)
  #:use-module (guix build-system python)
  #:use-module (guix licenses)
  #:use-module (guix gexp)
  #:use-module (gnu packages python)
  #:use-module (gnu packages python-xyz)
  #:use-module (gnu packages compression)
  #:use-module (gnu packages base)
  #:use-module (gnu packages perl)
  )

(define-public pysolfc
  (package
   (name "pysolfc")
   (version "2.6.4")
   (source
    (origin
     (method url-fetch)
     (uri (string-append
	   "https://github.com/shlomif/PySolFC/archive/pysolfc-"
	   version
	   ".tar.gz"))
     (sha256
      (base32
       "17r9mbn4fj6kbxhllsab74gfjac0j2mjdwkkwaxp6cqpy4dss3z8"))))
   (build-system python-build-system)
   (native-inputs
     `(("make" ,gnu-make)
       ("perl" ,perl)
       ("python2" ,python-2)
       ("moose" ,perl-moose)
       ("coreutils" ,coreutils)
       ("python2-pycotap" ,python2-pycotap)
       ))
    (propagated-inputs
     `(
       ("python2-six"  ,python2-six)
       ("python2-tkinter" ,python-2 "tk")
       ("python2-random2" ,python2-random2)
       ("python2" ,python-2)
       ("python-six"  ,python-six)
       ("python-tkinter" ,python "tk")
       ("python-random2" ,python-random2)))
   (arguments
    `(#:python ,python
      #:phases (modify-phases %standard-phases
			      ;; (add-before 'patch-generated-file-shebangs 'generate-tests
			      ;; 		 (lambda _
			      ;; 		   (begin
			      ;; 		     (invoke "make" "pretest")
			      ;; 		     (system "echo =================")
			      ;; 		     (invoke "echo" (string-append "#!" (which "env")))
			      ;; 		     (substitute* (find-files "tests" "\\.py$")
			      ;; 				  (("#!/usr/bin/env")
			      ;; 				   (string-append "#!" (which "env"))))
			      ;; 		     )))
			      ;; (add-before 'build 'make-test
			      ;; 		 (lambda _
			      ;; 		   (begin
			      ;; 		     (system "cat tests/unit-generated/*")
			      ;; 		     (invoke "false")
			      ;; 		     (invoke "make" "test")
			      ;; 		     )))
			      (add-before 'build 'make-rules
					  (lambda _
					    (begin
					      (invoke "make" "rules"))))
			      (add-after 'make-rules 'move-images
					 (lambda _
					   (begin
					     (invoke "false")
					     #t
					     ))))))
   
   (home-page "https://pysolfc.sourceforge.io/")
   (synopsis
    "Solitaire Collection, Written in Python")
   (description
    "PySol Fan Club Edition (PySolFC) is a collection of more than 1000 solitaire card games. It is a fork of PySol Solitaire.")
   (license gpl3+)))

(define python-random2
  (package
   (name "python-random2")
   (version "1.0.1")
   (source
    (origin
     (method url-fetch)
     (uri (pypi-uri "random2" version ".zip"))
     (sha256
      (base32
       "01y0s4747plsx8fdnxy0nz83dp69naddz58m81r9h0s1qfm31b9l"))))
   (native-inputs
    `(("unzip" ,unzip)))
   (build-system python-build-system)
   (arguments
    `(#:python ,python
      #:use-setuptools? #f))
   (home-page "http://pypi.python.org/pypi/random2")
   (synopsis
    "Python 3 compatible Pytohn 2 `random` Module.")
   (description
    "Python 3 compatible Pytohn 2 `random` Module.")
   (license #f)))

(define python2-random2
  (package
   (name "python2-random2")
   (version "1.0.1")
   (source
    (origin
     (method url-fetch)
     (uri (pypi-uri "random2" version ".zip"))
     (sha256
      (base32
       "01y0s4747plsx8fdnxy0nz83dp69naddz58m81r9h0s1qfm31b9l"))))
   (native-inputs
    `(("unzip" ,unzip)))
   (build-system python-build-system)
   (arguments
    `(#:python ,python-2
      #:use-setuptools? #f))
   (home-page "http://pypi.python.org/pypi/random2")
   (synopsis
    "Python 3 compatible Pytohn 2 `random` Module.")
   (description
    "Python 3 compatible Pytohn 2 `random` Module.")
   (license #f)))

(define-public python-pycotap
  (package
   (name "python-pycotap")
   (version "1.1.0")
   (source
    (origin
     (method url-fetch)
     (uri (pypi-uri "pycotap" version))
     (sha256
      (base32
       "128qn7zjn95nivcxbxjclkwjw2qif5sf9c1b8rrsczcpn78kckf1"))))
   (inputs `(("python" ,python)))
   (build-system python-build-system)
   (arguments
    `(#:python ,python))
   (home-page "https://el-tramo.be/pycotap")
   (synopsis
    "A tiny test runner that outputs TAP results to standard output.")
   (description
    "A tiny test runner that outputs TAP results to standard output.")
   (license expat)))

(define-public python2-pycotap
  (package
   (name "python2-pycotap")
   (version "1.1.0")
   (source
    (origin
     (method url-fetch)
     (uri (pypi-uri "pycotap" version))
     (sha256
      (base32
       "128qn7zjn95nivcxbxjclkwjw2qif5sf9c1b8rrsczcpn78kckf1"))))
   (build-system python-build-system)
   (arguments
    `(#:python ,python-2))
   (home-page "https://el-tramo.be/pycotap")
   (synopsis
    "A tiny test runner that outputs TAP results to standard output.")
   (description
    "A tiny test runner that outputs TAP results to standard output.")
   (license expat)))

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Ricardo Wurmus]

Hey Marius,

> If you are on Guix System and find that you need /usr/bin/env, you are
> already a "power user": you are venturing outside of what is provided by
> Guix alone.

I don’t follow this argument.

Using a custom script with a /usr/bin/env shebang is pretty common.  You
don’t need to be a power user for that, and certainly not a *Guix* power
user.

On the other hand I’d like to point out that pretty much all defaults we
provide in system services can be overridden — and that’s where it helps
being an advanced user.  This is true for things like the desktop
services (which aim to make setting up a generic desktop system easy and
non-surprising), but also for every other service.

Personally, I think it’s a good idea to default to having /usr/bin/env
shebangs just work on Guix Systems.

The argument that /usr/bin/env could make software work by accident when
testing on a Guix System is not very convincing to me.  We don’t have
/bin/sh or /usr/bin/env in the build environment.  Software behaviour is
also affected by the presence of /usr, /lib, /bin, etc, and these can
all exist at runtime.  We assume that building in an isolated
environment is usually sufficient; and yet we sometimes find that
applications behave differently when run inside of containers
(e.g. applications that call out to coreutils that are usually available
in a normal system).

With the same reasoning we could argue against having coreutils on PATH.
This is not an exaggeration: R depended on coreutils to be on PATH at
runtime and we only found out when we ran it in a container without
coreutils.

Am I missing something?

--
Ricardo

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Konrad Hinsen]

Hi Ricardo and everyone else,

> Using a custom script with a /usr/bin/env shebang is pretty common.  You
> don’t need to be a power user for that, and certainly not a *Guix* power
> user.

I definitely agree with this. In my work environment, it is very common 
for people to distribute shell or Python scripts with a /usr/bin/env 
shebang line, together with the instructions "make executable and put on 
your $PATH". Most of my colleagues can handle those instructions, but 
wouldn't consider themselves power users.

Konrad.

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Hartmut Goebel]

Am 08.09.19 um 13:55 schrieb Konrad Hinsen:
> Hi Ricardo and everyone else,
>
>> Using a custom script with a /usr/bin/env shebang is pretty common.  You
>> don’t need to be a power user for that, and certainly not a *Guix* power
>> user.
>
> I definitely agree with this. In my work environment, it is very
> common for people to distribute shell or Python scripts with a
> /usr/bin/env shebang line,  […]

I also support adding a ‘/usr/bin/env’ service and even make it a default.

In my daily work, I have many scripts which do not care about which
version of sh/bash/python/python3 is used. These scripts are placed in
some (small) project's directory and there is absolutely no need for
"installing" them.

Another use-case is when developing software: E.g. scripts for
maintaining the vcs-contolled files, like updating the copyright year.
These scripts are managed by the version control system, shall run from
the development tree on envery developers machine and may not even get
installed when installing the software as a guix package. 

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Ludovic Courtès]

Hi,

Jesse Gibbons <jgibbons2357@gmail.com> skribis:

> If I might chip in here to try to make this discussion a little more
> productive, a user suggested /usr/bin/env should be added by default[0]
> to solve a problem[1]. In summary, the user wanted to have a standard
> for scripting in guile and other common GNU distros. If including
> /usr/bin/env by default is not the best solution to the problem,
> perhaps we can find a better solution.
>
> I suggest we add the "guix shebang" command, which takes a script and
> returns a script with a shebang pointing to the proper source, like
> what 'patch-shebangs build phase does to all the scripts in the build
> source. It could replace the input script (perhaps when given the --
> replace option) or it could put the resulting script in the store and
> accept the --root= option.

Would “guix shebang” modify the script, or would it be used as the
shebang?

Either way, I’m not sure it’d really solve the initial use case very
well (the initial request was to be able to run scripts unmodified,
AIUI.)

Thanks,
Ludo’.

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Ludovic Courtès]

Hi,

Ricardo Wurmus <rekado@elephly.net> skribis:

> Using a custom script with a /usr/bin/env shebang is pretty common.  You
> don’t need to be a power user for that, and certainly not a *Guix* power
> user.

Like I wrote in <https://issues.guix.gnu.org/issue/35910#2> and in the
message it refers to, although I was initially mildly reluctant to
having /usr/bin/env by default, I’ve come to think that lack of
/usr/bin/env is a gratuitous annoyance—not to me of course, but to
newcomers as we’ve seen repeatedly, be they seasoned GNU/Linux users or
not.

With that in mind, adding /usr/bin/env by default is probably a good move.

Now, we can add a snippet in the manual with the ‘modify-services’ trick
to remove /usr/bin/env.  :-)

> The argument that /usr/bin/env could make software work by accident when
> testing on a Guix System is not very convincing to me.  We don’t have
> /bin/sh or /usr/bin/env in the build environment.  Software behaviour is
> also affected by the presence of /usr, /lib, /bin, etc, and these can
> all exist at runtime.  We assume that building in an isolated
> environment is usually sufficient; and yet we sometimes find that
> applications behave differently when run inside of containers
> (e.g. applications that call out to coreutils that are usually available
> in a normal system).

Yeah.

Well anyway, if we take a step back, we’re talking about a really tiny
issue in the grand scheme of things, and it’s certainly not worth losing
our hair over it.  :-)

Thanks,
Ludo’.

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]

Guix,

For the record, I've since restored this commit on master.

The Guix project is certainly not lacking in mailing lists or 
other communication channels; I'd appreciate it if the git commit 
history weren't (ab)used as such.

Thanks,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Jesse Gibbons]

Hello,
On Sun, 2019-09-08 at 23:48 +0200, Ludovic Courtès wrote:
> Hi,
> 
> Jesse Gibbons <jgibbons2357@gmail.com> skribis:
> 
> > If I might chip in here to try to make this discussion a little
> > more
> > productive, a user suggested /usr/bin/env should be added by
> > default[0]
> > to solve a problem[1]. In summary, the user wanted to have a
> > standard
> > for scripting in guile and other common GNU distros. If including
> > /usr/bin/env by default is not the best solution to the problem,
> > perhaps we can find a better solution.
> > 
> > I suggest we add the "guix shebang" command, which takes a script
> > and
> > returns a script with a shebang pointing to the proper source, like
> > what 'patch-shebangs build phase does to all the scripts in the
> > build
> > source. It could replace the input script (perhaps when given the
> > --
> > replace option) or it could put the resulting script in the store
> > and
> > accept the --root= option.
> 
> Would “guix shebang” modify the script, or would it be used as the
> shebang?
I suppose it could work either way. Since the scripts are intended to
be modified by the audience[1], it would be easier to have a command to
run as little as possible. On GuixSD, that would be as often as "guix
pull && guix upgrade" updates the program that runs the script; on
other systems it would be unnecessary. So modifying the script itself
would make the process simpler.

[1]https://lists.gnu.org/archive/html/help-guix/2019-09/msg00001.html
> 
> Either way, I’m not sure it’d really solve the initial use case very
> well (the initial request was to be able to run scripts unmodified,
> AIUI.)
I see two solutions to the problem:
1. Run the scripts unmodified (ideal). This can only be accomplished if
the shebang can rely on a program in a standard location (i.e.
/usr/bin/env). Adding /usr/bin/env by default currently is up for
debate in this thread.

2. Patch the shebang with a command that can be run as rarely as
possible, ideally only once. If we keep /usr/bin/env out of %standard-
services or %de then this is the last solution I can think of. It could
be an inline awk or sed script (like the scripts throughout LFS), which
would have to be pulled out every time a user has new script, or a guix
command which would always be available to all users with this
particular problem. Guix package is an immediate choice, but was
rejected because it would make things more commplicated[1], not to
mention simpler scripts are probably shorter than a package definition.
The simplest solution here would be something like "guix shebang".

[1] https://lists.gnu.org/archive/html/help-guix/2019-09/msg00001.html

I have no preference which solution is used, but if /usr/bin/env is not
available in a build environment, and we can find a way to detect calls
to programs in standard locations and patch them when we build them, I
know of no reason not to include /usr/bin/env by default.

If anyone has a fourth solution, I would love to see it.

> Thanks,
> Ludo’.

-- 
-Jesse

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2028 bytes --]

Hi everyone,

Ricardo Wurmus <rekado@elephly.net> writes:

> Using a custom script with a /usr/bin/env shebang is pretty common.  You
> don’t need to be a power user for that, and certainly not a *Guix* power
> user.
>
> [...]
>
> Personally, I think it’s a good idea to default to having /usr/bin/env
> shebangs just work on Guix Systems.
>
> [...]
>
> With the same reasoning we could argue against having coreutils on PATH.
> This is not an exaggeration: R depended on coreutils to be on PATH at
> runtime and we only found out when we ran it in a container without
> coreutils.

I agree.  At first, I thought I could do without /usr/bin/env, but I
always end up installing it because scripts that others share with me
frequently use it, and it's just too painful to have to patch them all
or package up those scripts in Guix package definitions.  I encounter
scrips like this frequently enough that I've decided I want
/usr/bin/env.

Ludovic Courtès <ludo@gnu.org> writes:

> Like I wrote in <https://issues.guix.gnu.org/issue/35910#2> and in the
> message it refers to, although I was initially mildly reluctant to
> having /usr/bin/env by default, I’ve come to think that lack of
> /usr/bin/env is a gratuitous annoyance—not to me of course, but to
> newcomers as we’ve seen repeatedly, be they seasoned GNU/Linux users or
> not.
>
> With that in mind, adding /usr/bin/env by default is probably a good move.
>
> Now, we can add a snippet in the manual with the ‘modify-services’ trick
> to remove /usr/bin/env.  :-)
>
> [...]
>
> Well anyway, if we take a step back, we’re talking about a really tiny
> issue in the grand scheme of things, and it’s certainly not worth losing
> our hair over it.  :-)

I feel the same way.

Regardless of how we came to discuss it, I'm glad we've discussed the
issue in this email thread, and I'm glad to hear that I'm not alone in
changing my mind about /usr/bin/env.  It's good to have it by default.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Bengt Richter]

On +2019-09-09 00:07:10 +0200, Ludovic Courtès wrote:
> Hi,
> 
> Ricardo Wurmus <rekado@elephly.net> skribis:
> 
> > Using a custom script with a /usr/bin/env shebang is pretty common.  You
> > don’t need to be a power user for that, and certainly not a *Guix* power
> > user.
> 
> Like I wrote in <https://issues.guix.gnu.org/issue/35910#2> and in the
> message it refers to, although I was initially mildly reluctant to
> having /usr/bin/env by default, I’ve come to think that lack of
> /usr/bin/env is a gratuitous annoyance—not to me of course, but to
> newcomers as we’ve seen repeatedly, be they seasoned GNU/Linux users or
> not.
>
> With that in mind, adding /usr/bin/env by default is probably a good move.
>

Hi Ludo,

I may be one of those (over-) seasoned (past best-before) users you mention :)
-- but with all due respect, is this not a compromise of the fundamental
idea of guix _transactional_ package management? Or did I misunderstand?

I thought the idea was that each mod to the system drew one unique
transaction-arc from current to next state (whether the mod was by
guix install, guix pull, or guix whatever) -- with the ideal goal
of being able to walk the graph transactionally back to any other
state, with no loose ends.

ISTM that the current state defining the behaviour of my system includes
the various config files such as ~/.bash_profile and ~/.guix-profile and
~/.emacs.d/* and all the rest, that programs read to condition their behaviour.

Not only that, but the whole deep tree of references used -- including
scripts and programs called in the body of scripts, not just in the
hash-bang line.

Does that not mean that, ideally :) the particular state of such
referenced entities ought to be captured in a closure belonging
to a particular generation, for that generation's sovereign use?

Implementation optimization is another matter. Things that in practice
don't change much could be shared COW entities, I guess?

I certainly agree with the goal of being able to share scripts without
manual changes, such as what a friend might attach to or include in an email,
or what one might copy/paste from a browser view of gitlab contents, etc.

But not at the price of fundamental compromises :)

Could emacs grow an "M-x pack-region-as" command to produce something
that could be installed with guix install, automatically taking care
of name collisions with existing foo to install foo-from-origin-mnemonic?

Then modding your system would produce a proper generation and could
be controlled.

If there are no tools to do things safely with pure transactions the
result will IME (doing it to myself :) be an unholy mess of unpredictable
future error messages with no easy way to figure out what happened, and
lots of rewrite work to make everything play nicely together for real.

IMO "works" "most of the time" is not a good rationale for compromising
fundamental principles.

I see this as a version of the pythonic (see python -c 'import this') argument
that "pacticality beats purity". Yes it does, but IMO _only_ in emergencies --
because if left to persist, each emergency hack adds to an eventual rats-nest of
tangled dependencies for which there is no "revert" but painful manual analysis
and re-implemention.

BTW, is there a guixian version of "python -c import this" ? "guix describe this"? ;-)

> Now, we can add a snippet in the manual with the ‘modify-services’ trick
> to remove /usr/bin/env.  :-)
> 
> > The argument that /usr/bin/env could make software work by accident when
> > testing on a Guix System is not very convincing to me.  We don’t have
> > /bin/sh or /usr/bin/env in the build environment.  Software behaviour is
> > also affected by the presence of /usr, /lib, /bin, etc, and these can
> > all exist at runtime.  We assume that building in an isolated
> > environment is usually sufficient; and yet we sometimes find that
> > applications behave differently when run inside of containers
> > (e.g. applications that call out to coreutils that are usually available
> > in a normal system).
> 
> Yeah.
> 
> Well anyway, if we take a step back, we’re talking about a really tiny
> issue in the grand scheme of things, and it’s certainly not worth losing
> our hair over it.  :-)
> 
> Thanks,
> Ludo’.
>

Here follows an example of a script one might receive in an email from a friend ;-)

What automation could be brought to bear to include this safely and transactionally
into your system to try? As a tool that could be used by a sender or by the receiver.

It shows unicode information about characters in its command line arguments
or piped in split by whitespace, e.g., (with control char for good measure :)

Invoked like:
unicode-info Ludovic Courtès $(echo -e "\x07")

"Ludovic":
    glyph  codepoint .....int  name...
    _L_     +U00004c       76  LATIN CAPITAL LETTER L  
    _u_     +U000075      117  LATIN SMALL LETTER U  
    _d_     +U000064      100  LATIN SMALL LETTER D  
    _o_     +U00006f      111  LATIN SMALL LETTER O  
    _v_     +U000076      118  LATIN SMALL LETTER V  
    _i_     +U000069      105  LATIN SMALL LETTER I  
    _c_     +U000063       99  LATIN SMALL LETTER C  

"Courtès":
    glyph  codepoint .....int  name...
    _C_     +U000043       67  LATIN CAPITAL LETTER C  
    _o_     +U00006f      111  LATIN SMALL LETTER O  
    _u_     +U000075      117  LATIN SMALL LETTER U  
    _r_     +U000072      114  LATIN SMALL LETTER R  
    _t_     +U000074      116  LATIN SMALL LETTER T  
    _è_     +U0000e8      232  LATIN SMALL LETTER E WITH GRAVE  
    _s_     +U000073      115  LATIN SMALL LETTER S  

"\a":
    glyph  codepoint .....int  name...
    _^G_    +U000007        7  ASCII bel, aka #\alarm

I used /usr/bin/env in the hash-bang which let me use the -S option
(which I wonder if guile couldn't be taught to emulate if called
directly instead of via env, BTW)

Sorry if this is an inappropriate way to pass on a jelly-bean...
Regards,
Bengt Richter
Oh, gpl3+ on the following, forgot to edit it in ;-/
------------------------------------------------
#!/usr/bin/env -S guile -e unicode-info -s
!#

(use-modules (ice-9 unicode))
(use-modules (ice-9 format))
(use-modules (ice-9 regex))
(use-modules (ice-9 rdelim))
;;(use-modules (ice-9 textual-ports))

;; <ESC> 1 <ESC> ! printf -v cc "\\\x%02x" {0..32};echo -ne "$cc"|od -a|cut -d ' ' -f2-
;; nul soh stx etx eot enq ack bel  bs  ht  nl  vt  ff  cr  so  si
;; dle dc1 dc2 dc3 dc4 nak syn etb can  em sub esc  fs  gs  rs  us
;;  sp
;; 0000041

;;;; ctl-names from od -a -- see above <ESC>... emacs shell command to capture names
(define ctl-names (map cons
		       (iota 33)
		       (map match:substring (list-matches "[a-z]+" (string-append  
	"nul soh stx etx eot enq ack bel  bs  ht  nl  vt  ff  cr  so  si "
	"dle dc1 dc2 dc3 dc4 nak syn etb can  em sub esc  fs  gs  rs  us  sp " )))))

(define (show-char c)
  (begin
    (let*((c_int (char->integer c))
	  (glyph_c (format #f "~:c" c)))
      (cond
       ((char<? c #\040) ;;(char=? c #\177));; ascii control
	(begin
	  (let*((c_name (cdr (assv c_int ctl-names))))
	    (format #t
	       ;    glyph  codepoint .....int  name...	   
	       "    _~:c_    +U~6,'0X ~8,d  ASCII ~a, aka ~:s\n"
	       c c_int c_int c_name c ))))
       (else
	(begin
	  (let*((glyph_a (format #f "_~c_" c))
		(c_formal (char->formal-name c))
		(glyph_a (if (not c_formal) "n/a" glyph_a))
		(big_glyph (>= (string-length glyph_a) 4))
		(glyph_out (if big_glyph "see->" glyph_a)))
	    (begin
	      (format #t
		 ;    glyph  codepoint .....int  name...	   
		 "    ~4,a    +U~6,'0X ~8,d  ~a  \n" glyph_out c_int c_int  c_formal)))))))))

(define (show-str s)
  (begin
    (format #t "\n~s:\n    glyph  codepoint .....int  name...\n" s)
    (map show-char (string->list s))))

(define (strings-from-readlines p)
  (let lp ((line (read-delimited "\n" p 'concat)) (slist '()))
    (if (eof-object? line)
	slist
	(lp (read-delimited "\n" p 'concat)
	    (append slist (map match:substring (list-matches "([ \t\f\n]*|[^ \t\f\n]+)[\n]?" line)))))))

(define (unicode-info args)
  (let*((as (cdr args))
	(ss (if (pair? as)
		as
		(strings-from-readlines (current-input-port)))))
    (map show-str ss)))
------------------------------------------------

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Ludovic Courtès]

Hi Bengt,

Bengt Richter <bokr@bokr.com> skribis:

> On +2019-09-09 00:07:10 +0200, Ludovic Courtès wrote:

[...]

>> Like I wrote in <https://issues.guix.gnu.org/issue/35910#2> and in the
>> message it refers to, although I was initially mildly reluctant to
>> having /usr/bin/env by default, I’ve come to think that lack of
>> /usr/bin/env is a gratuitous annoyance—not to me of course, but to
>> newcomers as we’ve seen repeatedly, be they seasoned GNU/Linux users or
>> not.
>>
>> With that in mind, adding /usr/bin/env by default is probably a good move.
>>
>
> Hi Ludo,
>
> I may be one of those (over-) seasoned (past best-before) users you mention :)
> -- but with all due respect, is this not a compromise of the fundamental
> idea of guix _transactional_ package management?

No it’s not.  It really doesn’t change anything.

First, you can always opt out—the discussion was about whether it should
remain opt-in, or whether it should become opt-out.

Second, and more importantly: packages (“derivations”) are still built
in an isolated environment that does not and will never contain
/usr/bin/env.

Sorry I’m not replying to your other points now, but I hope it clarifies
things!

Thanks,
Ludo’.