[bug#27438] [PATCH] Specify native search path for all ruby packages

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Christopher Baines]

[-- Attachment #1.1: Type: text/plain, Size: 135 bytes --]

Without specifying this explicitly in each definition, the GEM_PATH is
inherited and the version is that of the inherited package.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 858 bytes --]

[bug#27438] [PATCH 1/3] gnu: ruby-1.8: Fix search path specification.

[Christopher Baines]

* gnu/packages/ruby.scm (ruby-1.8)[native-search-paths]: Specify the native
  search path, such that the version included in the path is 1.8, rather than
  the inherited version.
---
 gnu/packages/ruby.scm | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 7680f4eae..d9eb93643 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -197,7 +197,13 @@ a focus on simplicity and productivity.")
                           "lib/mkmf.rb"
                           "process.c")
              (("/bin/sh") (which "sh"))))
-         %standard-phases)))))
+         %standard-phases)))
+    (native-search-paths
+     (list (search-path-specification
+            (variable "GEM_PATH")
+            (files (list (string-append "lib/ruby/gems/"
+                                        (version-major+minor version)
+                                        ".0"))))))))
 
 (define-public ruby-hoe
   (package
-- 
2.13.1

[bug#27438] [PATCH 2/3] gnu: ruby-2.2: Fix search path specification.

[Christopher Baines]

* gnu/packages/ruby.scm (ruby-2.2)[native-search-paths]: Specify the native
  search path, such that the version included in the path is 2.2, rather than
  the inherited version.
---
 gnu/packages/ruby.scm | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index d9eb93643..098024fc0 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -134,7 +134,13 @@ a focus on simplicity and productivity.")
                            "/ruby-" version ".tar.xz"))
        (sha256
         (base32
-         "0lyb7gnbbhs3a3v9grsjgbaixm20wxz6x3h0czyrxnj3cpp8lk13"))))))
+         "0lyb7gnbbhs3a3v9grsjgbaixm20wxz6x3h0czyrxnj3cpp8lk13"))))
+    (native-search-paths
+     (list (search-path-specification
+            (variable "GEM_PATH")
+            (files (list (string-append "lib/ruby/gems/"
+                                        (version-major+minor version)
+                                        ".0"))))))))
 
 (define-public ruby-2.1
   (package (inherit ruby)
-- 
2.13.1

[bug#27438] [PATCH 3/3] gnu: ruby-2.3: Fix search path specification.

[Christopher Baines]

* gnu/packages/ruby.scm (ruby-2.3)[native-search-paths]: Specify the native
  search path, such that the version included in the path is 2.3, rather than
  the inherited version.
---
 gnu/packages/ruby.scm | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 098024fc0..69835a771 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -120,7 +120,13 @@ a focus on simplicity and productivity.")
        (snippet `(begin
                    ;; Remove bundled libffi
                    (delete-file-recursively "ext/fiddle/libffi-3.2.1")
-                   #t))))))
+                   #t))))
+    (native-search-paths
+     (list (search-path-specification
+            (variable "GEM_PATH")
+            (files (list (string-append "lib/ruby/gems/"
+                                        (version-major+minor version)
+                                        ".0"))))))))
 
 (define-public ruby-2.2
   (package (inherit ruby)
-- 
2.13.1

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Ben Woodcroft]

Hi Chris,


On 21/06/17 16:36, Christopher Baines wrote:
> Without specifying this explicitly in each definition, the GEM_PATH is
> inherited and the version is that of the inherited package.
>
I'm not sure if this is by design, but the version of the gems folder is 
embedded in the build of each rubygem e.g. 'ruby-hoe' includes
/gnu/store/d867l5i2dqd5qnq4qlsrcwwb0x3443fl-ruby-hoe-3.16.0/lib/ruby/gems/2.4.0

So, in this environment, "require 'hoe'" fails after applying your patch
./pre-inst-env guix environment -C --ad-hoc ruby@2.2 ruby-hoe

Having said that, the gems bundled with each ruby are in a directory 
with the actual version e.g. ruby@2.2 has this path
/gnu/store/f43dirk254lfhmgjs3sf515cpfl8d46q-ruby-2.2.7/lib/ruby/gems/2.2.0

This isn't ideal, any ideas on what we should do? Specify both paths maybe?
Thanks,
ben

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Ludovic Courtès]

Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

> On 21/06/17 16:36, Christopher Baines wrote:
>> Without specifying this explicitly in each definition, the GEM_PATH is
>> inherited and the version is that of the inherited package.
>>
> I'm not sure if this is by design, but the version of the gems folder
> is embedded in the build of each rubygem e.g. 'ruby-hoe' includes
> /gnu/store/d867l5i2dqd5qnq4qlsrcwwb0x3443fl-ruby-hoe-3.16.0/lib/ruby/gems/2.4.0

Or should the search path spec include both lib/ruby/gems/2.2.0 and
lib/ruby/gems/2.4.0, in this order?

Ludo’, who has no experience with Ruby.

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Ben Woodcroft]

On 21/06/17 23:12, Ludovic Courtès wrote:
> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>
>> On 21/06/17 16:36, Christopher Baines wrote:
>>> Without specifying this explicitly in each definition, the GEM_PATH is
>>> inherited and the version is that of the inherited package.
>>>
>> I'm not sure if this is by design, but the version of the gems folder
>> is embedded in the build of each rubygem e.g. 'ruby-hoe' includes
>> /gnu/store/d867l5i2dqd5qnq4qlsrcwwb0x3443fl-ruby-hoe-3.16.0/lib/ruby/gems/2.4.0
> Or should the search path spec include both lib/ruby/gems/2.2.0 and
> lib/ruby/gems/2.4.0, in this order?
Exactly.

Chris, what is your experience? Did you propose the patch because you 
ran into a particular issue?

Ta, ben.

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Christopher Baines]

[-- Attachment #1.1: Type: text/plain, Size: 1589 bytes --]

On 22/06/17 06:27, Ben Woodcroft wrote:
> On 21/06/17 23:12, Ludovic Courtès wrote:
>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>
>>> On 21/06/17 16:36, Christopher Baines wrote:
>>>> Without specifying this explicitly in each definition, the GEM_PATH is
>>>> inherited and the version is that of the inherited package.
>>>>
>>> I'm not sure if this is by design, but the version of the gems folder
>>> is embedded in the build of each rubygem e.g. 'ruby-hoe' includes
>>> /gnu/store/d867l5i2dqd5qnq4qlsrcwwb0x3443fl-ruby-hoe-3.16.0/lib/ruby/gems/2.4.0
>>>
>> Or should the search path spec include both lib/ruby/gems/2.2.0 and
>> lib/ruby/gems/2.4.0, in this order?
> Exactly.
> 
> Chris, what is your experience? Did you propose the patch because you
> ran into a particular issue?

Yep, I ran in to problems trying to use the guix ruby-2.3 package with
the guix bundler package, when I build bundler with ruby-2.3.

Ben's email got me thinking about how this works in Debian, and it looks
like Debian uses a different location /usr/lib/ruby/vendor_ruby/ .

I think there might be benefits from doing similarly, but this needs a
bit of thought and testing, as I'm unsure how this might work,
especially in cases where libraries include native code that links
against ruby.

I've got a patch for the ruby-build-system to make a change roughly like
this, and I'll send that up soon. Relating this back to the issue at
hand, moving to a version independent directory would mean that the
GEM_PATH wouldn't be version specific.

Thanks,

Chris


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 858 bytes --]

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 4583 bytes --]

On Thu, 22 Jun 2017 06:40:17 +0100
Christopher Baines <mail@cbaines.net> wrote:

> On 22/06/17 06:27, Ben Woodcroft wrote:
> > On 21/06/17 23:12, Ludovic Courtès wrote:  
> >> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
> >>  
> >>> On 21/06/17 16:36, Christopher Baines wrote:  
> >>>> Without specifying this explicitly in each definition, the
> >>>> GEM_PATH is inherited and the version is that of the inherited
> >>>> package. 
> >>> I'm not sure if this is by design, but the version of the gems
> >>> folder is embedded in the build of each rubygem e.g. 'ruby-hoe'
> >>> includes /gnu/store/d867l5i2dqd5qnq4qlsrcwwb0x3443fl-ruby-hoe-3.16.0/lib/ruby/gems/2.4.0
> >>>  
> >> Or should the search path spec include both lib/ruby/gems/2.2.0 and
> >> lib/ruby/gems/2.4.0, in this order?  
> > Exactly.
> > 
> > Chris, what is your experience? Did you propose the patch because
> > you ran into a particular issue?  
> 
> Yep, I ran in to problems trying to use the guix ruby-2.3 package with
> the guix bundler package, when I build bundler with ruby-2.3.
> 
> Ben's email got me thinking about how this works in Debian, and it
> looks like Debian uses a different
> location /usr/lib/ruby/vendor_ruby/ .
> 
> I think there might be benefits from doing similarly, but this needs a
> bit of thought and testing, as I'm unsure how this might work,
> especially in cases where libraries include native code that links
> against ruby.
> 
> I've got a patch for the ruby-build-system to make a change roughly
> like this, and I'll send that up soon. Relating this back to the
> issue at hand, moving to a version independent directory would mean
> that the GEM_PATH wouldn't be version specific.

So... I kind of forgot about this, sorry about that. But I'll send up
the patch shortly.

As an example of what this change means, here is a example of loading
ruby gems with different versions of ruby, with the current and
alternative situations.

I think arel doesn't use native extensions, whereas nokogiri does.
Also, ruby@2.1 is used, as the search path for the guix package
reflects the version of ruby (whereas many others don't).

* Current

** With the version of ruby used to build the gem

→ guix environment --pure --ad-hoc ruby-arel ruby -- ruby -e "puts require 'arel'"
true

(I'm pretty sure the "true" means that it worked)

→ guix environment --pure --ad-hoc ruby-nokogiri ruby -- ruby -e "puts require 'nokogiri'"
true

** With a different version of ruby

→ guix environment --pure --ad-hoc ruby-arel ruby@2.1 -- ruby -e "puts require 'arel'"
/gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- arel (LoadError)
	from /gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from -e:1:in `<main>'

→ guix environment --pure --ad-hoc ruby-arel ruby@2.1 -- ruby -e "puts require 'nokogiri'"
/gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- nokogiri (LoadError)
	from /gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from -e:1:in `<main>'


* Alternative

** With the version of ruby used to build the gem

→ guix environment --pure --ad-hoc ruby-arel ruby -- ruby -e "puts require 'arel'"
true

→ guix environment --pure --ad-hoc ruby-nokogiri ruby -- ruby -e "puts require 'nokogiri'"
true

** With a different version of ruby

→ guix environment --pure --ad-hoc ruby-arel ruby@2.1 -- ruby -e "puts require 'arel'"
true

→ guix environment --pure --ad-hoc ruby-nokogiri ruby@2.1 -- ruby -e "puts require 'nokogiri'"
/gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/fileutils.rb:250:in `mkdir': Read-only file system @ dir_s_mkdir - /gnu/store/jxmi5lr8dbll3pzlzwkafpd9mj1wwxma-profile/lib/ruby/vendor_ruby/extensions/x86_64-linux/2.1.0-static (Errno::EROFS)
	from /gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/fileutils.rb:250:in `fu_mkdir'
        ...


So, putting the gems in a single location regardless of the version of
ruby they were built with means that a different version of ruby will
at least see them, however, it may still fail to load them.

I think this is an improvement, but I'm very uncertain about ruby. Does
anyone else have opinions on this?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[bug#27438] [PATCH] guix: build: ruby-build-system: Install to the vendor directory

[Christopher Baines]

From: Christopher Baines <christopher.baines@digital.cabinet-office.gov.uk>

* guix/build/ruby-build-system.scm (install): Install gems to the vendor
  directory, rather than the GEM_HOME. The vendor directory does not include
  the version of ruby used to install the gem in the path, which makes it
  easier to add it to the GEM_PATH for all versions of ruby to use.
* gnu/packages/ruby.scm (ruby,ruby-2.1)[native-search-paths]: Switch to
  lib/ruby/vendor_ruby.
---
 gnu/packages/ruby.scm            | 12 ++----------
 guix/build/ruby-build-system.scm | 38 +++++++++++++++++++++++---------------
 2 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 7eba68444..f634d5e71 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -93,9 +93,7 @@
     (native-search-paths
      (list (search-path-specification
             (variable "GEM_PATH")
-            (files (list (string-append "lib/ruby/gems/"
-                                        (version-major+minor version)
-                                        ".0"))))))
+            (files (list (string-append "lib/ruby/vendor_ruby"))))))
     (synopsis "Programming language interpreter")
     (description "Ruby is a dynamic object-oriented programming language with
 a focus on simplicity and productivity.")
@@ -159,13 +157,7 @@ a focus on simplicity and productivity.")
                           "lib/mkmf.rb"
                           "process.c")
              (("/bin/sh") (which "sh"))))
-         %standard-phases)))
-    (native-search-paths
-     (list (search-path-specification
-            (variable "GEM_PATH")
-            (files (list (string-append "lib/ruby/gems/"
-                                        (version-major+minor version)
-                                        ".0"))))))))
+         %standard-phases)))))
 
 (define-public ruby-1.8
   (package (inherit ruby)
diff --git a/guix/build/ruby-build-system.scm b/guix/build/ruby-build-system.scm
index c2d276627..f4ce2e5d8 100644
--- a/guix/build/ruby-build-system.scm
+++ b/guix/build/ruby-build-system.scm
@@ -129,40 +129,48 @@ GEM-FLAGS are passed to the 'gem' invokation, if present."
                                          (assoc-ref inputs "ruby"))
                            1))
          (out (assoc-ref outputs "out"))
-         (gem-home (string-append out "/lib/ruby/gems/" ruby-version ".0"))
+         (vendor-dir (string-append out "/lib/ruby/vendor_ruby"))
          (gem-file (first-matching-file "\\.gem$"))
          (gem-file-basename (basename gem-file))
          (gem-name (substring gem-file-basename
                               0
-                              (- (string-length gem-file-basename) 4)))
-         (gem-directory (string-append gem-home "/gems/" gem-name)))
-    (setenv "GEM_HOME" gem-home)
-    (mkdir-p gem-home)
-    (and (apply system* "gem" "install" gem-file
-                "--local" "--ignore-dependencies"
-                ;; Executables should go into /bin, not /lib/ruby/gems.
-                "--bindir" (string-append out "/bin")
-                gem-flags)
+                              (- (string-length gem-file-basename) 4))))
+    (setenv "GEM_VENDOR" vendor-dir)
+    (and (let ((install-succeeded?
+                (zero?
+                 (apply system* "gem" "install" gem-file
+                        "--local" "--ignore-dependencies" "--vendor"
+                        ;; Executables should go into /bin, not
+                        ;; /lib/ruby/gems.
+                        "--bindir" (string-append out "/bin")
+                        gem-flags))))
+           (or install-succeeded?
+               (begin
+                 (simple-format #t "installation failed\n")
+                 (let ((failed-output-dir (string-append (getcwd) "/out")))
+                   (mkdir failed-output-dir)
+                   (copy-recursively out failed-output-dir))
+                 #f)))
          (begin
            ;; Remove the cached gem file as this is unnecessary and contains
            ;; timestamped files rendering builds not reproducible.
-           (let ((cached-gem (string-append gem-home "/cache/" gem-file)))
+           (let ((cached-gem (string-append vendor-dir "/cache/" gem-file)))
              (log-file-deletion cached-gem)
              (delete-file cached-gem))
            ;; For gems with native extensions, several Makefile-related files
            ;; are created that contain timestamps or other elements making
            ;; them not reproducible.  They are unnecessary so we remove them.
-           (if (file-exists? (string-append gem-directory "/ext"))
+           (if (file-exists? (string-append vendor-dir "/ext"))
                (begin
                  (for-each (lambda (file)
                              (log-file-deletion file)
                              (delete-file file))
                            (append
-                            (find-files (string-append gem-home "/doc")
+                            (find-files (string-append vendor-dir "/doc")
                                         "page-Makefile.ri")
-                            (find-files (string-append gem-home "/extensions")
+                            (find-files (string-append vendor-dir "/extensions")
                                         "gem_make.out")
-                            (find-files (string-append gem-directory "/ext")
+                            (find-files (string-append vendor-dir "/ext")
                                         "Makefile")))))
            #t))))
 
-- 
2.13.1

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Ben Woodcroft]

Hi Chris,


On 17/07/17 03:37, Christopher Baines wrote:
> On Thu, 22 Jun 2017 06:40:17 +0100
> Christopher Baines <mail@cbaines.net> wrote:
>
>> On 22/06/17 06:27, Ben Woodcroft wrote:
>>> On 21/06/17 23:12, Ludovic Courtès wrote:
>>>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>>>   
>>>>> On 21/06/17 16:36, Christopher Baines wrote:
>>>>>> Without specifying this explicitly in each definition, the
>>>>>> GEM_PATH is inherited and the version is that of the inherited
>>>>>> package.
>>>>> I'm not sure if this is by design, but the version of the gems
>>>>> folder is embedded in the build of each rubygem e.g. 'ruby-hoe'
>>>>> includes /gnu/store/d867l5i2dqd5qnq4qlsrcwwb0x3443fl-ruby-hoe-3.16.0/lib/ruby/gems/2.4.0
>>>>>   
>>>> Or should the search path spec include both lib/ruby/gems/2.2.0 and
>>>> lib/ruby/gems/2.4.0, in this order?
>>> Exactly.
>>>
>>> Chris, what is your experience? Did you propose the patch because
>>> you ran into a particular issue?
>> Yep, I ran in to problems trying to use the guix ruby-2.3 package with
>> the guix bundler package, when I build bundler with ruby-2.3.
>>
>> Ben's email got me thinking about how this works in Debian, and it
>> looks like Debian uses a different
>> location /usr/lib/ruby/vendor_ruby/ .
>>
>> I think there might be benefits from doing similarly, but this needs a
>> bit of thought and testing, as I'm unsure how this might work,
>> especially in cases where libraries include native code that links
>> against ruby.
>>
>> I've got a patch for the ruby-build-system to make a change roughly
>> like this, and I'll send that up soon. Relating this back to the
>> issue at hand, moving to a version independent directory would mean
>> that the GEM_PATH wouldn't be version specific.
> [..]
>
> So, putting the gems in a single location regardless of the version of
> ruby they were built with means that a different version of ruby will
> at least see them, however, it may still fail to load them.
>
> I think this is an improvement, but I'm very uncertain about ruby. Does
> anyone else have opinions on this?
Thanks for working on this. A few thoughts:

What happens to the default gems that come bundled with ruby itself? I'm 
interpreting from your patch that these will not be available?

In general, except for some special circumstances, we don't support old 
versions of software. To fix the issue that you are encountering 
properly with nokogiri probably requires new package definitions using 
"package-with-ruby-2.3" or similar to be made, I suppose. Ludo did some 
nice work making this easier (see 
https://lists.gnu.org/archive/html/guix-patches/2017-04/msg00126.html), 
but I worry in general about the resources required to support older 
Ruby versions. WDYT?

The gem-home procedure in ruby-build-system will now return an incorrect 
path, I think.

Perhaps I'm slow, but what are the advantages of the "vendor_ruby" 
method over exporting multiple GEM_PATHs as Ludo and I suggested? 
Changing the directory seems like a heavier touch and so more likely to 
misbehave. WDYT?

Apologies if this comes across a little negative, I'm just trying to 
consider all the issues. Thanks again for your efforts on this.
ben

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 5264 bytes --]

On Thu, 20 Jul 2017 09:39:13 +1000
Ben Woodcroft <b.woodcroft@uq.edu.au> wrote:

> Hi Chris,
> 
> 
> On 17/07/17 03:37, Christopher Baines wrote:
> > On Thu, 22 Jun 2017 06:40:17 +0100
> > Christopher Baines <mail@cbaines.net> wrote:
> >  
> >> On 22/06/17 06:27, Ben Woodcroft wrote:  
> >>> On 21/06/17 23:12, Ludovic Courtès wrote:  
> >>>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
> >>>>     
> >>>>> On 21/06/17 16:36, Christopher Baines wrote:  
> >>>>>> Without specifying this explicitly in each definition, the
> >>>>>> GEM_PATH is inherited and the version is that of the inherited
> >>>>>> package.  
> >>>>> I'm not sure if this is by design, but the version of the gems
> >>>>> folder is embedded in the build of each rubygem e.g. 'ruby-hoe'
> >>>>> includes /gnu/store/d867l5i2dqd5qnq4qlsrcwwb0x3443fl-ruby-hoe-3.16.0/lib/ruby/gems/2.4.0
> >>>>>     
> >>>> Or should the search path spec include both lib/ruby/gems/2.2.0
> >>>> and lib/ruby/gems/2.4.0, in this order?  
> >>> Exactly.
> >>>
> >>> Chris, what is your experience? Did you propose the patch because
> >>> you ran into a particular issue?  
> >> Yep, I ran in to problems trying to use the guix ruby-2.3 package
> >> with the guix bundler package, when I build bundler with ruby-2.3.
> >>
> >> Ben's email got me thinking about how this works in Debian, and it
> >> looks like Debian uses a different
> >> location /usr/lib/ruby/vendor_ruby/ .
> >>
> >> I think there might be benefits from doing similarly, but this
> >> needs a bit of thought and testing, as I'm unsure how this might
> >> work, especially in cases where libraries include native code that
> >> links against ruby.
> >>
> >> I've got a patch for the ruby-build-system to make a change roughly
> >> like this, and I'll send that up soon. Relating this back to the
> >> issue at hand, moving to a version independent directory would mean
> >> that the GEM_PATH wouldn't be version specific.  
> > [..]
> >
> > So, putting the gems in a single location regardless of the version
> > of ruby they were built with means that a different version of ruby
> > will at least see them, however, it may still fail to load them.
> >
> > I think this is an improvement, but I'm very uncertain about ruby.
> > Does anyone else have opinions on this?  
> Thanks for working on this. A few thoughts:
> 
> What happens to the default gems that come bundled with ruby itself?
> I'm interpreting from your patch that these will not be available?

They seem to be:

→ guix environment --container --ad-hoc ruby -- irb
...
for 1 packages... done in 0.028 s 
irb(main):001:0> ENV
=> {"PS1"=>"\\u@\\h \\w [env]\\$ ", "TMPDIR"=>"/tmp",
"TEMPDIR"=>"/tmp", "TMP"=>"/tmp", "TEMP"=>"/tmp",
"HOME"=>"/home/chris",
"PATH"=>"/gnu/store/2dh34pi9yjq17qf1bw3dmgcdpri20fjs-profile/bin",
"GEM_PATH"=>"/gnu/store/2dh34pi9yjq17qf1bw3dmgcdpri20fjs-profile/lib/ruby/vendor_ruby",
"GUIX_ENVIRONMENT"=>"/gnu/store/2dh34pi9yjq17qf1bw3dmgcdpri20fjs-profile",
"LINES"=>"76", "COLUMNS"=>"190"}
irb(main):003:0> require 'rake'
=> true
irb(main):004:0> require 'minitest'
=> true


> In general, except for some special circumstances, we don't support
> old versions of software. To fix the issue that you are encountering 
> properly with nokogiri probably requires new package definitions
> using "package-with-ruby-2.3" or similar to be made, I suppose. Ludo
> did some nice work making this easier (see 
> https://lists.gnu.org/archive/html/guix-patches/2017-04/msg00126.html), 
> but I worry in general about the resources required to support older 
> Ruby versions. WDYT?

I'm not aware of any particular problems if you are working with the
package definitions in Guile, as it should be possible to make them use
the single ruby version that you want.

With the guix environment command I posted:

  guix environment --pure --ad-hoc ruby-nokogiri ruby@2.1 -- ruby -e
  "puts require 'nokogiri'"

It would be ideal if there was some way of telling guix environment to
rewrite the package definitions of all packages to use ruby@2.1 in
place of whatever ruby they might be using.

> The gem-home procedure in ruby-build-system will now return an
> incorrect path, I think.

Good spot. I'll have a look at that.

> Perhaps I'm slow, but what are the advantages of the "vendor_ruby" 
> method over exporting multiple GEM_PATHs as Ludo and I suggested? 
> Changing the directory seems like a heavier touch and so more likely
> to misbehave. WDYT?

I agree that it is heavier in some sense, but I like the simplicity of
getting rid of the version from the path.

The best documentation I've found for this is the NEWS of the release
where it was added [1]. While Guix blurs the lines between the "package
system" and the "user", using this vendor directory might come in
useful.

1: http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS

> Apologies if this comes across a little negative, I'm just trying to 
> consider all the issues. Thanks again for your efforts on this.

No need to apologise, I too think its important to consider the
details of the support for ruby, which is not something I'm
particularly knowledgeable on.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Ben Woodcroft]

Hi Chris, sorry for the delay on this.


On 22/07/17 20:06, Christopher Baines wrote:
> On Thu, 20 Jul 2017 09:39:13 +1000
> Ben Woodcroft <b.woodcroft@uq.edu.au> wrote:
>
>> Hi Chris,
>>
>>
>> [..]
>>
>> What happens to the default gems that come bundled with ruby itself?
>> I'm interpreting from your patch that these will not be available?
> They seem to be:
OK, excellent.

>> In general, except for some special circumstances, we don't support
>> old versions of software. To fix the issue that you are encountering
>> properly with nokogiri probably requires new package definitions
>> using "package-with-ruby-2.3" or similar to be made, I suppose. Ludo
>> did some nice work making this easier (see
>> https://lists.gnu.org/archive/html/guix-patches/2017-04/msg00126.html),
>> but I worry in general about the resources required to support older
>> Ruby versions. WDYT?
> I'm not aware of any particular problems if you are working with the
> package definitions in Guile, as it should be possible to make them use
> the single ruby version that you want.
>
> With the guix environment command I posted:
>
>    guix environment --pure --ad-hoc ruby-nokogiri ruby@2.1 -- ruby -e
>    "puts require 'nokogiri'"
>
> It would be ideal if there was some way of telling guix environment to
> rewrite the package definitions of all packages to use ruby@2.1 in
> place of whatever ruby they might be using.
Is "package-mapping" sufficient?

[..]

>> Perhaps I'm slow, but what are the advantages of the "vendor_ruby"
>> method over exporting multiple GEM_PATHs as Ludo and I suggested?
>> Changing the directory seems like a heavier touch and so more likely
>> to misbehave. WDYT?
> I agree that it is heavier in some sense, but I like the simplicity of
> getting rid of the version from the path.
>
> The best documentation I've found for this is the NEWS of the release
> where it was added [1]. While Guix blurs the lines between the "package
> system" and the "user", using this vendor directory might come in
> useful.
>
> 1: http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS
Ah, OK. I hadn't realised there was support for this baked into Ruby 
itself. Seems obvious in hindsight.

If all Ruby dependencies build with this change, then the change seems 
reasonable to me, details aside.
ben

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3022 bytes --]

On Sat, 5 Aug 2017 13:59:56 +1000
Ben Woodcroft <b.woodcroft@uq.edu.au> wrote:

> Hi Chris, sorry for the delay on this.

No problem :)

> On 22/07/17 20:06, Christopher Baines wrote:
> > On Thu, 20 Jul 2017 09:39:13 +1000
> > Ben Woodcroft <b.woodcroft@uq.edu.au> wrote:
> >  
> >> Hi Chris,
> >>
> >>
> >> [..]
> >>
> >> What happens to the default gems that come bundled with ruby
> >> itself? I'm interpreting from your patch that these will not be
> >> available?  
> > They seem to be:  
> OK, excellent.
> 
> >> In general, except for some special circumstances, we don't support
> >> old versions of software. To fix the issue that you are
> >> encountering properly with nokogiri probably requires new package
> >> definitions using "package-with-ruby-2.3" or similar to be made, I
> >> suppose. Ludo did some nice work making this easier (see
> >> https://lists.gnu.org/archive/html/guix-patches/2017-04/msg00126.html),
> >> but I worry in general about the resources required to support
> >> older Ruby versions. WDYT?  
> > I'm not aware of any particular problems if you are working with the
> > package definitions in Guile, as it should be possible to make them
> > use the single ruby version that you want.
> >
> > With the guix environment command I posted:
> >
> >    guix environment --pure --ad-hoc ruby-nokogiri ruby@2.1 -- ruby
> > -e "puts require 'nokogiri'"
> >
> > It would be ideal if there was some way of telling guix environment
> > to rewrite the package definitions of all packages to use ruby@2.1
> > in place of whatever ruby they might be using.  
> Is "package-mapping" sufficient?

I don't think so. The ruby used is in the case of the ruby-build-system
is an argument to the build system, so you need to traverse part of the
dependency graph, altering the arguments of packages using the
ruby-build-system.

Or, perhaps do the transformation at a lower level abstraction than the
package record...

> >> Perhaps I'm slow, but what are the advantages of the "vendor_ruby"
> >> method over exporting multiple GEM_PATHs as Ludo and I suggested?
> >> Changing the directory seems like a heavier touch and so more
> >> likely to misbehave. WDYT?  
> > I agree that it is heavier in some sense, but I like the simplicity
> > of getting rid of the version from the path.
> >
> > The best documentation I've found for this is the NEWS of the
> > release where it was added [1]. While Guix blurs the lines between
> > the "package system" and the "user", using this vendor directory
> > might come in useful.
> >
> > 1: http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS  
> Ah, OK. I hadn't realised there was support for this baked into Ruby 
> itself. Seems obvious in hindsight.
> 
> If all Ruby dependencies build with this change, then the change
> seems reasonable to me, details aside.

Ok, does anyone know a good process for testing if lots of packages
build? I think I've heard of Hydra building branches?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Ben Woodcroft]

Hi,

[..]
>>>> In general, except for some special circumstances, we don't support
>>>> old versions of software. To fix the issue that you are
>>>> encountering properly with nokogiri probably requires new package
>>>> definitions using "package-with-ruby-2.3" or similar to be made, I
>>>> suppose. Ludo did some nice work making this easier (see
>>>> https://lists.gnu.org/archive/html/guix-patches/2017-04/msg00126.html),
>>>> but I worry in general about the resources required to support
>>>> older Ruby versions. WDYT?
>>> I'm not aware of any particular problems if you are working with the
>>> package definitions in Guile, as it should be possible to make them
>>> use the single ruby version that you want.
>>>
>>> With the guix environment command I posted:
>>>
>>>     guix environment --pure --ad-hoc ruby-nokogiri ruby@2.1 -- ruby
>>> -e "puts require 'nokogiri'"
>>>
>>> It would be ideal if there was some way of telling guix environment
>>> to rewrite the package definitions of all packages to use ruby@2.1
>>> in place of whatever ruby they might be using.
>> Is "package-mapping" sufficient?
> I don't think so. The ruby used is in the case of the ruby-build-system
> is an argument to the build system, so you need to traverse part of the
> dependency graph, altering the arguments of packages using the
> ruby-build-system.
>
> Or, perhaps do the transformation at a lower level abstraction than the
> package record...
For most compiled gems, I think it should be possible to write some 
procedure so that this would provide you with what you want:

guix environment --ad-hoc ruby@2.1 
--environment='(package-with-explicit-ruby ruby-2.1 ruby-nokogiri)'

For this specific case though, ruby-2.1 might not be the best example, 
since it is explicitly not supported any more.
>> If all Ruby dependencies build with this change, then the change
>> seems reasonable to me, details aside.
> Ok, does anyone know a good process for testing if lots of packages
> build? I think I've heard of Hydra building branches?
I believe we can go by the old standby "guix refresh -l ruby", so 659 
packages are rebuilt. Given this it would be appropriate for the staging 
branch, due to be merged after core-updates.

ben

[bug#27438] [PATCH] Specify native search path for all ruby packages

[Ludovic Courtès]

Christopher Baines <mail@cbaines.net> skribis:

> Without specifying this explicitly in each definition, the GEM_PATH is
> inherited and the version is that of the inherited package.

All 3 patches LGTM, thank you!

Ludo’.