[PATCH] Add rubygems updater.

[PATCH] Add rubygems updater.

[Ben Woodcroft]

[-- Attachment #1: Type: text/plain, Size: 2965 bytes --]

Phew, you almost beat me to the first patch of the year there Pjotr..

It seems there's 30 packages to be updated, out of the 107 in ruby.scm. 
Going through each of these individually seems a little tedious, can we 
do them in bulk somehow or do they have to be committed individually? 
Building and testing all packages that require these packages would be a 
start - is there any way to list all dependent packages?

gnu/packages/ruby.scm:2807:13: ruby-cutest would be upgraded from 1.2.2 
to 1.2.3
gnu/packages/ruby.scm:333:13: ruby-rspec-mocks would be upgraded from 
3.2.1 to 3.4.0
gnu/packages/ruby.scm:1527:13: ruby-daemons would be upgraded from 1.2.2 
to 1.2.3
gnu/packages/ruby.scm:1581:13: ruby-slop would be upgraded from 4.1.0 to 
4.2.1
gnu/packages/ruby.scm:2707:13: ruby-cucumber-core would be upgraded from 
1.3.0 to 1.3.1
gnu/packages/ruby.scm:1486:13: ruby-minitest-sprint would be upgraded 
from 1.1.0 to 1.1.1
gnu/packages/ruby.scm:2221:13: ruby-rb-fsevent would be upgraded from 
0.9.6 to 0.9.7
gnu/packages/ruby.scm:1339:13: ruby-redcarpet would be upgraded from 
3.3.3 to 3.3.4
gnu/packages/ruby.scm:1427:13: ruby-net-ssh would be upgraded from 3.0.1 
to 3.0.2
gnu/packages/ruby.scm:2241:13: ruby-listen would be upgraded from 3.0.3 
to 3.0.5
gnu/packages/ruby.scm:1124:13: ruby-gettext would be upgraded from 3.1.7 
to 3.2.0
gnu/packages/ruby.scm:1765:13: ruby-pry would be upgraded from 0.10.1 to 
0.10.3
gnu/packages/ruby.scm:2467:13: ruby-pg would be upgraded from 0.18.2 to 
0.18.4
gnu/packages/ruby.scm:967:13: ruby-useragent would be upgraded from 
0.13.3 to 0.16.3
gnu/packages/ruby.scm:299:13: ruby-rspec-expectations would be upgraded 
from 3.2.1 to 3.4.0
gnu/packages/ruby.scm:1629:13: ruby-arel would be upgraded from 6.0.3 to 
7.0.0
gnu/packages/ruby.scm:797:13: ruby-notiffany would be upgraded from 
0.0.7 to 0.0.8
gnu/packages/ruby.scm:2684:13: ruby-gherkin3 would be upgraded from 
3.1.1 to 3.1.2
gnu/packages/ruby.scm:162:13: ruby-hoe would be upgraded from 3.13.1 to 
3.14.2
gnu/packages/ruby.scm:452:13: ruby-rjb would be upgraded from 1.5.3 to 1.5.4
gnu/packages/ruby.scm:367:13: ruby-rspec would be upgraded from 3.2.0 to 
3.4.0
gnu/packages/ruby.scm:940:13: ruby-simplecov would be upgraded from 
0.10.0 to 0.11.1
gnu/packages/ruby.scm:1448:13: ruby-minitest would be upgraded from 
5.7.0 to 5.8.3
gnu/packages/ruby.scm:1964:13: ruby-tins would be upgraded from 1.7.0 to 
1.8.1
gnu/packages/ruby.scm:2263:13: ruby-activesupport would be upgraded from 
4.2.4 to 4.2.5
gnu/packages/ruby.scm:1689:13: ruby-nokogiri would be upgraded from 
1.6.6.2 to 1.6.7.1
gnu/packages/ruby.scm:407:13: bundler would be upgraded from 1.10.6 to 
1.11.2
gnu/packages/ruby.scm:2405:13: ruby-ox would be upgraded from 2.2.1 to 2.2.3
gnu/packages/ruby.scm:246:13: ruby-rspec-core would be upgraded from 
3.2.3 to 3.4.1
gnu/packages/ruby.scm:2493:13: ruby-byebug would be upgraded from 6.0.2 
to 8.2.1


Thanks in advance for the review.
ben

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-import-gem-Add-updater.patch --]
[-- Type: text/x-patch; name="0001-import-gem-Add-updater.patch", Size: 6188 bytes --]

From 4f53236278eb75b3e7f89562ae675becf3033679 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft <donttrustben@gmail.com>
Date: Fri, 1 Jan 2016 16:56:07 +1000
Subject: [PATCH] import: gem: Add updater.

* guix/import/gem.scm (guix-package->gem-name,
  gem-package?, latest-release): New procedures.
  (%gem-updater): New variable.
  (rubygems-fetch): Wrap body in
  'call-with-output-file' and 'with-error-to-port'.
* guix/scripts/refresh.scm (%updaters): Add %GEM-UPDATER.
* doc/guix.texi (Invoking guix refresh): Mention RubyGems.
---
 doc/guix.texi            |  2 ++
 guix/import/gem.scm      | 63 +++++++++++++++++++++++++++++++++++++++++++++---
 guix/scripts/refresh.scm |  5 +++-
 3 files changed, 66 insertions(+), 4 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index a70fbe8..008fcf1 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -4416,6 +4416,8 @@ the updater for @uref{http://elpa.gnu.org/, ELPA} packages;
 the updater for @uref{http://cran.r-project.org/, CRAN} packages;
 @item pypi
 the updater for @uref{https://pypi.python.org, PyPI} packages.
+@item gem
+the updater for @uref{https://rubygems.org, RubyGems} packages.
 @end table
 
 For instance, the following commands only checks for updates of Emacs
diff --git a/guix/import/gem.scm b/guix/import/gem.scm
index c64c4e9..bb25dc8 100644
--- a/guix/import/gem.scm
+++ b/guix/import/gem.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2015 David Thompson <davet@gnu.org>
+;;; Copryight © 2016 Ben Woodcroft <donttrustben@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -19,21 +20,33 @@
 (define-module (guix import gem)
   #:use-module (ice-9 match)
   #:use-module (ice-9 pretty-print)
+  #:use-module (srfi srfi-1)
   #:use-module (rnrs bytevectors)
   #:use-module (json)
   #:use-module (web uri)
+  #:use-module ((guix download) #:prefix download:)
   #:use-module (guix import utils)
   #:use-module (guix import json)
   #:use-module (guix packages)
+  #:use-module (guix upstream)
   #:use-module (guix licenses)
   #:use-module (guix base32)
-  #:export (gem->guix-package))
+  #:use-module (guix build-system ruby)
+  #:use-module (gnu packages)
+  #:export (gem->guix-package
+            %gem-updater))
 
 (define (rubygems-fetch name)
   "Return an alist representation of the RubyGems metadata for the package NAME,
 or #f on failure."
-  (json-fetch
-   (string-append "https://rubygems.org/api/v1/gems/" name ".json")))
+  ;; XXX: We want to silence the download progress report, which is especially
+  ;; annoying for 'guix refresh', but we have to use a file port.
+  (call-with-output-file "/dev/null"
+    (lambda (null)
+      (with-error-to-port null
+        (lambda ()
+          (json-fetch
+           (string-append "https://rubygems.org/api/v1/gems/" name ".json")))))))
 
 (define (ruby-package-name name)
   "Given the NAME of a package on RubyGems, return a Guix-compliant name for
@@ -130,3 +143,47 @@ VERSION, HASH, HOME-PAGE, DESCRIPTION, DEPENDENCIES, and LICENSES."
                                   (assoc-ref package "licenses"))))
            (make-gem-sexp name version hash home-page
                           description dependencies licenses)))))
+
+(define (guix-package->gem-name package)
+  "Given a PACKAGE built from rubygems.org, return the name of the
+package on RubyGems."
+  (let ((source-url (and=> (package-source package) origin-uri)))
+    ;; The URL has the form:
+    ;; 'https://rubygems.org/downloads/' +
+    ;; package name + '-' + version + '.gem'
+    ;; e.g. "https://rubygems.org/downloads/hashery-2.1.1.gem"
+    (substring source-url 31 (string-rindex source-url #\-))))
+
+(define (gem-package? package)
+  "Return true if PACKAGE is a gem package from RubyGems."
+
+  (define (rubygems-url? url)
+    (string-prefix? "https://rubygems.org/downloads/" url))
+
+  (let ((source-url (and=> (package-source package) origin-uri))
+        (fetch-method (and=> (package-source package) origin-method)))
+    (and (eq? fetch-method download:url-fetch)
+         (match source-url
+           ((? string?)
+            (rubygems-url? source-url))
+           ((source-url ...)
+            (any rubygems-url? source-url))))))
+
+(define (latest-release guix-package)
+  "Return an <upstream-source> for the latest release of GUIX-PACKAGE."
+  (let* ((gem-name (guix-package->gem-name
+                    (specification->package guix-package)))
+         (metadata (rubygems-fetch gem-name))
+         (version (assoc-ref metadata "version"))
+         (url (rubygems-uri gem-name version)))
+    (upstream-source
+     (package guix-package)
+     (version version)
+     (urls (list url)))))
+
+(define %gem-updater
+  (upstream-updater
+   (name 'gem)
+   (description "Updater for RubyGem packages")
+   (pred gem-package?)
+   (latest latest-release)))
diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index a5834d1..b2274ff 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
 ;;; Copyright © 2014 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2015 Alex Kost <alezost@gmail.com>
+;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -34,6 +35,7 @@
                 #:select (%gnu-updater %gnome-updater))
   #:use-module (guix import elpa)
   #:use-module (guix import cran)
+  #:use-module (guix import gem)
   #:use-module (guix gnupg)
   #:use-module (gnu packages)
   #:use-module ((gnu packages commencement) #:select (%final-inputs))
@@ -195,7 +197,8 @@ unavailable optional dependencies such as Guile-JSON."
                  %gnome-updater
                  %elpa-updater
                  %cran-updater
-                 ((guix import pypi) => %pypi-updater)))
+                 ((guix import pypi) => %pypi-updater)
+                 %gem-updater))
 
 (define (lookup-updater name)
   "Return the updater called NAME."
-- 
2.6.3

Re: [PATCH] Add rubygems updater.

[Pjotr Prins]

On Fri, Jan 01, 2016 at 06:27:21PM +1000, Ben Woodcroft wrote:
> Phew, you almost beat me to the first patch of the year there Pjotr..

Mine is trivial ;)

> It seems there's 30 packages to be updated, out of the 107 in
> ruby.scm. Going through each of these individually seems a little
> tedious, can we do them in bulk somehow or do they have to be
> committed individually? Building and testing all packages that
> require these packages would be a start - is there any way to list
> all dependent packages?
> 
> gnu/packages/ruby.scm:2807:13: ruby-cutest would be upgraded from
> 1.2.2 to 1.2.3
> gnu/packages/ruby.scm:333:13: ruby-rspec-mocks would be upgraded
> from 3.2.1 to 3.4.0
(etc)

I don't think it is a good idea to automatically update
packages. Reason being that packages should be updated by someone who
is actively using that new version. Automated tests are one thing,
real user feedback another. Not to mention that many gems don't have
tests ;).

What is useful is to generate (export) an updated package using the
old one as an input. Or show a diff of version + SHA. That way it
becomes reasonably easy to update packages.

Pj.

Re: [PATCH] Add rubygems updater.

[Ben Woodcroft]

On 01/01/16 19:28, Pjotr Prins wrote:
> On Fri, Jan 01, 2016 at 06:27:21PM +1000, Ben Woodcroft wrote:
>> It seems there's 30 packages to be updated, out of the 107 in
>> ruby.scm. Going through each of these individually seems a little
>> tedious, can we do them in bulk somehow or do they have to be
>> committed individually? Building and testing all packages that
>> require these packages would be a start - is there any way to list
>> all dependent packages?
>>
>> gnu/packages/ruby.scm:2807:13: ruby-cutest would be upgraded from
>> 1.2.2 to 1.2.3
>> gnu/packages/ruby.scm:333:13: ruby-rspec-mocks would be upgraded
>> from 3.2.1 to 3.4.0
> (etc)
>
> I don't think it is a good idea to automatically update
> packages. Reason being that packages should be updated by someone who
> is actively using that new version. Automated tests are one thing,
> real user feedback another. Not to mention that many gems don't have
> tests ;).
I think we should update the package definitions so that more have 
tests, and failing that import the library so we know it can at least be 
loaded, like this:

+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'check
+           (lambda _
+             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))

I would prefer to do this testing in an environment where only the propagated inputs are loaded, but I'm not sure how to do this. But I digress.

Do you think it would be a good idea to provide a "bleeding edge" repository so that users can more easily help with testing? Perhaps also a branch that only updates according to semantic versioning?


>
> What is useful is to generate (export) an updated package using the
> old one as an input. Or show a diff of version + SHA. That way it
> becomes reasonably easy to update packages.
I'm not sure what you mean. guix refresh has the --update flag, which 
updates the version and source SHA hash in the source code - useful.

Thanks,
ben

Re: [PATCH] Add rubygems updater.

[Pjotr Prins]

On Fri, Jan 01, 2016 at 09:18:54PM +1000, Ben Woodcroft wrote:
> I think we should update the package definitions so that more have
> tests, and failing that import the library so we know it can at
> least be loaded, like this:
> 
> +     `(#:phases
> +       (modify-phases %standard-phases
> +         (replace 'check
> +           (lambda _
> +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))

That is a good idea. When gems lack tests, either they don't have
them, or the authors don't add them to gems. We can try and request
upstream or add the tests ourselves.

> I would prefer to do this testing in an environment where only the propagated inputs are loaded, but I'm not sure how to do this. But I digress.
> 
> Do you think it would be a good idea to provide a "bleeding edge" repository so that users can more easily help with testing? Perhaps also a branch that only updates according to semantic versioning?

That may be interesting, at least to a Ruby specific audience.

> 
> >
> >What is useful is to generate (export) an updated package using the
> >old one as an input. Or show a diff of version + SHA. That way it
> >becomes reasonably easy to update packages.
> I'm not sure what you mean. guix refresh has the --update flag,
> which updates the version and source SHA hash in the source code -
> useful.

You just taught me something :)

Pj.

Re: [PATCH] Add rubygems updater.

[Ludovic Courtès]

Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

> On 01/01/16 19:28, Pjotr Prins wrote:
>> On Fri, Jan 01, 2016 at 06:27:21PM +1000, Ben Woodcroft wrote:
>>> It seems there's 30 packages to be updated, out of the 107 in
>>> ruby.scm. Going through each of these individually seems a little
>>> tedious, can we do them in bulk somehow or do they have to be
>>> committed individually? Building and testing all packages that
>>> require these packages would be a start - is there any way to list
>>> all dependent packages?
>>>
>>> gnu/packages/ruby.scm:2807:13: ruby-cutest would be upgraded from
>>> 1.2.2 to 1.2.3
>>> gnu/packages/ruby.scm:333:13: ruby-rspec-mocks would be upgraded
>>> from 3.2.1 to 3.4.0
>> (etc)
>>
>> I don't think it is a good idea to automatically update
>> packages. Reason being that packages should be updated by someone who
>> is actively using that new version. Automated tests are one thing,
>> real user feedback another. Not to mention that many gems don't have
>> tests ;).
> I think we should update the package definitions so that more have
> tests, and failing that import the library so we know it can at least
> be loaded, like this:
>
> +     `(#:phases
> +       (modify-phases %standard-phases
> +         (replace 'check
> +           (lambda _
> +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))

The only case where this would make a difference is for leaf packages,
no?  In all the other cases, building dependent packages will ensure
that the package at hand works as expected.

Ludo’.

Re: [PATCH] Add rubygems updater.

[Ben Woodcroft]

On 02/01/16 04:17, Ludovic Courtès wrote:
> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>
>> On 01/01/16 19:28, Pjotr Prins wrote:
>>> On Fri, Jan 01, 2016 at 06:27:21PM +1000, Ben Woodcroft wrote:
>>>> It seems there's 30 packages to be updated, out of the 107 in
>>>> ruby.scm. Going through each of these individually seems a little
>>>> tedious, can we do them in bulk somehow or do they have to be
>>>> committed individually? Building and testing all packages that
>>>> require these packages would be a start - is there any way to list
>>>> all dependent packages?
>>>>
>>>> gnu/packages/ruby.scm:2807:13: ruby-cutest would be upgraded from
>>>> 1.2.2 to 1.2.3
>>>> gnu/packages/ruby.scm:333:13: ruby-rspec-mocks would be upgraded
>>>> from 3.2.1 to 3.4.0
>>> (etc)
>>>
>>> I don't think it is a good idea to automatically update
>>> packages. Reason being that packages should be updated by someone who
>>> is actively using that new version. Automated tests are one thing,
>>> real user feedback another. Not to mention that many gems don't have
>>> tests ;).
>> I think we should update the package definitions so that more have
>> tests, and failing that import the library so we know it can at least
>> be loaded, like this:
>>
>> +     `(#:phases
>> +       (modify-phases %standard-phases
>> +         (replace 'check
>> +           (lambda _
>> +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))
> The only case where this would make a difference is for leaf packages,
> no?  In all the other cases, building dependent packages will ensure
> that the package at hand works as expected.
Sure, but even in the case where they aren't leaf packages at least the 
build error gets thrown when building the package at fault. There's also 
the important difference that it makes the packager feel less bad about 
the disappointing lack of tests or the necessity of disabling them 
because of circular dependencies.

ben

Re: [PATCH] Add rubygems updater.

[Ludovic Courtès]

Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

> On 02/01/16 04:17, Ludovic Courtès wrote:
>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>
>>> On 01/01/16 19:28, Pjotr Prins wrote:
>>>> On Fri, Jan 01, 2016 at 06:27:21PM +1000, Ben Woodcroft wrote:
>>>>> It seems there's 30 packages to be updated, out of the 107 in
>>>>> ruby.scm. Going through each of these individually seems a little
>>>>> tedious, can we do them in bulk somehow or do they have to be
>>>>> committed individually? Building and testing all packages that
>>>>> require these packages would be a start - is there any way to list
>>>>> all dependent packages?
>>>>>
>>>>> gnu/packages/ruby.scm:2807:13: ruby-cutest would be upgraded from
>>>>> 1.2.2 to 1.2.3
>>>>> gnu/packages/ruby.scm:333:13: ruby-rspec-mocks would be upgraded
>>>>> from 3.2.1 to 3.4.0
>>>> (etc)
>>>>
>>>> I don't think it is a good idea to automatically update
>>>> packages. Reason being that packages should be updated by someone who
>>>> is actively using that new version. Automated tests are one thing,
>>>> real user feedback another. Not to mention that many gems don't have
>>>> tests ;).
>>> I think we should update the package definitions so that more have
>>> tests, and failing that import the library so we know it can at least
>>> be loaded, like this:
>>>
>>> +     `(#:phases
>>> +       (modify-phases %standard-phases
>>> +         (replace 'check
>>> +           (lambda _
>>> +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))
>> The only case where this would make a difference is for leaf packages,
>> no?  In all the other cases, building dependent packages will ensure
>> that the package at hand works as expected.
> Sure, but even in the case where they aren't leaf packages at least
> the build error gets thrown when building the package at
> fault. There's also the important difference that it makes the
> packager feel less bad about the disappointing lack of tests or the
> necessity of disabling them because of circular dependencies.

Right.  The only downside I can think of is if packagers have to copy
the above 4 lines in each and every package.  Can you think of a way
that would avoid that?

Ludo’.

Re: [PATCH] Add rubygems updater.

[Ben Woodcroft]

On 03/01/16 06:54, Ludovic Courtès wrote:
> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>
>> On 02/01/16 04:17, Ludovic Courtès wrote:
>>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>>
>>>> On 01/01/16 19:28, Pjotr Prins wrote:
>>>>> On Fri, Jan 01, 2016 at 06:27:21PM +1000, Ben Woodcroft wrote:
>>>>>> It seems there's 30 packages to be updated, out of the 107 in
>>>>>> ruby.scm. Going through each of these individually seems a little
>>>>>> tedious, can we do them in bulk somehow or do they have to be
>>>>>> committed individually? Building and testing all packages that
>>>>>> require these packages would be a start - is there any way to list
>>>>>> all dependent packages?
>>>>>>
>>>>>> gnu/packages/ruby.scm:2807:13: ruby-cutest would be upgraded from
>>>>>> 1.2.2 to 1.2.3
>>>>>> gnu/packages/ruby.scm:333:13: ruby-rspec-mocks would be upgraded
>>>>>> from 3.2.1 to 3.4.0
>>>>> (etc)
>>>>>
>>>>> I don't think it is a good idea to automatically update
>>>>> packages. Reason being that packages should be updated by someone who
>>>>> is actively using that new version. Automated tests are one thing,
>>>>> real user feedback another. Not to mention that many gems don't have
>>>>> tests ;).
>>>> I think we should update the package definitions so that more have
>>>> tests, and failing that import the library so we know it can at least
>>>> be loaded, like this:
>>>>
>>>> +     `(#:phases
>>>> +       (modify-phases %standard-phases
>>>> +         (replace 'check
>>>> +           (lambda _
>>>> +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))
>>> The only case where this would make a difference is for leaf packages,
>>> no?  In all the other cases, building dependent packages will ensure
>>> that the package at hand works as expected.
>> Sure, but even in the case where they aren't leaf packages at least
>> the build error gets thrown when building the package at
>> fault. There's also the important difference that it makes the
>> packager feel less bad about the disappointing lack of tests or the
>> necessity of disabling them because of circular dependencies.
> Right.  The only downside I can think of is if packagers have to copy
> the above 4 lines in each and every package.  Can you think of a way
> that would avoid that?
I have only been adding these in cases where testing is impossible, but 
we could make it a wider policy.

We could bake it into the build system, by adding an optional argument 
#:import so that you could do

     (build-system ruby-build-system)
     (arguments
      `(#:import "ansi"
        #:tests? #f)) ; tests require circular dependencies

Probably in that case makes sense to have a new phase 'check-import so 
that more complex cases can be handled, rather than replacing 'check. 
There's no way to run this phase with the native-inputs disappeared is 
there so it more closely mirrors a user's experience?

We could even default this to the expected name of the library guessed 
from the name of the package when #:import is not given. However, this 
would unfortunately break packages that have been written outside of 
Guix, so I imagine you don't feel this is a good idea.

WDYT?

Thanks,
ben

Re: [PATCH] Add rubygems updater.

[Ludovic Courtès]

Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

> On 03/01/16 06:54, Ludovic Courtès wrote:
>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>
>>> On 02/01/16 04:17, Ludovic Courtès wrote:
>>>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

[...]

>>>>> +     `(#:phases
>>>>> +       (modify-phases %standard-phases
>>>>> +         (replace 'check
>>>>> +           (lambda _
>>>>> +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))
>>>> The only case where this would make a difference is for leaf packages,
>>>> no?  In all the other cases, building dependent packages will ensure
>>>> that the package at hand works as expected.
>>> Sure, but even in the case where they aren't leaf packages at least
>>> the build error gets thrown when building the package at
>>> fault. There's also the important difference that it makes the
>>> packager feel less bad about the disappointing lack of tests or the
>>> necessity of disabling them because of circular dependencies.
>> Right.  The only downside I can think of is if packagers have to copy
>> the above 4 lines in each and every package.  Can you think of a way
>> that would avoid that?
> I have only been adding these in cases where testing is impossible,
> but we could make it a wider policy.
>
> We could bake it into the build system, by adding an optional argument
> #:import so that you could do
>
>     (build-system ruby-build-system)
>     (arguments
>      `(#:import "ansi"
>        #:tests? #f)) ; tests require circular dependencies

The problem is that the “-Ilib” in the command above cannot be guessed,
can it?

> Probably in that case makes sense to have a new phase 'check-import so
> that more complex cases can be handled, rather than replacing
> 'check.

Agreed.

> There's no way to run this phase with the native-inputs disappeared is
> there so it more closely mirrors a user's experience?

Not easily.  The phase would have to recompute the RUBYPATH (or whatever
it’s called.)

> We could even default this to the expected name of the library guessed
> from the name of the package when #:import is not given. However, this
> would unfortunately break packages that have been written outside of
> Guix, so I imagine you don't feel this is a good idea.

We could choose the package name as a default value, but often that’s
not going to work, notably because of the “ruby-” prefix.

WDYT?

Ludo’.

Re: [PATCH] Add rubygems updater.

[Ben Woodcroft]

On 04/01/16 00:06, Ludovic Courtès wrote:
> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>
>> On 03/01/16 06:54, Ludovic Courtès wrote:
>>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>>
>>>> On 02/01/16 04:17, Ludovic Courtès wrote:
>>>>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
> [...]
>
>>>>>> +     `(#:phases
>>>>>> +       (modify-phases %standard-phases
>>>>>> +         (replace 'check
>>>>>> +           (lambda _
>>>>>> +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))
>>>>> The only case where this would make a difference is for leaf packages,
>>>>> no?  In all the other cases, building dependent packages will ensure
>>>>> that the package at hand works as expected.
>>>> Sure, but even in the case where they aren't leaf packages at least
>>>> the build error gets thrown when building the package at
>>>> fault. There's also the important difference that it makes the
>>>> packager feel less bad about the disappointing lack of tests or the
>>>> necessity of disabling them because of circular dependencies.
>>> Right.  The only downside I can think of is if packagers have to copy
>>> the above 4 lines in each and every package.  Can you think of a way
>>> that would avoid that?
>> I have only been adding these in cases where testing is impossible,
>> but we could make it a wider policy.
>>
>> We could bake it into the build system, by adding an optional argument
>> #:import so that you could do
>>
>>      (build-system ruby-build-system)
>>      (arguments
>>       `(#:import "ansi"
>>         #:tests? #f)) ; tests require circular dependencies
> The problem is that the “-Ilib” in the command above cannot be guessed,
> can it?
My understanding is that the the "-Ilib" is almost invariant because 
putting imported code in the lib subdirectory is a convention that most 
gems adhere to. In those cases where it fails, the 'check-import phase 
can be replaced or removed.

[..]
>> We could even default this to the expected name of the library guessed
>> from the name of the package when #:import is not given. However, this
>> would unfortunately break packages that have been written outside of
>> Guix, so I imagine you don't feel this is a good idea.
> We could choose the package name as a default value, but often that’s
> not going to work, notably because of the “ruby-” prefix.
>
> WDYT?
Removing the "ruby-" from the package name sounds like a reasonable 
default, but won't work every time because some imports use underscores 
where some use dashes e.g. "minitest-pretty_diff".

I'm keen to make sure you understand what I'm attempting to say though. 
By "default" I mean when the #:import flag is missing from arguments, 
"ruby -Ilib -r <guessed_package_name>" will be run. So if I have 
previously packaged a rubygem outside Guix and it is working fine, 
implementing the default might break my package making me unhappy. If 
you instead interpreted "default" as the guessed value that "guix 
import" generates, then that is less likely to end in unhappiness.

WDYT?

Thanks,
ben

Re: [PATCH] Add rubygems updater.

[Ricardo Wurmus]

Ben Woodcroft <b.woodcroft@uq.edu.au> writes:

>>> We could even default this to the expected name of the library guessed
>>> from the name of the package when #:import is not given. However, this
>>> would unfortunately break packages that have been written outside of
>>> Guix, so I imagine you don't feel this is a good idea.
>> We could choose the package name as a default value, but often that’s
>> not going to work, notably because of the “ruby-” prefix.
>>
>> WDYT?
> Removing the "ruby-" from the package name sounds like a reasonable 
> default, but won't work every time because some imports use underscores 
> where some use dashes e.g. "minitest-pretty_diff".

For R packages we’re using something like this

        (properties `((upstream-name . "gridExtra")))

to hold the upstream name.  The Guix package is called “r-gridextra”.
Maybe that’s acceptable for Ruby packages too?

~~ Ricardo

Re: [PATCH] Add rubygems updater.

[Ludovic Courtès]

Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

> On 04/01/16 00:06, Ludovic Courtès wrote:
>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>
>>> On 03/01/16 06:54, Ludovic Courtès wrote:
>>>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>>>
>>>>> On 02/01/16 04:17, Ludovic Courtès wrote:
>>>>>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>> [...]
>>
>>>>>>> +     `(#:phases
>>>>>>> +       (modify-phases %standard-phases
>>>>>>> +         (replace 'check
>>>>>>> +           (lambda _
>>>>>>> +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))

[...]

>>>      (build-system ruby-build-system)
>>>      (arguments
>>>       `(#:import "ansi"
>>>         #:tests? #f)) ; tests require circular dependencies
>> The problem is that the “-Ilib” in the command above cannot be guessed,
>> can it?
> My understanding is that the the "-Ilib" is almost invariant because
> putting imported code in the lib subdirectory is a convention that
> most gems adhere to. In those cases where it fails, the 'check-import
> phase can be replaced or removed.

OK.  My point is that, since we’re talking about saving 4 lines of code,
we have to make sure that the default thing works for the vast majority
of Ruby packages.

> [..]
>>> We could even default this to the expected name of the library guessed
>>> from the name of the package when #:import is not given. However, this
>>> would unfortunately break packages that have been written outside of
>>> Guix, so I imagine you don't feel this is a good idea.
>> We could choose the package name as a default value, but often that’s
>> not going to work, notably because of the “ruby-” prefix.
>>
>> WDYT?
> Removing the "ruby-" from the package name sounds like a reasonable
> default, but won't work every time because some imports use
> underscores where some use dashes e.g. "minitest-pretty_diff".
>
> I'm keen to make sure you understand what I'm attempting to say
> though. By "default" I mean when the #:import flag is missing from
> arguments, "ruby -Ilib -r <guessed_package_name>" will be run. So if I
> have previously packaged a rubygem outside Guix and it is working
> fine, implementing the default might break my package making me
> unhappy. If you instead interpreted "default" as the guessed value
> that "guix import" generates, then that is less likely to end in
> unhappiness.

I was thinking of the #:import default value in ‘ruby-build-system’.

Using properties as Ricardo suggests woudln’t be any more concise than
using #:import "the-right-name".

Thanks,
Ludo’.

Re: [PATCH] Add rubygems updater.

[Pjotr Prins]

On Fri, Jan 01, 2016 at 07:17:17PM +0100, Ludovic Courtès wrote:
> > I think we should update the package definitions so that more have
> > tests, and failing that import the library so we know it can at least
> > be loaded, like this:
> >
> > +     `(#:phases
> > +       (modify-phases %standard-phases
> > +         (replace 'check
> > +           (lambda _
> > +             (zero? (system* "ruby" "-Ilib" "-r" "ansi")))))))
> 
> The only case where this would make a difference is for leaf packages,
> no?  In all the other cases, building dependent packages will ensure
> that the package at hand works as expected.

Yes, this is a cheap and cheerful integration test. It does point out earlier where
packages don't load. I think it is a great idea because it is 'free'.
We could also do this for python packages and the like.

Pj.