* Re: [PATCH] build: ruby: Rewrite build system to use gem archives.
2015-08-16 22:19 ` Thompson, David
@ 2015-08-17 5:43 ` Pjotr Prins
2015-08-17 23:31 ` Thompson, David
2015-08-18 15:18 ` Ludovic Courtès
2 siblings, 0 replies; 11+ messages in thread
From: Pjotr Prins @ 2015-08-17 5:43 UTC (permalink / raw)
To: Thompson, David; +Cc: guix-devel
Made my day :)
On Sun, Aug 16, 2015 at 06:19:21PM -0400, Thompson, David wrote:
> Update:
>
> On Tue, Jul 28, 2015 at 9:09 AM, David Thompson
> <dthompson2@worcester.edu> wrote:
>
> > One other caveat is that downloading gems from rubygems.org currently
> > *does not* work because the HTTP responses include an invalid
> > Content-Type header. I added a temporary hack to do my development
> > work (not in this patch), and I'm currently waiting on a response from
> > the rubygems.org admins to see if they will fix the issue. So, this
> > patch should not be applied until this is resolved.
> >
> > See: http://help.rubygems.org/discussions/problems/21223-invalid-content-type-header-when-downloading-gem-archive?unresolve=true
>
> This issue has been fixed by the rubygems.org admins! I would like to
> apply the below patch ASAP and get to work on 'guix import gem' if
> there are no objections.
>
> The major change from the previous version of this patch is that I
> figured how to unpack the source gem, allow the package build script
> to make any necessary dynamic patches, and then install a repacked gem
> as opposed to the source gem.
>
> - Dave
> From ee027442dd968734831eb372fcf502dcfaad7623 Mon Sep 17 00:00:00 2001
> From: David Thompson <dthompson2@worcester.edu>
> Date: Sun, 26 Jul 2015 22:01:54 -0400
> Subject: [PATCH] build: ruby: Rewrite build system to use gem archives.
>
> Co-Authored-By: Pjotr Prins <pjotr.public01@thebird.nl>
>
> * guix/build-system/ruby.scm (lower): Remove git dependency.
> (rubygems-uri): New procedure.
> * guix/build/ruby-build-system (gitify, build): Delete.
> (unpack): Use 'gem unpack' utility.
> (check): Add docstring.
> (install): Rebuild unpacked gem and install it.
> (%standard-phases): Remove gitify and build phases.
> * gnu/packages/ruby.scm (ruby-hoe, ruby-rake-compiler, ruby-i18n,
> ruby-rspec-support, ruby-rspec-core, ruby-diff-lcs-for-rspec,
> ruby-rspec-expectations, ruby-rspec-mocks, ruby-rspec, bundler,
> ruby-useragent, ruby-bacon, ruby-arel, ruby-connection-pool,
> ruby-net-http-persistent, ruby-minitest, ruby-minitest-sprint,
> ruby-minitest-bacon, ruby-daemons, ruby-git, ruby-slop,
> ruby-multipart-post): Convert to new build system.
> ---
> gnu/packages/ruby.scm | 222 +++++++++++----------------------------
> guix/build-system/ruby.scm | 40 +++----
> guix/build/ruby-build-system.scm | 60 +++++++----
> 3 files changed, 125 insertions(+), 197 deletions(-)
>
> diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
> index a481365..196094c 100644
> --- a/gnu/packages/ruby.scm
> +++ b/gnu/packages/ruby.scm
> @@ -154,13 +154,11 @@ a focus on simplicity and productivity.")
> (name "ruby-hoe")
> (version "3.13.1")
> (source (origin
> - (method git-fetch)
> - (uri (git-reference
> - (url "https://github.com/seattlerb/hoe.git")
> - (commit "0c11836"))) ; no release tags :(
> + (method url-fetch)
> + (uri (rubygems-uri "hoe" version))
> (sha256
> (base32
> - "0i8dimf8kxcjgqj9x65bbi3l6hc9p9gbfbb1vmrz42764a4jjbz9"))) )
> + "1mac13krdrasn9819dd65xj27kklfy0xdbj3p6s2ij4vlcb46h8q"))) )
> (build-system ruby-build-system)
> (synopsis "Ruby project management helper")
> (description
> @@ -178,22 +176,13 @@ announcement.")
> (version "0.9.5")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/rake-compiler/rake-compiler/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "rake-compiler" version))
> (sha256
> (base32
> - "07lk1vl0jqcaqwjjhmg0qshqwcxdyr5kscc9xxm13m03835xgpf3"))))
> + "1k8im2vzj849xdgjk6wafspkiwwapqwm738majchb4dnhnsk64cx"))))
> (build-system ruby-build-system)
> (arguments
> - '(#:tests? #f ; needs cucumber
> - #:phases (modify-phases %standard-phases
> - (add-before 'build 'remove-cucumber-rake-task
> - (lambda _
> - ;; Remove cucumber test file because the
> - ;; dependencies are not available right now.
> - (delete-file "tasks/cucumber.rake"))))))
> + '(#:tests? #f)) ; needs cucumber
> (synopsis "Building and packaging helper for Ruby native extensions")
> (description "Rake-compiler provides a framework for building and
> packaging native C and Java extensions in Ruby.")
> @@ -206,12 +195,10 @@ packaging native C and Java extensions in Ruby.")
> (version "0.6.11")
> (source (origin
> (method url-fetch)
> - (uri (string-append "https://github.com/svenfuchs/i18n/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "i18n" version))
> (sha256
> (base32
> - "1fdhnhh1p5g8vibv44d770z8nq208zrms3m2nswdvr54072y1m6k"))))
> + "0fwjlgmgry2blf8zlxn9c555cf4a16p287l599kz5104ncjxlzdk"))))
> (build-system ruby-build-system)
> (arguments
> '(#:tests? #f)) ; requires bundler
> @@ -234,13 +221,10 @@ an extensible architecture with a swappable backend.")
> (version "3.2.2")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/rspec/rspec-support/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "rspec-support" version))
> (sha256
> (base32
> - "1pvzfrqgy0z0gwmdgjp9f2vz1d9c0cajyzfqj9z8i2ssxnzmj4bv"))))
> + "194zry5195ls2hni7r9824vqb5d3qfg4jb15fgj8glfy0rvw3zxl"))))
> (build-system ruby-build-system)
> (arguments
> '(#:tests? #f)) ; avoid dependency cycles
> @@ -255,13 +239,10 @@ an extensible architecture with a swappable backend.")
> (version "3.2.3")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/rspec/rspec-core/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "rspec-core" version))
> (sha256
> (base32
> - "1clsa4lkh5c9c7xc3xa336ym00ycr67pchpg1bv4y3fz5hvzw8ki"))))
> + "0k2471iw30gc2cvv67damrx666pmsvx8l0ahk3hm20dhfnmcmpvv"))))
> (build-system ruby-build-system)
> (arguments
> '(#:tests? #f)) ; avoid dependency cycles
> @@ -279,13 +260,10 @@ groups.")
> (version "1.2.5")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/halostatue/diff-lcs/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "diff-lcs" version))
> (sha256
> (base32
> - "0kmfz2qdwbfjf97rx27hh9fm39mv3z9avjmvsajqnb5wxj2l5l4s"))))
> + "1vf9civd41bnqi6brr5d9jifdw73j9khc6fkhfl1f8r9cpkdvlx1"))))
> (build-system ruby-build-system)
> (arguments
> '(#:tests? #f)) ; avoid dependency cycles
> @@ -303,13 +281,10 @@ standard diff-like tool.")
> (version "3.2.1")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/rspec/rspec-expectations/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "rspec-expectations" version))
> (sha256
> (base32
> - "0h0rpprbh6h59gmksiyi1b8w6cvcai4wdbkikajwx3w1asxi6f7x"))))
> + "01kmchabgpdcaqdsqg8r0g5gy385xhp1k1jsds3w264ypin17n14"))))
> (build-system ruby-build-system)
> (arguments
> '(#:tests? #f)) ; avoid dependency cycles
> @@ -328,13 +303,10 @@ outcomes of a code example.")
> (version "3.2.1")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/rspec/rspec-mocks/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "rspec-mocks" version))
> (sha256
> (base32
> - "1xzxsg0idxkg7czmjgqq10lcd821ibw1hjzn404sk9j6rw0fbx2g"))))
> + "09yig1lwgxl8fsns71z3xhv7wkg7zvagydh37pvaqpw92dz55jv2"))))
> (build-system ruby-build-system)
> (arguments
> '(#:tests? #f)) ; avoid dependency cycles
> @@ -353,13 +325,10 @@ support for stubbing and mocking.")
> (version "3.2.0")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/rspec/rspec/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "rspec" version))
> (sha256
> (base32
> - "1jg38dbaknsdhiav5vnrwfccg524fwcg6sq1715441vx1xl6p54q"))))
> + "0lkz01j4yxcwb3g5w6r1l9khnyw3sxib4rrh4npd2pxh390fcc4f"))))
> (build-system ruby-build-system)
> (arguments
> '(#:tests? #f)) ; avoid dependency cycles
> @@ -382,12 +351,10 @@ expectations and mocks frameworks.")
> (version "1.9.9")
> (source (origin
> (method url-fetch)
> - (uri (string-append "https://github.com/bundler/bundler/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "bundler" version))
> (sha256
> (base32
> - "08flx3n9hb3yz8mm5k16cdz0sb7g774f6vxn6gc3wfh5la83vfyx"))))
> + "12qk1569pswa9mmid6lsqy2napn9fmkbmv0k7xkl52nyfh8rsy4d"))))
> (build-system ruby-build-system)
> (arguments
> '(#:tests? #f)) ; avoid dependency cycles
> @@ -403,19 +370,13 @@ specified in a \"Gemfile\", as well as their dependencies.")
> (version "0.13.3")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/gshutler/useragent/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "useragent" version))
> (sha256
> (base32
> - "1hj00fw06i0y3rwxxhxmnrqxhpnffv4zfqx2sqqpc5qc4fdvd2x9"))))
> + "0kz7yyz7528bv4a2kfymvkcm8whqcddhmgaw1ksw1d90n30hhkpc"))))
> (build-system ruby-build-system)
> (arguments
> - '(#:test-target "spec"))
> - (native-inputs
> - `(("ruby-rspec" ,ruby-rspec)
> - ("bundler" ,bundler)))
> + '(#:tests? #f)) ; no test suite
> (synopsis "HTTP user agent parser for Ruby")
> (description "UserAgent is a Ruby library that parses and compares HTTP
> User Agents.")
> @@ -425,26 +386,14 @@ User Agents.")
> (define-public ruby-bacon
> (package
> (name "ruby-bacon")
> - (version "1.2")
> + (version "1.2.0")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/chneukirchen/bacon/archive/"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "bacon" version))
> (sha256
> (base32
> - "0g03fxilrrx17dijww68n1lq5d8s69hrxgpga8c1i2k35bzcw5jc"))))
> + "1f06gdj77bmwzc1k5iragl1595hbn67yc7sqvs56ca8plrr2vmai"))))
> (build-system ruby-build-system)
> - (arguments
> - `(#:phases (modify-phases %standard-phases
> - (add-before 'build 'generate-docs
> - (lambda _
> - ;; This rake task also tries to generate a ChangeLog
> - ;; file from the Git log, which we don't have. It fails
> - ;; but creates an empty file, allowing the rest of the
> - ;; build to succeed.
> - (zero? (system* "rake" "predist")))))))
> (synopsis "Small RSpec clone")
> (description "Bacon is a small RSpec clone providing all essential
> features.")
> @@ -457,16 +406,13 @@ features.")
> (version "6.0.0")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/rails/arel/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "arel" version))
> (sha256
> (base32
> - "0fldwp2hmrmddx22xf7hdmybngzv97qnv5rvz3qw61m94ddd6w4n"))))
> + "18wnfnzr2i5p3fygsddjbi1cimws6823nbk8drxidmnj8jz7h0ar"))))
> (build-system ruby-build-system)
> - (native-inputs
> - `(("bundler" ,bundler)))
> + (arguments
> + '(#:tests? #f)) ; no test suite
> (synopsis "SQL AST manager for Ruby")
> (description "Arel is a SQL AST manager for Ruby. It simplifies the
> generation of complex SQL queries and adapts to various relational database
> @@ -480,13 +426,10 @@ implementations.")
> (version "2.2.0")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/mperham/connection_pool/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "connection_pool" version))
> (sha256
> (base32
> - "02s5rwhmgy8qqns7a5y1daa0yaw38x6lzpwyvmy46h1yrj9mc6zf"))))
> + "1b2bb3k39ni5mzcnqlv9y4yjkbin20s7dkwzp0jw2jf1rmzcgrmy"))))
> (build-system ruby-build-system)
> (native-inputs
> `(("bundler" ,bundler)))
> @@ -502,13 +445,10 @@ interface for Ruby programs.")
> (version "2.9.4")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/drbrain/net-http-persistent/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "net-http-persistent" version))
> (sha256
> (base32
> - "1q18vji31w8gfr6ajziqkqs8n5lzkw0bl00dm2a8s05zhavzw9j9"))))
> + "1y9fhaax0d9kkslyiqi1zys6cvpaqx9a0y0cywp24rpygwh4s9r4"))))
> (build-system ruby-build-system)
> (native-inputs
> `(("ruby-connection-pool" ,ruby-connection-pool)
> @@ -524,16 +464,11 @@ using Net::HTTP, supporting reconnection and retry according to RFC 2616.")
> (name "ruby-minitest")
> (version "5.7.0")
> (source (origin
> - (method git-fetch)
> - ;; No release tarballs nor git tags. This is the commit
> - ;; corresponding to the addition of the release notes to
> - ;; History.rdoc.
> - (uri (git-reference
> - (url "https://github.com/seattlerb/minitest.git")
> - (commit "e975248")))
> + (method url-fetch)
> + (uri (rubygems-uri "minitest" version))
> (sha256
> (base32
> - "09xjiahk7q8hid1i39ahrmghaslpj9n36zna72i3ah7kf1bh2l01"))))
> + "0rxqfakp629mp3vwda7zpgb57lcns5znkskikbfd0kriwv8i1vq8"))))
> (build-system ruby-build-system)
> (native-inputs
> `(("ruby-hoe" ,ruby-hoe)))
> @@ -548,14 +483,11 @@ facilities supporting TDD, BDD, mocking, and benchmarking.")
> (name "ruby-minitest-sprint")
> (version "1.1.0")
> (source (origin
> - (method git-fetch)
> - ;; Same story as ruby-minitest.
> - (uri (git-reference
> - (url "https://github.com/seattlerb/minitest-sprint.git")
> - (commit "49c02bc")))
> + (method url-fetch)
> + (uri (rubygems-uri "minitest-sprint" version))
> (sha256
> (base32
> - "0rbmxz94lqg5vjz60p8v2bzq8adwvmx501amvk0l124sfwmw94ms"))))
> + "179d6pj56l9xzm46fqsqj10mzjkr1f9fv4cxa8wvchs97hqz33w1"))))
> (build-system ruby-build-system)
> (native-inputs
> `(("ruby-hoe" ,ruby-hoe)
> @@ -571,14 +503,11 @@ easier to re-run individual failing tests.")
> (name "ruby-minitest-bacon")
> (version "1.0.2")
> (source (origin
> - (method git-fetch)
> - ;; Same story as ruby-minitest.
> - (uri (git-reference
> - (url "https://github.com/seattlerb/minitest-bacon.git")
> - (commit "38551d5")))
> + (method url-fetch)
> + (uri (rubygems-uri "minitest-bacon" version))
> (sha256
> (base32
> - "19r9fm41i0mm1xncqls8frbj1i9nr3sq1cx2mh878r6kdl02d70h"))))
> + "0cm7r68422743i3b6fm4rrm0r6cnnjmglq5gcmmgl1f0rk5hnf6r"))))
> (build-system ruby-build-system)
> (native-inputs
> `(("ruby-hoe" ,ruby-hoe)))
> @@ -596,13 +525,10 @@ functionality, making it easier to migrate test suites from bacon to minitest.")
> (version "1.2.2")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/thuehlinger/daemons/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "daemons" version))
> (sha256
> (base32
> - "1v5bpdvpvhk240pc7fkn44vfclppl44pp6wd42ipi5sd5lkk7zfd"))))
> + "121c7vkimg3baxga69xvdkwxiq8wkmxqvdbyqi5i82vhih5d3cn3"))))
> (build-system ruby-build-system)
> (arguments
> `(#:tests? #f)) ; no test suite
> @@ -618,41 +544,27 @@ run as a daemon and to be controlled by simple start/stop/restart commands.")
> (version "1.2.9.1")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/schacon/ruby-git/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "git" version))
> (sha256
> (base32
> - "08zg20zc7f7yy34ix2qdd8jbiz7xhjc8alk370869sq3h75hs9jc"))))
> + "1sqfj8lmhl7c5zamcckkpik4izfph2zkv6krw0i8mzj5pdws5acs"))))
> (build-system ruby-build-system)
> (arguments
> - '(#:phases (modify-phases %standard-phases
> - (add-before 'build 'patch-git-binary
> - (lambda* (#:key inputs #:allow-other-keys)
> + `(#:tests? #f ; no tests
> + #:phases (modify-phases %standard-phases
> + (add-after 'install 'patch-git-binary
> + (lambda* (#:key inputs outputs #:allow-other-keys)
> ;; Make the default git binary an absolute path to the
> ;; store.
> - (let ((git (string-append (assoc-ref inputs "git")
> - "/bin/git")))
> - (substitute* '("lib/git/config.rb")
> + (let ((git (string-append (assoc-ref inputs "git")
> + "/bin/git"))
> + (config (string-append (getenv "GEM_HOME")
> + "/gems/git-" ,version
> + "/lib/git/config.rb")))
> + (substitute* (list config)
> (("'git'")
> (string-append "'" git "'")))
> - ;; Fix a test that expects the binary to be simply
> - ;; 'git'.
> - (substitute* '("tests/units/test_logger.rb")
> - (("def test_logger")
> - (string-append
> - "def test_logger\n"
> - "Git::Base.config.binary_path = 'git'")))
> - #t)))
> - (add-before 'check 'create-fake-home
> - (lambda _
> - ;; The test suite runs 'git config --global' commands,
> - ;; so a fake home directory is needed for them to
> - ;; succeed.
> - (let ((fake-home (string-append (getcwd) "/fake-home")))
> - (mkdir fake-home)
> - (setenv "HOME" fake-home)))))))
> + #t))))))
> (inputs
> `(("git" ,git)))
> (synopsis "Ruby wrappers for Git")
> @@ -667,13 +579,10 @@ and manipulate Git repositories by wrapping system calls to the git binary.")
> (version "4.1.0")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/leejarvis/slop/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "slop" version))
> (sha256
> (base32
> - "0cqs50a0b99kjd19xpln8jpnki07cjyp3l7wxbfr44ycasr6nznh"))))
> + "0dj0ps6v1mqd02k84mgwd7hp578n2bzl7c51h3grdhxfl3jkfsj5"))))
> (build-system ruby-build-system)
> (native-inputs
> `(("ruby-minitest" ,ruby-minitest)))
> @@ -689,13 +598,10 @@ options and parsing command line flags.")
> (version "2.0.0")
> (source (origin
> (method url-fetch)
> - (uri (string-append
> - "https://github.com/nicksieger/multipart-post/archive/v"
> - version ".tar.gz"))
> - (file-name (string-append name "-" version ".tar.gz"))
> + (uri (rubygems-uri "multipart-post" version))
> (sha256
> (base32
> - "03n271i3knfx4j9aingxzz2bajd379dw9nswsllviqc177lq1anm"))))
> + "09k0b3cybqilk1gwrwwain95rdypixb2q9w65gd44gfzsd84xi1x"))))
> (build-system ruby-build-system)
> (native-inputs
> `(("bundler" ,bundler)))
> diff --git a/guix/build-system/ruby.scm b/guix/build-system/ruby.scm
> index 135eda6..8142e85 100644
> --- a/guix/build-system/ruby.scm
> +++ b/guix/build-system/ruby.scm
> @@ -26,10 +26,16 @@
> #:use-module (guix build-system)
> #:use-module (guix build-system gnu)
> #:use-module (ice-9 match)
> - #:export (%ruby-build-system-modules
> + #:export (rubygems-uri
> + %ruby-build-system-modules
> ruby-build
> ruby-build-system))
>
> +(define (rubygems-uri name version)
> + "Return a URI string for the gem archive for the release corresponding to
> +NAME and VERSION."
> + (string-append "https://rubygems.org/downloads/" name "-" version ".gem"))
> +
> (define %ruby-build-system-modules
> ;; Build-side modules imported by default.
> `((guix build ruby-build-system)
> @@ -50,24 +56,22 @@
> (define private-keywords
> '(#:source #:target #:ruby #:inputs #:native-inputs))
>
> - (let ((version-control (resolve-interface '(gnu packages version-control))))
> - (and (not target) ;XXX: no cross-compilation
> - (bag
> - (name name)
> - (system system)
> - (host-inputs `(,@(if source
> - `(("source" ,source))
> - '())
> - ,@inputs
> + (and (not target) ;XXX: no cross-compilation
> + (bag
> + (name name)
> + (system system)
> + (host-inputs `(,@(if source
> + `(("source" ,source))
> + '())
> + ,@inputs
>
> - ;; Keep the standard inputs of 'gnu-build-system'.
> - ,@(standard-packages)))
> - (build-inputs `(("ruby" ,ruby)
> - ("git" ,(module-ref version-control 'git))
> - ,@native-inputs))
> - (outputs outputs)
> - (build ruby-build)
> - (arguments (strip-keyword-arguments private-keywords arguments))))))
> + ;; Keep the standard inputs of 'gnu-build-system'.
> + ,@(standard-packages)))
> + (build-inputs `(("ruby" ,ruby)
> + ,@native-inputs))
> + (outputs outputs)
> + (build ruby-build)
> + (arguments (strip-keyword-arguments private-keywords arguments)))))
>
> (define* (ruby-build store name inputs
> #:key
> diff --git a/guix/build/ruby-build-system.scm b/guix/build/ruby-build-system.scm
> index 307ac91..f82e4d4 100644
> --- a/guix/build/ruby-build-system.scm
> +++ b/guix/build/ruby-build-system.scm
> @@ -21,6 +21,7 @@
> #:use-module ((guix build gnu-build-system) #:prefix gnu:)
> #:use-module (guix build utils)
> #:use-module (ice-9 match)
> + #:use-module (ice-9 popen)
> #:use-module (ice-9 regex)
> #:use-module (srfi srfi-1)
> #:use-module (srfi srfi-26)
> @@ -40,41 +41,58 @@ directory."
> ((file-name . _) file-name)
> (() (error "No files matching pattern: " pattern))))
>
> -;; Most gemspecs assume that builds are taking place within a git repository
> -;; by include calls to 'git ls-files'. In order for these gemspecs to work
> -;; as-is, every file in the source tree is added to the staging area.
> -(define gitify
> - (lambda _
> - (and (zero? (system* "git" "init"))
> - (zero? (system* "git" "add" ".")))))
> -
> -(define build
> - (lambda _
> - (match (find-files "." "\\.gemspec$")
> - ;; No gemspec, try 'rake gem' instead.
> - (()
> - (zero? (system* "rake" "gem")))
> - ;; Build the first matching gemspec.
> - ((gemspec . _)
> - (zero? (system* "gem" "build" gemspec))))))
> +(define* (unpack #:key source #:allow-other-keys)
> + "Unpack the gem SOURCE and enter the resulting directory."
> + (and (zero? (system* "gem" "unpack" source))
> + (begin
> + ;; The unpacked gem directory is named the same as the archive, sans
> + ;; the ".gem" extension.
> + (chdir (match:substring (string-match "^(.*)\\.gem$"
> + (basename source))
> + 1))
> + #t)))
>
> (define* (check #:key tests? test-target #:allow-other-keys)
> + "Run the gem's test suite rake task TEST-TARGET. Skip the tests if TESTS?
> +is #f."
> (if tests?
> (zero? (system* "rake" test-target))
> #t))
>
> (define* (install #:key source inputs outputs (gem-flags '())
> #:allow-other-keys)
> + "Install the gem archive SOURCE to the output store item. Additional
> +GEM-FLAGS are passed to the 'gem' invokation, if present."
> (let* ((ruby-version
> (match:substring (string-match "ruby-(.*)\\.[0-9]$"
> (assoc-ref inputs "ruby"))
> 1))
> (out (assoc-ref outputs "out"))
> (gem-home (string-append out "/lib/ruby/gems/" ruby-version ".0")))
> +
> + ;; Extract gemspec from source and build a new gem archive. This allows
> + ;; any dynamic patching done in previous phases to be present in the
> + ;; installed gem.
> + (let ((pipe (open-pipe* OPEN_READ "gem" "spec" "--ruby" source)))
> + (dynamic-wind
> + (const #t)
> + (lambda ()
> + (call-with-output-file ".gemspec"
> + (lambda (out)
> + ;; 'gem spec' writes to stdout, but 'gem build' only reads
> + ;; gemspecs from a file, so we redirect the output to a file.
> + (while (not (eof-object? (peek-char pipe)))
> + (write-char (read-char pipe) out)))))
> + (lambda ()
> + (close-pipe pipe))))
> +
> + (system* "gem" "build" ".gemspec")
> +
> + ;; Install to GEM_HOME.
> (setenv "GEM_HOME" gem-home)
> (mkdir-p gem-home)
> - (zero? (apply system* "gem" "install" "--local"
> - (first-matching-file "\\.gem$")
> + (zero? (apply system* "gem" "install" (first-matching-file "\\.gem$")
> + "--local" "--ignore-dependencies"
> ;; Executables should go into /bin, not /lib/ruby/gems.
> "--bindir" (string-append out "/bin")
> gem-flags))))
> @@ -82,8 +100,8 @@ directory."
> (define %standard-phases
> (modify-phases gnu:%standard-phases
> (delete 'configure)
> - (add-after 'unpack 'gitify gitify)
> - (replace 'build build)
> + (delete 'build)
> + (replace 'unpack unpack)
> (replace 'install install)
> (replace 'check check)))
>
> --
> 2.4.3
>
--
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH] build: ruby: Rewrite build system to use gem archives.
2015-08-16 22:19 ` Thompson, David
2015-08-17 5:43 ` Pjotr Prins
@ 2015-08-17 23:31 ` Thompson, David
2015-08-18 6:40 ` Ricardo Wurmus
2015-08-18 15:18 ` Ludovic Courtès
2 siblings, 1 reply; 11+ messages in thread
From: Thompson, David @ 2015-08-17 23:31 UTC (permalink / raw)
To: guix-devel
[-- Attachment #1: Type: text/plain, Size: 1634 bytes --]
On Sun, Aug 16, 2015 at 6:19 PM, Thompson, David
<dthompson2@worcester.edu> wrote:
> Update:
>
> On Tue, Jul 28, 2015 at 9:09 AM, David Thompson
> <dthompson2@worcester.edu> wrote:
>
>> One other caveat is that downloading gems from rubygems.org currently
>> *does not* work because the HTTP responses include an invalid
>> Content-Type header. I added a temporary hack to do my development
>> work (not in this patch), and I'm currently waiting on a response from
>> the rubygems.org admins to see if they will fix the issue. So, this
>> patch should not be applied until this is resolved.
>>
>> See: http://help.rubygems.org/discussions/problems/21223-invalid-content-type-header-when-downloading-gem-archive?unresolve=true
>
> This issue has been fixed by the rubygems.org admins! I would like to
> apply the below patch ASAP and get to work on 'guix import gem' if
> there are no objections.
>
> The major change from the previous version of this patch is that I
> figured how to unpack the source gem, allow the package build script
> to make any necessary dynamic patches, and then install a repacked gem
> as opposed to the source gem.
I found a problem with this system when it came to running certain
test suites that used Bundler to read the gemspec. A lot of upstream
gemspec include things like shell commands to run 'git ls-files' and
things. In the former build system, the 'gitify' phase took care of
this. Now, we can more easily deal with it by asking the 'gem'
utility to generate a gemspec for us based on the source gem archive.
Hope that wasn't too confusing for the non-Rubyists.
TIA for review.
- Dave
[-- Attachment #2: 0001-build-ruby-Rewrite-build-system-to-use-gem-archives.patch --]
[-- Type: text/x-patch, Size: 28319 bytes --]
From 17b7307b11dd5825d7ff881624c4b823056bed50 Mon Sep 17 00:00:00 2001
From: David Thompson <dthompson2@worcester.edu>
Date: Sun, 26 Jul 2015 22:01:54 -0400
Subject: [PATCH] build: ruby: Rewrite build system to use gem archives.
Co-Authored-By: Pjotr Prins <pjotr.public01@thebird.nl>
* guix/build-system/ruby.scm (lower): Remove git dependency.
(rubygems-uri): New procedure.
* guix/build/ruby-build-system (gitify): Delete.
(unpack): Use 'gem unpack' utility.
(check): Add docstring.
(build): Repack modified gem.
(install): Rebuild unpacked gem and install it.
(%standard-phases): Remove gitify and build phases.
* gnu/packages/ruby.scm (ruby-hoe, ruby-rake-compiler, ruby-i18n,
ruby-rspec-support, ruby-rspec-core, ruby-diff-lcs-for-rspec,
ruby-rspec-expectations, ruby-rspec-mocks, ruby-rspec, bundler,
ruby-useragent, ruby-bacon, ruby-arel, ruby-connection-pool,
ruby-net-http-persistent, ruby-minitest, ruby-minitest-sprint,
ruby-minitest-bacon, ruby-daemons, ruby-git, ruby-slop,
ruby-multipart-post): Convert to new build system.
---
gnu/packages/ruby.scm | 222 +++++++++++----------------------------
guix/build-system/ruby.scm | 40 +++----
guix/build/ruby-build-system.scm | 72 +++++++++----
3 files changed, 138 insertions(+), 196 deletions(-)
diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index a481365..196094c 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -154,13 +154,11 @@ a focus on simplicity and productivity.")
(name "ruby-hoe")
(version "3.13.1")
(source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/seattlerb/hoe.git")
- (commit "0c11836"))) ; no release tags :(
+ (method url-fetch)
+ (uri (rubygems-uri "hoe" version))
(sha256
(base32
- "0i8dimf8kxcjgqj9x65bbi3l6hc9p9gbfbb1vmrz42764a4jjbz9"))) )
+ "1mac13krdrasn9819dd65xj27kklfy0xdbj3p6s2ij4vlcb46h8q"))) )
(build-system ruby-build-system)
(synopsis "Ruby project management helper")
(description
@@ -178,22 +176,13 @@ announcement.")
(version "0.9.5")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/rake-compiler/rake-compiler/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "rake-compiler" version))
(sha256
(base32
- "07lk1vl0jqcaqwjjhmg0qshqwcxdyr5kscc9xxm13m03835xgpf3"))))
+ "1k8im2vzj849xdgjk6wafspkiwwapqwm738majchb4dnhnsk64cx"))))
(build-system ruby-build-system)
(arguments
- '(#:tests? #f ; needs cucumber
- #:phases (modify-phases %standard-phases
- (add-before 'build 'remove-cucumber-rake-task
- (lambda _
- ;; Remove cucumber test file because the
- ;; dependencies are not available right now.
- (delete-file "tasks/cucumber.rake"))))))
+ '(#:tests? #f)) ; needs cucumber
(synopsis "Building and packaging helper for Ruby native extensions")
(description "Rake-compiler provides a framework for building and
packaging native C and Java extensions in Ruby.")
@@ -206,12 +195,10 @@ packaging native C and Java extensions in Ruby.")
(version "0.6.11")
(source (origin
(method url-fetch)
- (uri (string-append "https://github.com/svenfuchs/i18n/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "i18n" version))
(sha256
(base32
- "1fdhnhh1p5g8vibv44d770z8nq208zrms3m2nswdvr54072y1m6k"))))
+ "0fwjlgmgry2blf8zlxn9c555cf4a16p287l599kz5104ncjxlzdk"))))
(build-system ruby-build-system)
(arguments
'(#:tests? #f)) ; requires bundler
@@ -234,13 +221,10 @@ an extensible architecture with a swappable backend.")
(version "3.2.2")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/rspec/rspec-support/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "rspec-support" version))
(sha256
(base32
- "1pvzfrqgy0z0gwmdgjp9f2vz1d9c0cajyzfqj9z8i2ssxnzmj4bv"))))
+ "194zry5195ls2hni7r9824vqb5d3qfg4jb15fgj8glfy0rvw3zxl"))))
(build-system ruby-build-system)
(arguments
'(#:tests? #f)) ; avoid dependency cycles
@@ -255,13 +239,10 @@ an extensible architecture with a swappable backend.")
(version "3.2.3")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/rspec/rspec-core/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "rspec-core" version))
(sha256
(base32
- "1clsa4lkh5c9c7xc3xa336ym00ycr67pchpg1bv4y3fz5hvzw8ki"))))
+ "0k2471iw30gc2cvv67damrx666pmsvx8l0ahk3hm20dhfnmcmpvv"))))
(build-system ruby-build-system)
(arguments
'(#:tests? #f)) ; avoid dependency cycles
@@ -279,13 +260,10 @@ groups.")
(version "1.2.5")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/halostatue/diff-lcs/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "diff-lcs" version))
(sha256
(base32
- "0kmfz2qdwbfjf97rx27hh9fm39mv3z9avjmvsajqnb5wxj2l5l4s"))))
+ "1vf9civd41bnqi6brr5d9jifdw73j9khc6fkhfl1f8r9cpkdvlx1"))))
(build-system ruby-build-system)
(arguments
'(#:tests? #f)) ; avoid dependency cycles
@@ -303,13 +281,10 @@ standard diff-like tool.")
(version "3.2.1")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/rspec/rspec-expectations/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "rspec-expectations" version))
(sha256
(base32
- "0h0rpprbh6h59gmksiyi1b8w6cvcai4wdbkikajwx3w1asxi6f7x"))))
+ "01kmchabgpdcaqdsqg8r0g5gy385xhp1k1jsds3w264ypin17n14"))))
(build-system ruby-build-system)
(arguments
'(#:tests? #f)) ; avoid dependency cycles
@@ -328,13 +303,10 @@ outcomes of a code example.")
(version "3.2.1")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/rspec/rspec-mocks/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "rspec-mocks" version))
(sha256
(base32
- "1xzxsg0idxkg7czmjgqq10lcd821ibw1hjzn404sk9j6rw0fbx2g"))))
+ "09yig1lwgxl8fsns71z3xhv7wkg7zvagydh37pvaqpw92dz55jv2"))))
(build-system ruby-build-system)
(arguments
'(#:tests? #f)) ; avoid dependency cycles
@@ -353,13 +325,10 @@ support for stubbing and mocking.")
(version "3.2.0")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/rspec/rspec/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "rspec" version))
(sha256
(base32
- "1jg38dbaknsdhiav5vnrwfccg524fwcg6sq1715441vx1xl6p54q"))))
+ "0lkz01j4yxcwb3g5w6r1l9khnyw3sxib4rrh4npd2pxh390fcc4f"))))
(build-system ruby-build-system)
(arguments
'(#:tests? #f)) ; avoid dependency cycles
@@ -382,12 +351,10 @@ expectations and mocks frameworks.")
(version "1.9.9")
(source (origin
(method url-fetch)
- (uri (string-append "https://github.com/bundler/bundler/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "bundler" version))
(sha256
(base32
- "08flx3n9hb3yz8mm5k16cdz0sb7g774f6vxn6gc3wfh5la83vfyx"))))
+ "12qk1569pswa9mmid6lsqy2napn9fmkbmv0k7xkl52nyfh8rsy4d"))))
(build-system ruby-build-system)
(arguments
'(#:tests? #f)) ; avoid dependency cycles
@@ -403,19 +370,13 @@ specified in a \"Gemfile\", as well as their dependencies.")
(version "0.13.3")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/gshutler/useragent/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "useragent" version))
(sha256
(base32
- "1hj00fw06i0y3rwxxhxmnrqxhpnffv4zfqx2sqqpc5qc4fdvd2x9"))))
+ "0kz7yyz7528bv4a2kfymvkcm8whqcddhmgaw1ksw1d90n30hhkpc"))))
(build-system ruby-build-system)
(arguments
- '(#:test-target "spec"))
- (native-inputs
- `(("ruby-rspec" ,ruby-rspec)
- ("bundler" ,bundler)))
+ '(#:tests? #f)) ; no test suite
(synopsis "HTTP user agent parser for Ruby")
(description "UserAgent is a Ruby library that parses and compares HTTP
User Agents.")
@@ -425,26 +386,14 @@ User Agents.")
(define-public ruby-bacon
(package
(name "ruby-bacon")
- (version "1.2")
+ (version "1.2.0")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/chneukirchen/bacon/archive/"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "bacon" version))
(sha256
(base32
- "0g03fxilrrx17dijww68n1lq5d8s69hrxgpga8c1i2k35bzcw5jc"))))
+ "1f06gdj77bmwzc1k5iragl1595hbn67yc7sqvs56ca8plrr2vmai"))))
(build-system ruby-build-system)
- (arguments
- `(#:phases (modify-phases %standard-phases
- (add-before 'build 'generate-docs
- (lambda _
- ;; This rake task also tries to generate a ChangeLog
- ;; file from the Git log, which we don't have. It fails
- ;; but creates an empty file, allowing the rest of the
- ;; build to succeed.
- (zero? (system* "rake" "predist")))))))
(synopsis "Small RSpec clone")
(description "Bacon is a small RSpec clone providing all essential
features.")
@@ -457,16 +406,13 @@ features.")
(version "6.0.0")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/rails/arel/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "arel" version))
(sha256
(base32
- "0fldwp2hmrmddx22xf7hdmybngzv97qnv5rvz3qw61m94ddd6w4n"))))
+ "18wnfnzr2i5p3fygsddjbi1cimws6823nbk8drxidmnj8jz7h0ar"))))
(build-system ruby-build-system)
- (native-inputs
- `(("bundler" ,bundler)))
+ (arguments
+ '(#:tests? #f)) ; no test suite
(synopsis "SQL AST manager for Ruby")
(description "Arel is a SQL AST manager for Ruby. It simplifies the
generation of complex SQL queries and adapts to various relational database
@@ -480,13 +426,10 @@ implementations.")
(version "2.2.0")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/mperham/connection_pool/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "connection_pool" version))
(sha256
(base32
- "02s5rwhmgy8qqns7a5y1daa0yaw38x6lzpwyvmy46h1yrj9mc6zf"))))
+ "1b2bb3k39ni5mzcnqlv9y4yjkbin20s7dkwzp0jw2jf1rmzcgrmy"))))
(build-system ruby-build-system)
(native-inputs
`(("bundler" ,bundler)))
@@ -502,13 +445,10 @@ interface for Ruby programs.")
(version "2.9.4")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/drbrain/net-http-persistent/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "net-http-persistent" version))
(sha256
(base32
- "1q18vji31w8gfr6ajziqkqs8n5lzkw0bl00dm2a8s05zhavzw9j9"))))
+ "1y9fhaax0d9kkslyiqi1zys6cvpaqx9a0y0cywp24rpygwh4s9r4"))))
(build-system ruby-build-system)
(native-inputs
`(("ruby-connection-pool" ,ruby-connection-pool)
@@ -524,16 +464,11 @@ using Net::HTTP, supporting reconnection and retry according to RFC 2616.")
(name "ruby-minitest")
(version "5.7.0")
(source (origin
- (method git-fetch)
- ;; No release tarballs nor git tags. This is the commit
- ;; corresponding to the addition of the release notes to
- ;; History.rdoc.
- (uri (git-reference
- (url "https://github.com/seattlerb/minitest.git")
- (commit "e975248")))
+ (method url-fetch)
+ (uri (rubygems-uri "minitest" version))
(sha256
(base32
- "09xjiahk7q8hid1i39ahrmghaslpj9n36zna72i3ah7kf1bh2l01"))))
+ "0rxqfakp629mp3vwda7zpgb57lcns5znkskikbfd0kriwv8i1vq8"))))
(build-system ruby-build-system)
(native-inputs
`(("ruby-hoe" ,ruby-hoe)))
@@ -548,14 +483,11 @@ facilities supporting TDD, BDD, mocking, and benchmarking.")
(name "ruby-minitest-sprint")
(version "1.1.0")
(source (origin
- (method git-fetch)
- ;; Same story as ruby-minitest.
- (uri (git-reference
- (url "https://github.com/seattlerb/minitest-sprint.git")
- (commit "49c02bc")))
+ (method url-fetch)
+ (uri (rubygems-uri "minitest-sprint" version))
(sha256
(base32
- "0rbmxz94lqg5vjz60p8v2bzq8adwvmx501amvk0l124sfwmw94ms"))))
+ "179d6pj56l9xzm46fqsqj10mzjkr1f9fv4cxa8wvchs97hqz33w1"))))
(build-system ruby-build-system)
(native-inputs
`(("ruby-hoe" ,ruby-hoe)
@@ -571,14 +503,11 @@ easier to re-run individual failing tests.")
(name "ruby-minitest-bacon")
(version "1.0.2")
(source (origin
- (method git-fetch)
- ;; Same story as ruby-minitest.
- (uri (git-reference
- (url "https://github.com/seattlerb/minitest-bacon.git")
- (commit "38551d5")))
+ (method url-fetch)
+ (uri (rubygems-uri "minitest-bacon" version))
(sha256
(base32
- "19r9fm41i0mm1xncqls8frbj1i9nr3sq1cx2mh878r6kdl02d70h"))))
+ "0cm7r68422743i3b6fm4rrm0r6cnnjmglq5gcmmgl1f0rk5hnf6r"))))
(build-system ruby-build-system)
(native-inputs
`(("ruby-hoe" ,ruby-hoe)))
@@ -596,13 +525,10 @@ functionality, making it easier to migrate test suites from bacon to minitest.")
(version "1.2.2")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/thuehlinger/daemons/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "daemons" version))
(sha256
(base32
- "1v5bpdvpvhk240pc7fkn44vfclppl44pp6wd42ipi5sd5lkk7zfd"))))
+ "121c7vkimg3baxga69xvdkwxiq8wkmxqvdbyqi5i82vhih5d3cn3"))))
(build-system ruby-build-system)
(arguments
`(#:tests? #f)) ; no test suite
@@ -618,41 +544,27 @@ run as a daemon and to be controlled by simple start/stop/restart commands.")
(version "1.2.9.1")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/schacon/ruby-git/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "git" version))
(sha256
(base32
- "08zg20zc7f7yy34ix2qdd8jbiz7xhjc8alk370869sq3h75hs9jc"))))
+ "1sqfj8lmhl7c5zamcckkpik4izfph2zkv6krw0i8mzj5pdws5acs"))))
(build-system ruby-build-system)
(arguments
- '(#:phases (modify-phases %standard-phases
- (add-before 'build 'patch-git-binary
- (lambda* (#:key inputs #:allow-other-keys)
+ `(#:tests? #f ; no tests
+ #:phases (modify-phases %standard-phases
+ (add-after 'install 'patch-git-binary
+ (lambda* (#:key inputs outputs #:allow-other-keys)
;; Make the default git binary an absolute path to the
;; store.
- (let ((git (string-append (assoc-ref inputs "git")
- "/bin/git")))
- (substitute* '("lib/git/config.rb")
+ (let ((git (string-append (assoc-ref inputs "git")
+ "/bin/git"))
+ (config (string-append (getenv "GEM_HOME")
+ "/gems/git-" ,version
+ "/lib/git/config.rb")))
+ (substitute* (list config)
(("'git'")
(string-append "'" git "'")))
- ;; Fix a test that expects the binary to be simply
- ;; 'git'.
- (substitute* '("tests/units/test_logger.rb")
- (("def test_logger")
- (string-append
- "def test_logger\n"
- "Git::Base.config.binary_path = 'git'")))
- #t)))
- (add-before 'check 'create-fake-home
- (lambda _
- ;; The test suite runs 'git config --global' commands,
- ;; so a fake home directory is needed for them to
- ;; succeed.
- (let ((fake-home (string-append (getcwd) "/fake-home")))
- (mkdir fake-home)
- (setenv "HOME" fake-home)))))))
+ #t))))))
(inputs
`(("git" ,git)))
(synopsis "Ruby wrappers for Git")
@@ -667,13 +579,10 @@ and manipulate Git repositories by wrapping system calls to the git binary.")
(version "4.1.0")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/leejarvis/slop/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "slop" version))
(sha256
(base32
- "0cqs50a0b99kjd19xpln8jpnki07cjyp3l7wxbfr44ycasr6nznh"))))
+ "0dj0ps6v1mqd02k84mgwd7hp578n2bzl7c51h3grdhxfl3jkfsj5"))))
(build-system ruby-build-system)
(native-inputs
`(("ruby-minitest" ,ruby-minitest)))
@@ -689,13 +598,10 @@ options and parsing command line flags.")
(version "2.0.0")
(source (origin
(method url-fetch)
- (uri (string-append
- "https://github.com/nicksieger/multipart-post/archive/v"
- version ".tar.gz"))
- (file-name (string-append name "-" version ".tar.gz"))
+ (uri (rubygems-uri "multipart-post" version))
(sha256
(base32
- "03n271i3knfx4j9aingxzz2bajd379dw9nswsllviqc177lq1anm"))))
+ "09k0b3cybqilk1gwrwwain95rdypixb2q9w65gd44gfzsd84xi1x"))))
(build-system ruby-build-system)
(native-inputs
`(("bundler" ,bundler)))
diff --git a/guix/build-system/ruby.scm b/guix/build-system/ruby.scm
index 135eda6..8142e85 100644
--- a/guix/build-system/ruby.scm
+++ b/guix/build-system/ruby.scm
@@ -26,10 +26,16 @@
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
#:use-module (ice-9 match)
- #:export (%ruby-build-system-modules
+ #:export (rubygems-uri
+ %ruby-build-system-modules
ruby-build
ruby-build-system))
+(define (rubygems-uri name version)
+ "Return a URI string for the gem archive for the release corresponding to
+NAME and VERSION."
+ (string-append "https://rubygems.org/downloads/" name "-" version ".gem"))
+
(define %ruby-build-system-modules
;; Build-side modules imported by default.
`((guix build ruby-build-system)
@@ -50,24 +56,22 @@
(define private-keywords
'(#:source #:target #:ruby #:inputs #:native-inputs))
- (let ((version-control (resolve-interface '(gnu packages version-control))))
- (and (not target) ;XXX: no cross-compilation
- (bag
- (name name)
- (system system)
- (host-inputs `(,@(if source
- `(("source" ,source))
- '())
- ,@inputs
+ (and (not target) ;XXX: no cross-compilation
+ (bag
+ (name name)
+ (system system)
+ (host-inputs `(,@(if source
+ `(("source" ,source))
+ '())
+ ,@inputs
- ;; Keep the standard inputs of 'gnu-build-system'.
- ,@(standard-packages)))
- (build-inputs `(("ruby" ,ruby)
- ("git" ,(module-ref version-control 'git))
- ,@native-inputs))
- (outputs outputs)
- (build ruby-build)
- (arguments (strip-keyword-arguments private-keywords arguments))))))
+ ;; Keep the standard inputs of 'gnu-build-system'.
+ ,@(standard-packages)))
+ (build-inputs `(("ruby" ,ruby)
+ ,@native-inputs))
+ (outputs outputs)
+ (build ruby-build)
+ (arguments (strip-keyword-arguments private-keywords arguments)))))
(define* (ruby-build store name inputs
#:key
diff --git a/guix/build/ruby-build-system.scm b/guix/build/ruby-build-system.scm
index 307ac91..90fab92 100644
--- a/guix/build/ruby-build-system.scm
+++ b/guix/build/ruby-build-system.scm
@@ -21,6 +21,7 @@
#:use-module ((guix build gnu-build-system) #:prefix gnu:)
#:use-module (guix build utils)
#:use-module (ice-9 match)
+ #:use-module (ice-9 popen)
#:use-module (ice-9 regex)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -40,41 +41,72 @@ directory."
((file-name . _) file-name)
(() (error "No files matching pattern: " pattern))))
-;; Most gemspecs assume that builds are taking place within a git repository
-;; by include calls to 'git ls-files'. In order for these gemspecs to work
-;; as-is, every file in the source tree is added to the staging area.
-(define gitify
- (lambda _
- (and (zero? (system* "git" "init"))
- (zero? (system* "git" "add" ".")))))
+(define* (unpack #:key source #:allow-other-keys)
+ "Unpack the gem SOURCE and enter the resulting directory."
+ (and (zero? (system* "gem" "unpack" source))
+ (begin
+ ;; The unpacked gem directory is named the same as the archive, sans
+ ;; the ".gem" extension.
+ (chdir (match:substring (string-match "^(.*)\\.gem$"
+ (basename source))
+ 1))
+ #t)))
-(define build
- (lambda _
- (match (find-files "." "\\.gemspec$")
- ;; No gemspec, try 'rake gem' instead.
- (()
- (zero? (system* "rake" "gem")))
- ;; Build the first matching gemspec.
- ((gemspec . _)
- (zero? (system* "gem" "build" gemspec))))))
+(define* (build #:key source #:allow-other-keys)
+ "Build a new gem using the gemspec from the SOURCE gem."
+
+ ;; Remove the original gemspec, if present, and replace it with a new one.
+ ;; This avoids issues with upstream gemspecs requiring tools such as git to
+ ;; generate the files list.
+ (let ((gemspec (or (false-if-exception
+ (first-matching-file "\\.gemspec$"))
+ ;; Make new gemspec if one wasn't shipped.
+ ".gemspec")))
+
+ (when (file-exists? gemspec) (delete-file gemspec))
+
+ ;; Extract gemspec from source gem.
+ (let ((pipe (open-pipe* OPEN_READ "gem" "spec" "--ruby" source)))
+ (dynamic-wind
+ (const #t)
+ (lambda ()
+ (call-with-output-file gemspec
+ (lambda (out)
+ ;; 'gem spec' writes to stdout, but 'gem build' only reads
+ ;; gemspecs from a file, so we redirect the output to a file.
+ (while (not (eof-object? (peek-char pipe)))
+ (write-char (read-char pipe) out))))
+ #t)
+ (lambda ()
+ (close-pipe pipe))))
+
+ ;; Build a new gem from the current working directory. This also allows any
+ ;; dynamic patching done in previous phases to be present in the installed
+ ;; gem.
+ (zero? (system* "gem" "build" gemspec))))
(define* (check #:key tests? test-target #:allow-other-keys)
+ "Run the gem's test suite rake task TEST-TARGET. Skip the tests if TESTS?
+is #f."
(if tests?
(zero? (system* "rake" test-target))
#t))
-(define* (install #:key source inputs outputs (gem-flags '())
+(define* (install #:key inputs outputs (gem-flags '())
#:allow-other-keys)
+ "Install the gem archive SOURCE to the output store item. Additional
+GEM-FLAGS are passed to the 'gem' invokation, if present."
(let* ((ruby-version
(match:substring (string-match "ruby-(.*)\\.[0-9]$"
(assoc-ref inputs "ruby"))
1))
(out (assoc-ref outputs "out"))
(gem-home (string-append out "/lib/ruby/gems/" ruby-version ".0")))
+
(setenv "GEM_HOME" gem-home)
(mkdir-p gem-home)
- (zero? (apply system* "gem" "install" "--local"
- (first-matching-file "\\.gem$")
+ (zero? (apply system* "gem" "install" (first-matching-file "\\.gem$")
+ "--local" "--ignore-dependencies"
;; Executables should go into /bin, not /lib/ruby/gems.
"--bindir" (string-append out "/bin")
gem-flags))))
@@ -82,8 +114,8 @@ directory."
(define %standard-phases
(modify-phases gnu:%standard-phases
(delete 'configure)
- (add-after 'unpack 'gitify gitify)
(replace 'build build)
+ (replace 'unpack unpack)
(replace 'install install)
(replace 'check check)))
--
2.4.3
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH] build: ruby: Rewrite build system to use gem archives.
2015-08-17 23:31 ` Thompson, David
@ 2015-08-18 6:40 ` Ricardo Wurmus
2015-08-18 12:34 ` Thompson, David
0 siblings, 1 reply; 11+ messages in thread
From: Ricardo Wurmus @ 2015-08-18 6:40 UTC (permalink / raw)
To: Thompson, David; +Cc: guix-devel
Thompson, David <dthompson2@worcester.edu> writes:
>> The major change from the previous version of this patch is that I
>> figured how to unpack the source gem, allow the package build script
>> to make any necessary dynamic patches, and then install a repacked gem
>> as opposed to the source gem.
>
> I found a problem with this system when it came to running certain
> test suites that used Bundler to read the gemspec. A lot of upstream
> gemspec include things like shell commands to run 'git ls-files' and
> things. In the former build system, the 'gitify' phase took care of
> this. Now, we can more easily deal with it by asking the 'gem'
> utility to generate a gemspec for us based on the source gem archive.
> Hope that wasn't too confusing for the non-Rubyists.
I just read through the patch set. Looks good! (Note: I haven’t
actually tested this yet.)
It seems that you had to disable tests for “useragent”, “arel”, “git”; I
assume that’s because the gems do not come with the necessary test
files. I remember a link to a discussion you once shared, in which it
was said that the declaration of test files in gemspec manifests is
deprecated. Do you think the absence of tests in the published gems is
going to be a problem for many more packages?
~~ Ricardo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] build: ruby: Rewrite build system to use gem archives.
2015-08-18 6:40 ` Ricardo Wurmus
@ 2015-08-18 12:34 ` Thompson, David
2015-08-18 14:55 ` Pjotr Prins
0 siblings, 1 reply; 11+ messages in thread
From: Thompson, David @ 2015-08-18 12:34 UTC (permalink / raw)
To: Ricardo Wurmus; +Cc: guix-devel
On Tue, Aug 18, 2015 at 2:40 AM, Ricardo Wurmus <rekado@elephly.net> wrote:
>
> Thompson, David <dthompson2@worcester.edu> writes:
>
>>> The major change from the previous version of this patch is that I
>>> figured how to unpack the source gem, allow the package build script
>>> to make any necessary dynamic patches, and then install a repacked gem
>>> as opposed to the source gem.
>>
>> I found a problem with this system when it came to running certain
>> test suites that used Bundler to read the gemspec. A lot of upstream
>> gemspec include things like shell commands to run 'git ls-files' and
>> things. In the former build system, the 'gitify' phase took care of
>> this. Now, we can more easily deal with it by asking the 'gem'
>> utility to generate a gemspec for us based on the source gem archive.
>> Hope that wasn't too confusing for the non-Rubyists.
>
> I just read through the patch set. Looks good! (Note: I haven’t
> actually tested this yet.)
>
> It seems that you had to disable tests for “useragent”, “arel”, “git”; I
> assume that’s because the gems do not come with the necessary test
> files. I remember a link to a discussion you once shared, in which it
> was said that the declaration of test files in gemspec manifests is
> deprecated. Do you think the absence of tests in the published gems is
> going to be a problem for many more packages?
It's hit and miss. Some have them, some don't. I imported a bunch of
gems to mess around with last night and a good number of them had
runnable test suites, but not all. Pjotr showed me that the official
RubyGems guide recommends including test files:
http://guides.rubygems.org/what-is-a-gem/
I had a conversation with Mark about this in the context of dealing
with Node packages, which are definitely more of a pain than gems, and
he convinced me that we cannot let the perfect get in the way of the
good. Making it really easy to use the gem archives that Ruby
developers are releasing with Guix is an important first step to
making packaging better for everyone. So, if we have to disable some
test suites to get our Ruby support off the ground, so be it.
Thanks for the review,
- Dave
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] build: ruby: Rewrite build system to use gem archives.
2015-08-18 12:34 ` Thompson, David
@ 2015-08-18 14:55 ` Pjotr Prins
0 siblings, 0 replies; 11+ messages in thread
From: Pjotr Prins @ 2015-08-18 14:55 UTC (permalink / raw)
To: Thompson, David; +Cc: guix-devel
On Tue, Aug 18, 2015 at 08:34:44AM -0400, Thompson, David wrote:
> making packaging better for everyone. So, if we have to disable some
> test suites to get our Ruby support off the ground, so be it.
It is actually not a big deal for gem writers to include their tests
(most have them). Amounts to 2-3 lines in the gemspec, e.g.
https://github.com/pjotrp/bioruby-vcf/blob/master/bio-vcf.gemspec#L31
For rapid progress, I think it is a good idea for Guix to disable
tests for a gem when they are not included, and at the same time
suggest the authors to include tests in their next release (as per the
rubygems spec). With luck we can enable testing again down the line.
It won't always happen, but I think we can aim for 70%+ of gems to
have their tests included.
Pj.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] build: ruby: Rewrite build system to use gem archives.
2015-08-16 22:19 ` Thompson, David
2015-08-17 5:43 ` Pjotr Prins
2015-08-17 23:31 ` Thompson, David
@ 2015-08-18 15:18 ` Ludovic Courtès
2015-08-18 22:09 ` Thompson, David
2 siblings, 1 reply; 11+ messages in thread
From: Ludovic Courtès @ 2015-08-18 15:18 UTC (permalink / raw)
To: Thompson, David; +Cc: guix-devel
"Thompson, David" <dthompson2@worcester.edu> skribis:
> Update:
>
> On Tue, Jul 28, 2015 at 9:09 AM, David Thompson
> <dthompson2@worcester.edu> wrote:
>
>> One other caveat is that downloading gems from rubygems.org currently
>> *does not* work because the HTTP responses include an invalid
>> Content-Type header. I added a temporary hack to do my development
>> work (not in this patch), and I'm currently waiting on a response from
>> the rubygems.org admins to see if they will fix the issue. So, this
>> patch should not be applied until this is resolved.
>>
>> See: http://help.rubygems.org/discussions/problems/21223-invalid-content-type-header-when-downloading-gem-archive?unresolve=true
>
> This issue has been fixed by the rubygems.org admins!
Nice, thanks to them!
> I would like to apply the below patch ASAP and get to work on 'guix
> import gem' if there are no objections.
No objections here.
> From ee027442dd968734831eb372fcf502dcfaad7623 Mon Sep 17 00:00:00 2001
> From: David Thompson <dthompson2@worcester.edu>
> Date: Sun, 26 Jul 2015 22:01:54 -0400
> Subject: [PATCH] build: ruby: Rewrite build system to use gem archives.
>
> Co-Authored-By: Pjotr Prins <pjotr.public01@thebird.nl>
>
> * guix/build-system/ruby.scm (lower): Remove git dependency.
> (rubygems-uri): New procedure.
> * guix/build/ruby-build-system (gitify, build): Delete.
> (unpack): Use 'gem unpack' utility.
> (check): Add docstring.
> (install): Rebuild unpacked gem and install it.
> (%standard-phases): Remove gitify and build phases.
> * gnu/packages/ruby.scm (ruby-hoe, ruby-rake-compiler, ruby-i18n,
> ruby-rspec-support, ruby-rspec-core, ruby-diff-lcs-for-rspec,
> ruby-rspec-expectations, ruby-rspec-mocks, ruby-rspec, bundler,
> ruby-useragent, ruby-bacon, ruby-arel, ruby-connection-pool,
> ruby-net-http-persistent, ruby-minitest, ruby-minitest-sprint,
> ruby-minitest-bacon, ruby-daemons, ruby-git, ruby-slop,
> ruby-multipart-post): Convert to new build system.
Does guix.texi need an update?
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] build: ruby: Rewrite build system to use gem archives.
2015-08-18 15:18 ` Ludovic Courtès
@ 2015-08-18 22:09 ` Thompson, David
0 siblings, 0 replies; 11+ messages in thread
From: Thompson, David @ 2015-08-18 22:09 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
On Tue, Aug 18, 2015 at 11:18 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> "Thompson, David" <dthompson2@worcester.edu> skribis
>
>> I would like to apply the below patch ASAP and get to work on 'guix
>> import gem' if there are no objections.
>
> No objections here.
>
>> From ee027442dd968734831eb372fcf502dcfaad7623 Mon Sep 17 00:00:00 2001
>> From: David Thompson <dthompson2@worcester.edu>
>> Date: Sun, 26 Jul 2015 22:01:54 -0400
>> Subject: [PATCH] build: ruby: Rewrite build system to use gem archives.
>>
>> Co-Authored-By: Pjotr Prins <pjotr.public01@thebird.nl>
>>
>> * guix/build-system/ruby.scm (lower): Remove git dependency.
>> (rubygems-uri): New procedure.
>> * guix/build/ruby-build-system (gitify, build): Delete.
>> (unpack): Use 'gem unpack' utility.
>> (check): Add docstring.
>> (install): Rebuild unpacked gem and install it.
>> (%standard-phases): Remove gitify and build phases.
>> * gnu/packages/ruby.scm (ruby-hoe, ruby-rake-compiler, ruby-i18n,
>> ruby-rspec-support, ruby-rspec-core, ruby-diff-lcs-for-rspec,
>> ruby-rspec-expectations, ruby-rspec-mocks, ruby-rspec, bundler,
>> ruby-useragent, ruby-bacon, ruby-arel, ruby-connection-pool,
>> ruby-net-http-persistent, ruby-minitest, ruby-minitest-sprint,
>> ruby-minitest-bacon, ruby-daemons, ruby-git, ruby-slop,
>> ruby-multipart-post): Convert to new build system.
>
> Does guix.texi need an update?
I added a paragraph about the requiring a gem archive in the source
field and pushed.
Thanks!
- Dave
^ permalink raw reply [flat|nested] 11+ messages in thread