Giving up on RubyGems

[David Thompson <dthompson2@worcester.edu>]

Hello Guix hackers,

As some of you know, I've been working on Ruby support for Guix for
about a year now.  In that time, I helped write and rewrite a Ruby gem
build system, wrote an importer for <https://rubygems.org>, and packaged
many Ruby gems.

At various points, I've had my doubts about the gem archives hosted on
the RubyGems website: Are they source code?  Are they binaries?  After a
good deal of debate, we came to the conclusion that they are source
code.  This seems to be the case when you inspect any given gem.  The
Ruby source code is there, and so is the C source code needed for native
extensions when there is a native extension to be built.  Furthermore,
the RubyGems website says that, among other things, gems should contain
"code (including tests and supporting utilities)." [0]

However, it has become clear that the RubyGems maintainers do not
actually feel this way.  From their perspective, gems are binaries, not
source code.  I discovered this once I noticed that several popular Ruby
gems such as Arel do not, and refuse to [1], ship the test suite in
their releases.  This is because they view gems as binaries that need to
be as slim as possible, containing only necessary runtime files, to cut
down on bandwidth usage and storage space.  Unfortunately, they have no
notion of a source package that corresponds to a given binary.

In practice, I've found that all the gems I've packaged come with source
code and no binaries, they might just be missing the test suite.  So, I
asked the RubyGems maintainers to consider the use-cases for including
test suites, which spawned a large thread on their GitHub page
yesterday. [2]  The end result is this depressing quote:

    Yes, gems are effectively binary packages delivered to
    end-users. Some gems contain ruby source code, some contain
    pre-compiled binaries, some contain both. The internals of a
    particular gem aren't relevant from the perspective of RubyGems
    itself.
    
    As has been pointed out here, RubyGems does not provide packages
    containing gem source code. To be honest, RubyGems as a system does
    not care about gem source code—it accepts .gem files from gem
    authors, and distributes those files on request. Any gem author who
    wishes to provide a link to the source code used to produce a gem is
    welcome to use the gemspec metadata fields to do so.

I've grown very tired of trying to convince people that independent user
verification of binary releases is an important thing to prioritize, but
they think that users do not want the source code.  I've tried to make
my arguments as clear as I could, yet they've been misunderstood by some
and rejected entirely by others, and now it is time to give up.  I don't
know what the best way forward for Ruby support in Guix is.  Things like
the RubyGems importer seem useless now.  Just downloading release
tarballs from GitHub doesn't work without major hacks because almost
every gem (thanks to a terrible script in Bundler that generates
boilerplate for new gems) relies on running 'git ls-files', which of
course requires a Git commit database, in order to build at all.  This
won't do because the '.git' directory is non-deterministic when running
'git clone', as many of us know.  The entire stack, from the build
system to the package management system are broken and are effectively
beyond repair because no one else believes that there are problems.
This effort has drained too much of my enthusiasm, and now I need a
break.

Sorry if this comes across too ranty and complainy, I suppose it is
both.  I hope your hacking is happier than mine.

-- 
David Thompson
GPG Key: 0FF1D807

[0] http://guides.rubygems.org/what-is-a-gem/
[1] https://github.com/rails/arel/issues/384
[2] https://github.com/rubygems/rubygems/issues/1364