Re: Giving up on RubyGems

[Pjotr Prins <pjotr.public12@thebird.nl>]

Hi Dave,

You know I am not so much interested in fixing upstream concepts which
appear to be mixed up (indeed). I think you are heroic for trying to
discuss this with the Rubygem system authors. Kudos for trying.

Still: GNU Guix Rubygem support is a major achievement.

Fact is that:

1. We have successfully packaged rubygems for guix in a simple and
   elegant way. I use them daily.

2. Enough tests are in there to make sure things work - in fact I was
   mighty surprised that Nokogiri works on ARM+MIPS, despite the fact
   that we haven't gotten the Nokogiri test system to work - and that
   it works was tested by virtue of my bio-blastxmlparser gem which
   *has* working tests.

3. Rubygems does not dictate how people package their gems. In fact, 
   if need be, we can also repackage into rubygems and deploy those.
   We may even influence the upstream authors. 

I agree the Rubygem situation is not ideal and the maybe Rubygems
people are misguided in their architectural choices. But I think they
will probably converge towards our ideas in time. When a choice, such
as no tests, or using git fuzzily, starts to bite, they will want to
revert on that. I also love your argumentation that they should
provide the source code next to the 'binary' distribution. Totally
valid and in line with most licenses in use, including the Ruby
license. Maybe the FSF should threaten a case.

With Linux distributions and languages you can just see people
reinventing the wheel (npm, pip, docker, firefox anyone?). It is
amazing how much energy goes into all the combined packaging systems
when they could simply be using Guix ;)

Still, it is a form of evolution. Each his own (imperfect) niche.

For us, I think we have to be purely pragmatic. I love the current
Ruby build system in Guix and will use it for those gems that allow
inclusion. It has to be source (anyway) to build against the Guix
ruby-build-system.

And when it fails we find some other way. It is simply software, i.e.,
we can always fix it.

So, yes, I think it is wise to give up on the rubygem authors for now.

But please don't stop loving - the people who use - Ruby :)

Pj.



On Tue, Oct 20, 2015 at 08:51:22AM -0400, David Thompson wrote:
> Hello Guix hackers,
> 
> As some of you know, I've been working on Ruby support for Guix for
> about a year now.  In that time, I helped write and rewrite a Ruby gem
> build system, wrote an importer for <https://rubygems.org>, and packaged
> many Ruby gems.
> 
> At various points, I've had my doubts about the gem archives hosted on
> the RubyGems website: Are they source code?  Are they binaries?  After a
> good deal of debate, we came to the conclusion that they are source
> code.  This seems to be the case when you inspect any given gem.  The
> Ruby source code is there, and so is the C source code needed for native
> extensions when there is a native extension to be built.  Furthermore,
> the RubyGems website says that, among other things, gems should contain
> "code (including tests and supporting utilities)." [0]
> 
> However, it has become clear that the RubyGems maintainers do not
> actually feel this way.  From their perspective, gems are binaries, not
> source code.  I discovered this once I noticed that several popular Ruby
> gems such as Arel do not, and refuse to [1], ship the test suite in
> their releases.  This is because they view gems as binaries that need to
> be as slim as possible, containing only necessary runtime files, to cut
> down on bandwidth usage and storage space.  Unfortunately, they have no
> notion of a source package that corresponds to a given binary.
> 
> In practice, I've found that all the gems I've packaged come with source
> code and no binaries, they might just be missing the test suite.  So, I
> asked the RubyGems maintainers to consider the use-cases for including
> test suites, which spawned a large thread on their GitHub page
> yesterday. [2]  The end result is this depressing quote:
> 
>     Yes, gems are effectively binary packages delivered to
>     end-users. Some gems contain ruby source code, some contain
>     pre-compiled binaries, some contain both. The internals of a
>     particular gem aren't relevant from the perspective of RubyGems
>     itself.
>     
>     As has been pointed out here, RubyGems does not provide packages
>     containing gem source code. To be honest, RubyGems as a system does
>     not care about gem source code—it accepts .gem files from gem
>     authors, and distributes those files on request. Any gem author who
>     wishes to provide a link to the source code used to produce a gem is
>     welcome to use the gemspec metadata fields to do so.
> 
> I've grown very tired of trying to convince people that independent user
> verification of binary releases is an important thing to prioritize, but
> they think that users do not want the source code.  I've tried to make
> my arguments as clear as I could, yet they've been misunderstood by some
> and rejected entirely by others, and now it is time to give up.  I don't
> know what the best way forward for Ruby support in Guix is.  Things like
> the RubyGems importer seem useless now.  Just downloading release
> tarballs from GitHub doesn't work without major hacks because almost
> every gem (thanks to a terrible script in Bundler that generates
> boilerplate for new gems) relies on running 'git ls-files', which of
> course requires a Git commit database, in order to build at all.  This
> won't do because the '.git' directory is non-deterministic when running
> 'git clone', as many of us know.  The entire stack, from the build
> system to the package management system are broken and are effectively
> beyond repair because no one else believes that there are problems.
> This effort has drained too much of my enthusiasm, and now I need a
> break.
> 
> Sorry if this comes across too ranty and complainy, I suppose it is
> both.  I hope your hacking is happier than mine.
> 
> -- 
> David Thompson
> GPG Key: 0FF1D807
> 
> [0] http://guides.rubygems.org/what-is-a-gem/
> [1] https://github.com/rails/arel/issues/384
> [2] https://github.com/rubygems/rubygems/issues/1364
> 
> 

--