From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pjotr Prins Subject: Re: Giving up on RubyGems Date: Wed, 21 Oct 2015 11:46:06 +0200 Message-ID: <20151021094605.GA20582@thebird.nl> References: <87eggpya7p.fsf@izanagi.i-did-not-set--mail-host-address--so-tickle-me> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45364) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zoq0q-00029A-2s for guix-devel@gnu.org; Wed, 21 Oct 2015 05:48:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zoq0m-0000tu-IQ for guix-devel@gnu.org; Wed, 21 Oct 2015 05:48:52 -0400 Received: from mail.thebird.nl ([95.154.246.10]:49755) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zoq0m-0000tQ-9w for guix-devel@gnu.org; Wed, 21 Oct 2015 05:48:48 -0400 Content-Disposition: inline In-Reply-To: <87eggpya7p.fsf@izanagi.i-did-not-set--mail-host-address--so-tickle-me> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: David Thompson Cc: guix-devel@gnu.org Hi Dave, You know I am not so much interested in fixing upstream concepts which appear to be mixed up (indeed). I think you are heroic for trying to discuss this with the Rubygem system authors. Kudos for trying. Still: GNU Guix Rubygem support is a major achievement. Fact is that: 1. We have successfully packaged rubygems for guix in a simple and elegant way. I use them daily. 2. Enough tests are in there to make sure things work - in fact I was mighty surprised that Nokogiri works on ARM+MIPS, despite the fact that we haven't gotten the Nokogiri test system to work - and that it works was tested by virtue of my bio-blastxmlparser gem which *has* working tests. 3. Rubygems does not dictate how people package their gems. In fact,=20 if need be, we can also repackage into rubygems and deploy those. We may even influence the upstream authors.=20 I agree the Rubygem situation is not ideal and the maybe Rubygems people are misguided in their architectural choices. But I think they will probably converge towards our ideas in time. When a choice, such as no tests, or using git fuzzily, starts to bite, they will want to revert on that. I also love your argumentation that they should provide the source code next to the 'binary' distribution. Totally valid and in line with most licenses in use, including the Ruby license. Maybe the FSF should threaten a case. With Linux distributions and languages you can just see people reinventing the wheel (npm, pip, docker, firefox anyone?). It is amazing how much energy goes into all the combined packaging systems when they could simply be using Guix ;) Still, it is a form of evolution. Each his own (imperfect) niche. For us, I think we have to be purely pragmatic. I love the current Ruby build system in Guix and will use it for those gems that allow inclusion. It has to be source (anyway) to build against the Guix ruby-build-system. And when it fails we find some other way. It is simply software, i.e., we can always fix it. So, yes, I think it is wise to give up on the rubygem authors for now. But please don't stop loving - the people who use - Ruby :) Pj. On Tue, Oct 20, 2015 at 08:51:22AM -0400, David Thompson wrote: > Hello Guix hackers, >=20 > As some of you know, I've been working on Ruby support for Guix for > about a year now. In that time, I helped write and rewrite a Ruby gem > build system, wrote an importer for , and package= d > many Ruby gems. >=20 > At various points, I've had my doubts about the gem archives hosted on > the RubyGems website: Are they source code? Are they binaries? After = a > good deal of debate, we came to the conclusion that they are source > code. This seems to be the case when you inspect any given gem. The > Ruby source code is there, and so is the C source code needed for nativ= e > extensions when there is a native extension to be built. Furthermore, > the RubyGems website says that, among other things, gems should contain > "code (including tests and supporting utilities)." [0] >=20 > However, it has become clear that the RubyGems maintainers do not > actually feel this way. From their perspective, gems are binaries, not > source code. I discovered this once I noticed that several popular Rub= y > gems such as Arel do not, and refuse to [1], ship the test suite in > their releases. This is because they view gems as binaries that need t= o > be as slim as possible, containing only necessary runtime files, to cut > down on bandwidth usage and storage space. Unfortunately, they have no > notion of a source package that corresponds to a given binary. >=20 > In practice, I've found that all the gems I've packaged come with sourc= e > code and no binaries, they might just be missing the test suite. So, I > asked the RubyGems maintainers to consider the use-cases for including > test suites, which spawned a large thread on their GitHub page > yesterday. [2] The end result is this depressing quote: >=20 > Yes, gems are effectively binary packages delivered to > end-users. Some gems contain ruby source code, some contain > pre-compiled binaries, some contain both. The internals of a > particular gem aren't relevant from the perspective of RubyGems > itself. > =20 > As has been pointed out here, RubyGems does not provide packages > containing gem source code. To be honest, RubyGems as a system does > not care about gem source code=E2=80=94it accepts .gem files from g= em > authors, and distributes those files on request. Any gem author who > wishes to provide a link to the source code used to produce a gem i= s > welcome to use the gemspec metadata fields to do so. >=20 > I've grown very tired of trying to convince people that independent use= r > verification of binary releases is an important thing to prioritize, bu= t > they think that users do not want the source code. I've tried to make > my arguments as clear as I could, yet they've been misunderstood by som= e > and rejected entirely by others, and now it is time to give up. I don'= t > know what the best way forward for Ruby support in Guix is. Things lik= e > the RubyGems importer seem useless now. Just downloading release > tarballs from GitHub doesn't work without major hacks because almost > every gem (thanks to a terrible script in Bundler that generates > boilerplate for new gems) relies on running 'git ls-files', which of > course requires a Git commit database, in order to build at all. This > won't do because the '.git' directory is non-deterministic when running > 'git clone', as many of us know. The entire stack, from the build > system to the package management system are broken and are effectively > beyond repair because no one else believes that there are problems. > This effort has drained too much of my enthusiasm, and now I need a > break. >=20 > Sorry if this comes across too ranty and complainy, I suppose it is > both. I hope your hacking is happier than mine. >=20 > --=20 > David Thompson > GPG Key: 0FF1D807 >=20 > [0] http://guides.rubygems.org/what-is-a-gem/ > [1] https://github.com/rails/arel/issues/384 > [2] https://github.com/rubygems/rubygems/issues/1364 >=20 >=20 --=20