From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Thompson Subject: Giving up on RubyGems Date: Tue, 20 Oct 2015 08:51:22 -0400 Message-ID: <87eggpya7p.fsf@izanagi.i-did-not-set--mail-host-address--so-tickle-me> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:43006) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZoWOD-0007G2-TL for guix-devel@gnu.org; Tue, 20 Oct 2015 08:51:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZoWO8-00064M-SG for guix-devel@gnu.org; Tue, 20 Oct 2015 08:51:41 -0400 Received: from mail-qg0-f41.google.com ([209.85.192.41]:35436) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZoWO8-00064H-Oc for guix-devel@gnu.org; Tue, 20 Oct 2015 08:51:36 -0400 Received: by qgbb65 with SMTP id b65so13393917qgb.2 for ; Tue, 20 Oct 2015 05:51:36 -0700 (PDT) Received: from izanagi ([38.88.209.18]) by smtp.gmail.com with ESMTPSA id p90sm1177626qgd.10.2015.10.20.05.51.35 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Oct 2015 05:51:35 -0700 (PDT) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: guix-devel@gnu.org Hello Guix hackers, As some of you know, I've been working on Ruby support for Guix for about a year now. In that time, I helped write and rewrite a Ruby gem build system, wrote an importer for , and packaged many Ruby gems. At various points, I've had my doubts about the gem archives hosted on the RubyGems website: Are they source code? Are they binaries? After a good deal of debate, we came to the conclusion that they are source code. This seems to be the case when you inspect any given gem. The Ruby source code is there, and so is the C source code needed for native extensions when there is a native extension to be built. Furthermore, the RubyGems website says that, among other things, gems should contain "code (including tests and supporting utilities)." [0] However, it has become clear that the RubyGems maintainers do not actually feel this way. From their perspective, gems are binaries, not source code. I discovered this once I noticed that several popular Ruby gems such as Arel do not, and refuse to [1], ship the test suite in their releases. This is because they view gems as binaries that need to be as slim as possible, containing only necessary runtime files, to cut down on bandwidth usage and storage space. Unfortunately, they have no notion of a source package that corresponds to a given binary. In practice, I've found that all the gems I've packaged come with source code and no binaries, they might just be missing the test suite. So, I asked the RubyGems maintainers to consider the use-cases for including test suites, which spawned a large thread on their GitHub page yesterday. [2] The end result is this depressing quote: Yes, gems are effectively binary packages delivered to end-users. Some gems contain ruby source code, some contain pre-compiled binaries, some contain both. The internals of a particular gem aren't relevant from the perspective of RubyGems itself. =20=20=20=20 As has been pointed out here, RubyGems does not provide packages containing gem source code. To be honest, RubyGems as a system does not care about gem source code=E2=80=94it accepts .gem files from gem authors, and distributes those files on request. Any gem author who wishes to provide a link to the source code used to produce a gem is welcome to use the gemspec metadata fields to do so. I've grown very tired of trying to convince people that independent user verification of binary releases is an important thing to prioritize, but they think that users do not want the source code. I've tried to make my arguments as clear as I could, yet they've been misunderstood by some and rejected entirely by others, and now it is time to give up. I don't know what the best way forward for Ruby support in Guix is. Things like the RubyGems importer seem useless now. Just downloading release tarballs from GitHub doesn't work without major hacks because almost every gem (thanks to a terrible script in Bundler that generates boilerplate for new gems) relies on running 'git ls-files', which of course requires a Git commit database, in order to build at all. This won't do because the '.git' directory is non-deterministic when running 'git clone', as many of us know. The entire stack, from the build system to the package management system are broken and are effectively beyond repair because no one else believes that there are problems. This effort has drained too much of my enthusiasm, and now I need a break. Sorry if this comes across too ranty and complainy, I suppose it is both. I hope your hacking is happier than mine. --=20 David Thompson GPG Key: 0FF1D807 [0] http://guides.rubygems.org/what-is-a-gem/ [1] https://github.com/rails/arel/issues/384 [2] https://github.com/rubygems/rubygems/issues/1364