From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pjotr Prins Subject: Re: Input needed: Plan for packaging scala Date: Sun, 26 Feb 2017 07:01:43 +0000 Message-ID: <20170226070143.GA15812@mail.thebird.nl> References: <878tovwt43.fsf@gmail.com> <20170224194522.30b2ac58@scratchpost.org> <878touupj6.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:52305) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1chstZ-0003a0-7z for guix-devel@gnu.org; Sun, 26 Feb 2017 02:05:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1chstV-0001KZ-9N for guix-devel@gnu.org; Sun, 26 Feb 2017 02:05:25 -0500 Received: from mail.thebird.nl ([95.154.246.10]:38243) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1chstV-0001Ay-0O for guix-devel@gnu.org; Sun, 26 Feb 2017 02:05:21 -0500 Content-Disposition: inline In-Reply-To: <878touupj6.fsf@gmail.com> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Katherine Cox-Buday Cc: guix-devel@gnu.org On Sat, Feb 25, 2017 at 11:51:57AM -0600, Katherine Cox-Buday wrote: > Looks like not quite! After speaking to some friendly scala community > members, it looks[1] like the bootstrapping process is much more > harrowing. The last version of scala which only used Java to compile was > pre 2.0 (for reference the 2.0 commit appears to be here[2]). So we'd be > looking at a chain of scala packages numbering possibly in the hundreds > and going back 10+ years. Maybe we can skip quite a few - it is expected that a Scala compiler will compile with much older versions of Scala. But it is forensic work and it is sad that the project itself does not care. I know, for example, that the D community is aware and wants to make sure there is a sane bootstrap path to support different backends. The good news that if the compiler compiles and tests pass you should be good - one advantage of strong typing. So you could automate testing compilers and going forward in time from the earliest versions. It would be an interesting exercise. > >> The build for scala 2.9.2 attempts to pull some libraries from a > >> remote > >> source, so I'm assuming I'll need to also package those? > > > > Yes please. > > > >> I do, however, want to confirm that this is a sane plan > >> before I set out on this journey. > > > > Sounds good. > > > >> It would be easier if I could just do > >> a binary package to bootstrap the toolchain and back-fill when I > >> have > >> more experience, but I wasn't sure if that is acceptable. > > > > It's acceptable. You should make sure to eventually do a reproducable > > build - but it's OK to start with a binary package as a builder for > > the time being. Yes. And if Guix does not accept it on master you can provide an alternative package repository and publishing host. This is possible today and with the planned 'guix channnels' feature it will get even easier for users to use that. https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22629 > For the aforementioned reasons, I'm going to start by just packaging the > source of sbt v0.13, but also the pre-built binary. This should help us > at least bootstrap scala in a sane way, if not immediately > sbt. Hopefully this still sounds OK? > > > The question is whether it's actually easier that way. Guix uses > > non-standard locations (for good reasons) - so a random executable you > > download won't run because it won't find any libraries. The situation > > is not that bad if it's in Java, though - you can always specify the > > classpath when invoking the VM. sbt is much like Java ant and Ruby bundler that it pulls in dependencies as binary blobs. Circumventing that behaviour is possible by providing local jar files - so sbt can find them - but they have to be in place. I.e., they have been built earlier. To be honest, I never liked sbt because there was too much 'magic' going on. Only good thing about sbt is that the definitions are not XML but plain Scala ;). What should really be a simple Makefile has become a system to deal with development dependencies. A job much better left to a proper deployment system such as Guix. That is why I wrote about Ruby being in that boat and getting rid of rvm, rbenv and bundler: https://github.com/pjotrp/guix-notes/blob/master/RUBY.org#getting-rid-of-rvm-rbenv-and-bundler Same goes for Python virtualenv etc. Essentially these tools are solving the wrong problem. But then, they were not using Guix! Poor man's dependency resolution. In Guix we do not use Ruby bundler. The ruby-build-system is based on Ruby gem to build gem's locally - that is possible because the gems contain the full ruby source of the gem. For sbt may eventually need something similar. Take a look at ./guix/build-system/ruby.scm ./guix/build/ruby-build-system.scm Don't worry about this too much now. I remember we wrote the ruby-build-system fairly quickly once you decide what solution to pursue. Key is to decide where to get the sources of Scala projects, compile them as packages using the scala-build-system and tell sbt to use these without trying to pull in jars. Ricardo and friends have achieved this for other JVM tools. Pj.