Re: Input needed: Plan for packaging scala

[Pjotr Prins <pjotr.public12@thebird.nl>]

On Sat, Feb 25, 2017 at 11:51:57AM -0600, Katherine Cox-Buday wrote:
> Looks like not quite! After speaking to some friendly scala community
> members, it looks[1] like the bootstrapping process is much more
> harrowing. The last version of scala which only used Java to compile was
> pre 2.0 (for reference the 2.0 commit appears to be here[2]). So we'd be
> looking at a chain of scala packages numbering possibly in the hundreds
> and going back 10+ years.

Maybe we can skip quite a few - it is expected that a Scala compiler
will compile with much older versions of Scala. But it is forensic
work and it is sad that the project itself does not care. I know, for
example, that the D community is aware and wants to make sure there is
a sane bootstrap path to support different backends.

The good news that if the compiler compiles and tests pass you should
be good - one advantage of strong typing. So you could automate
testing compilers and going forward in time from the earliest
versions. It would be an interesting exercise.

> >> The build for scala 2.9.2 attempts to pull some libraries from a
> >> remote
> >> source, so I'm assuming I'll need to also package those?
> >
> > Yes please.
> >
> >> I do, however, want to confirm that this is a sane plan
> >> before I set out on this journey. 
> >
> > Sounds good.
> >
> >> It would be easier if I could just do
> >> a binary package to bootstrap the toolchain and back-fill when I
> >> have
> >> more experience, but I wasn't sure if that is acceptable.
> >
> > It's acceptable. You should make sure to eventually do a reproducable
> > build - but it's OK to start with a binary package as a builder for
> > the time being.

Yes. And if Guix does not accept it on master you can provide an
alternative package repository and publishing host. This is possible
today and with the planned 'guix channnels' feature it will get even
easier for users to use that.

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22629

> For the aforementioned reasons, I'm going to start by just packaging the
> source of sbt v0.13, but also the pre-built binary. This should help us
> at least bootstrap scala in a sane way, if not immediately
> sbt. Hopefully this still sounds OK?
> 
> > The question is whether it's actually easier that way. Guix uses
> > non-standard locations (for good reasons) - so a random executable you
> > download won't run because it won't find any libraries. The situation
> > is not that bad if it's in Java, though - you can always specify the
> > classpath when invoking the VM.

sbt is much like Java ant and Ruby bundler that it pulls in
dependencies as binary blobs. Circumventing that behaviour is possible
by providing local jar files - so sbt can find them - but they have to
be in place. I.e., they have been built earlier. To be honest, I never
liked sbt because there was too much 'magic' going on. Only good thing
about sbt is that the definitions are not XML but plain Scala ;). What
should really be a simple Makefile has become a system to deal with
development dependencies. A job much better left to a proper
deployment system such as Guix. That is why I wrote about Ruby being
in that boat and getting rid of rvm, rbenv and bundler:

  https://github.com/pjotrp/guix-notes/blob/master/RUBY.org#getting-rid-of-rvm-rbenv-and-bundler

Same goes for Python virtualenv etc. Essentially these tools are
solving the wrong problem. But then, they were not using Guix! Poor
man's dependency resolution.

In Guix we do not use Ruby bundler. The ruby-build-system is based on
Ruby gem to build gem's locally - that is possible because the gems
contain the full ruby source of the gem. For sbt may eventually need
something similar. Take a look at

  ./guix/build-system/ruby.scm 
  ./guix/build/ruby-build-system.scm 

Don't worry about this too much now. I remember we wrote the
ruby-build-system fairly quickly once you decide what solution to
pursue. Key is to decide where to get the sources of Scala projects,
compile them as packages using the scala-build-system and tell sbt to
use these without trying to pull in jars.

Ricardo and friends have achieved this for other JVM tools.

Pj.