More stability needed in our Rust packages, for IceCat 60

More stability needed in our Rust packages, for IceCat 60

[Mark H Weaver]

Hi Danny,

dannym@scratchpost.org (Danny Milosavljevic) writes:

> dannym pushed a commit to branch master
> in repository guix.
>
> commit ec944a1de97c8a523d6e1089b8c4430424297f93
> Author: Danny Milosavljevic <dannym@scratchpost.org>
> Date:   Fri Sep 21 15:42:56 2018 +0200
>
>     gnu: rust: Remove unused "ar" setting.
>     
>     * gnu/packages/rust.scm (rust-1.20)[arguments]<#:phases>[configure]: Remove
>     unused "ar" setting.

Thanks to your hard work on our Rust packages, I was able to update our
IceCat package to a pre-release of 60.2.0.  I recently pushed it to
master and Hydra is currently busy bootstrapping the chain of Rust
compilers in order to eventually build IceCat.

Unfortunately, I see that you've pushed some more small patches that
have already rendered those builds obsolete.  The words below are not
complaints -- far from it -- but are merely meant to illustrate a
practical problem we now face.

It took my fastest laptop about 36 hours to build the chain of 5 rust
compilers required to get to the latest Rust release, and to build
IceCat.  By the time I had gotten IceCat built and tested, you had
pushed a few more improvements to our Rust packages, rendering my
testing obsolete.  I decided to push it anyway, since the IceCat update
is a critical security update.

Since then, I started my fastest laptop building the new Rust compilers,
and also generated an evaluation on Hydra to build the new IceCat.  Both
of these build efforts are less than half through the Rust bootstrap,
and now I see that I'll need to cancel these jobs and start again.

Of course, I'm extremely grateful for all of the work you are doing on
this, but it has also become apparent to me that if our IceCat package
is to be effectively usable, we'll need to drastically reduce the
frequency with which our Rust compilers are modified on the master
branch.

How would you feel about doing further Rust work on a separate branch,
and periodically (maybe 2-4 times per month) merging the collected
improvements into master?

       Mark

Re: More stability needed in our Rust packages, for IceCat 60

[Pjotr Prins]

On Fri, Sep 21, 2018 at 11:35:17PM -0400, Mark H Weaver wrote:
> How would you feel about doing further Rust work on a separate branch,
> and periodically (maybe 2-4 times per month) merging the collected
> improvements into master?

Or have a stable package which is a good idea anyway, especially with
boostrapping compilers. 

Pj.

Re: More stability needed in our Rust packages, for IceCat 60

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 2069 bytes --]

Hi Mark,

On Fri, 21 Sep 2018 23:35:17 -0400
Mark H Weaver <mhw@netris.org> wrote:

> It took my fastest laptop about 36 hours to build the chain of 5 rust
> compilers required to get to the latest Rust release, and to build
> IceCat.  By the time I had gotten IceCat built and tested, you had
> pushed a few more improvements to our Rust packages, rendering my
> testing obsolete.  I decided to push it anyway, since the IceCat update
> is a critical security update.

Yes, I agree that it was correct to do that.

> Since then, I started my fastest laptop building the new Rust compilers,
> and also generated an evaluation on Hydra to build the new IceCat.  Both
> of these build efforts are less than half through the Rust bootstrap,
> and now I see that I'll need to cancel these jobs and start again.
> 
> Of course, I'm extremely grateful for all of the work you are doing on
> this, but it has also become apparent to me that if our IceCat package
> is to be effectively usable, we'll need to drastically reduce the
> frequency with which our Rust compilers are modified on the master
> branch.
> 
> How would you feel about doing further Rust work on a separate branch,
> and periodically (maybe 2-4 times per month) merging the collected
> improvements into master?

I agree actually.  Rust is taking up way too much of my own build
capacity right now.  Even small changes have to be tested on at least
a few of the compilers, because Rust seems to change what settings one
can use regularily - and any of those could break master (and did before).

I've reverted the last commit mentioning "ar".  It's not strictly the
right fix to revert it, but it's the most stable version to have on
master.

I've also been talking to a new volunteer, so now is actually a good
time for oponeing a new branch.

I've now opened wip-rust and will be pushing it after some rust versions
built.

If anyone wants a simple way to contribute to getting Rust bootstrapped,
please provide some cpu time to help with compiling Rust :)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: More stability needed in our Rust packages, for IceCat 60

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 1136 bytes --]

Hi Pjotr,

On Sat, 22 Sep 2018 09:11:42 +0200
Pjotr Prins <pjotr.public12@thebird.nl> wrote:

> On Fri, Sep 21, 2018 at 11:35:17PM -0400, Mark H Weaver wrote:
> > How would you feel about doing further Rust work on a separate branch,
> > and periodically (maybe 2-4 times per month) merging the collected
> > improvements into master?  
> 
> Or have a stable package which is a good idea anyway, especially with
> boostrapping compilers. 

Hmm... what do you mean by this?

The entire bootstrap chain build is not succeeding yet - so the packages
are not stable and cannot be stable yet.

Bootstrapping rust is not done yet.   It might look like it is, but if you
look closely you'll see that the rust-1.23 package cheats by using a binary
blob compiler.

I'm still trying to find the magical combination of settings that will work
for all Rust versions - otherwise, maintenance overhead would explode.

Right now, mrustc and rust 1.19.0 are reasonably finished and will probably
not change again in the next month.

All the other rusts, good luck.

In any case, further work will continue in wip-rust :)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: More stability needed in our Rust packages, for IceCat 60

[Joshua Branson]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> Hi Mark,
>
> On Fri, 21 Sep 2018 23:35:17 -0400
> Mark H Weaver <mhw@netris.org> wrote:
>
>
> If anyone wants a simple way to contribute to getting Rust bootstrapped,
> please provide some cpu time to help with compiling Rust :)

I've got a 4 year old AMD APU 4 core sitting at my parents' house
collecting dust.  If I ever figure out how to get it to connect to the
outside world, I'd be glad to donate some of it's CPU time to compiling.

Re: More stability needed in our Rust packages, for IceCat 60

[Mark H Weaver]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> writes:

>> How would you feel about doing further Rust work on a separate branch,
>> and periodically (maybe 2-4 times per month) merging the collected
>> improvements into master?
>
> I agree actually.  Rust is taking up way too much of my own build
> capacity right now.  Even small changes have to be tested on at least
> a few of the compilers, because Rust seems to change what settings one
> can use regularily - and any of those could break master (and did before).
>
> I've reverted the last commit mentioning "ar".  It's not strictly the
> right fix to revert it, but it's the most stable version to have on
> master.
>
> I've also been talking to a new volunteer, so now is actually a good
> time for oponeing a new branch.
>
> I've now opened wip-rust and will be pushing it after some rust versions
> built.

Sounds good.  Thanks again!

     Mark

Re: More stability needed in our Rust packages, for IceCat 60

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 514 bytes --]

On Fri, Sep 21, 2018 at 23:35:17 -0400, Mark H Weaver wrote:
> Thanks to your hard work on our Rust packages, I was able to update our
> IceCat package to a pre-release of 60.2.0.  I recently pushed it to
> master and Hydra is currently busy bootstrapping the chain of Rust
> compilers in order to eventually build IceCat.

This is wonderful news!  I'm really excited to give this a try once
Hydra is ready.  Thank you!

And thank you too, Danny, since your work made this possible.

-- 
Mike Gerwitz

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: More stability needed in our Rust packages, for IceCat 60

[Ludovic Courtès]

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Fri, 21 Sep 2018 23:35:17 -0400
> Mark H Weaver <mhw@netris.org> wrote:

[...]

>> How would you feel about doing further Rust work on a separate branch,
>> and periodically (maybe 2-4 times per month) merging the collected
>> improvements into master?
>
> I agree actually.

[...]

> I've now opened wip-rust and will be pushing it after some rust versions
> built.

Awesome.  Thanks for the two of you for your tireless work on IceCat and
Rust.  I’m pleasantly surprised to see progress being made on Rust, and
was really happy to see IceCat 60 in ‘master’!

Ludo’.

Re: More stability needed in our Rust packages, for IceCat 60

[Clément Lassieur]

Joshua Branson <jbranso@fastmail.com> writes:

> Danny Milosavljevic <dannym@scratchpost.org> writes:
>
>> Hi Mark,
>>
>> On Fri, 21 Sep 2018 23:35:17 -0400
>> Mark H Weaver <mhw@netris.org> wrote:
>>
>>
>> If anyone wants a simple way to contribute to getting Rust bootstrapped,
>> please provide some cpu time to help with compiling Rust :)
>
> I've got a 4 year old AMD APU 4 core sitting at my parents' house
> collecting dust.  If I ever figure out how to get it to connect to the
> outside world, I'd be glad to donate some of it's CPU time to compiling.

They are very, very slow :-)  I don't think it would help, to be honest.
(Disclaimer: I have two APUs and I use them every day.)

Re: More stability needed in our Rust packages, for IceCat 60

[Clément Lassieur]

Mark H Weaver <mhw@netris.org> writes:

> Thanks to your hard work on our Rust packages, I was able to update our
> IceCat package to a pre-release of 60.2.0.  I recently pushed it to
> master and Hydra is currently busy bootstrapping the chain of Rust
> compilers in order to eventually build IceCat.

This is awesome, really.  Thank you so much, Mark!

Re: More stability needed in our Rust packages, for IceCat 60

[Joshua Branson]

Clément Lassieur <clement@lassieur.org> writes:

> Joshua Branson <jbranso@fastmail.com> writes:
>
>> Danny Milosavljevic <dannym@scratchpost.org> writes:
>>
>>> Hi Mark,
>>>
>>> On Fri, 21 Sep 2018 23:35:17 -0400
>>> Mark H Weaver <mhw@netris.org> wrote:
>>>
>>>
>>> If anyone wants a simple way to contribute to getting Rust bootstrapped,
>>> please provide some cpu time to help with compiling Rust :)
>>
>> I've got a 4 year old AMD APU 4 core sitting at my parents' house
>> collecting dust.  If I ever figure out how to get it to connect to the
>> outside world, I'd be glad to donate some of it's CPU time to compiling.
>
> They are very, very slow :-)  I don't think it would help, to be honest.
> (Disclaimer: I have two APUs and I use them every day.)

Fair enough.