core-updates frozen!

core-updates frozen!

[Leo Famulari]

On Tue, Feb 14, 2017 at 10:05:04AM +0100, Ludovic Courtès wrote:
> So, here’s a plan:
> 
>   • Once Efraim has pushed some of the aarch64 patches, do another
>     evaluation of the “core” package set for that branch, and check for
>     anything wrong.  From there on, forbid full-rebuild changes.

Let's freeze core-updates and try to build it. No more rebuild the world
changes except to fix breakage.

>   • Once the “core” subset builds correctly on all the supported
>     platforms (those that Hydra supports), merge ‘master’.  Maybe update
>     a couple of things like GnuTLS while we’re at it.  From there on
>     forbid non-trivial changes.
> 
>   • Build all the packages.  (To do that, someone with access to Hydra
>     must change the “subset” argument to “all” in the config of the
>     ‘core-updates’ jobset.)
> 
>   • Fix things.
> 
>   • Once most regressions have been addressed and most binaries are
>     available, merge ‘core-updates’ into ‘master’.
> 
> How does that sound?
> 
> Thanks,
> Ludo’.
> 

Re: core-updates frozen!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

On Mon, Feb 27, 2017 at 03:30:59PM -0500, Leo Famulari wrote:
> On Tue, Feb 14, 2017 at 10:05:04AM +0100, Ludovic Courtès wrote:
> > So, here’s a plan:
> > 
> >   • Once Efraim has pushed some of the aarch64 patches, do another
> >     evaluation of the “core” package set for that branch, and check for
> >     anything wrong.  From there on, forbid full-rebuild changes.
> 
> Let's freeze core-updates and try to build it. No more rebuild the world
> changes except to fix breakage.

Once these changes are pushed, I'll start a new evaluation:

http://lists.gnu.org/archive/html/guix-devel/2017-03/msg00063.html

Please, no more rebuild the world changes except to fix breakage in the
core packages.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: core-updates frozen!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1158 bytes --]

Leo Famulari <leo@famulari.name> writes:

> On Mon, Feb 27, 2017 at 03:30:59PM -0500, Leo Famulari wrote:
>> On Tue, Feb 14, 2017 at 10:05:04AM +0100, Ludovic Courtès wrote:
>> > So, here’s a plan:
>> > 
>> >   • Once Efraim has pushed some of the aarch64 patches, do another
>> >     evaluation of the “core” package set for that branch, and check for
>> >     anything wrong.  From there on, forbid full-rebuild changes.
>> 
>> Let's freeze core-updates and try to build it. No more rebuild the world
>> changes except to fix breakage.
>
> Once these changes are pushed, I'll start a new evaluation:
>
> http://lists.gnu.org/archive/html/guix-devel/2017-03/msg00063.html
>
> Please, no more rebuild the world changes except to fix breakage in the
> core packages.

xorg-server@1.19.2 was *just* released[0] and contains some important
bug fixes (notably CVE-2017-2624).

[0] https://lists.x.org/archives/xorg-announce/2017-March/002779.html

AFAICT it has not been built yet on Hydra since it's not part of the
"core" package set. Is it okay to push?

Unfortunately I have not been able to backport the fix to 1.18.4.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates frozen!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1265 bytes --]

On Fri, Mar 03, 2017 at 01:02:04AM +0100, Marius Bakke wrote:
> Leo Famulari <leo@famulari.name> writes:
> > Once these changes are pushed, I'll start a new evaluation:
> >
> > http://lists.gnu.org/archive/html/guix-devel/2017-03/msg00063.html
> >
> > Please, no more rebuild the world changes except to fix breakage in the
> > core packages.
> 
> xorg-server@1.19.2 was *just* released[0] and contains some important
> bug fixes (notably CVE-2017-2624).
> 
> [0] https://lists.x.org/archives/xorg-announce/2017-March/002779.html

It looks like a relatively small set of changes.

> AFAICT it has not been built yet on Hydra since it's not part of the
> "core" package set. Is it okay to push?

xorg-server is not a core package, so I guess it's fine. I'll do the
update when pushing the libgd changes from the thread linked above.

I've been reconfiguring my GuixSD system on core-updates, so I've fixed
and then suffered breakage in a few non-core packages so far :)

At some arbitrary point we have to stop changing the branch and just
build it. Newer important updates will have to be grafted.

> Unfortunately I have not been able to backport the fix to 1.18.4.

Okay, hopefully we can figure it out or copy from another distro.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: core-updates frozen!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1643 bytes --]

Leo Famulari <leo@famulari.name> writes:

> On Fri, Mar 03, 2017 at 01:02:04AM +0100, Marius Bakke wrote:
>> Leo Famulari <leo@famulari.name> writes:
>> > Once these changes are pushed, I'll start a new evaluation:
>> >
>> > http://lists.gnu.org/archive/html/guix-devel/2017-03/msg00063.html
>> >
>> > Please, no more rebuild the world changes except to fix breakage in the
>> > core packages.
>> 
>> xorg-server@1.19.2 was *just* released[0] and contains some important
>> bug fixes (notably CVE-2017-2624).
>> 
>> [0] https://lists.x.org/archives/xorg-announce/2017-March/002779.html
>
> It looks like a relatively small set of changes.
>
>> AFAICT it has not been built yet on Hydra since it's not part of the
>> "core" package set. Is it okay to push?
>
> xorg-server is not a core package, so I guess it's fine. I'll do the
> update when pushing the libgd changes from the thread linked above.

Note that 1.19.3 will be released shortly since 1.19.2 had a release bug
that requires running `autoreconf`. See:

https://lists.x.org/archives/xorg-announce/2017-March/002780.html

> I've been reconfiguring my GuixSD system on core-updates, so I've fixed
> and then suffered breakage in a few non-core packages so far :)

Wow, nice :)

> At some arbitrary point we have to stop changing the branch and just
> build it. Newer important updates will have to be grafted.

Agreed. Let's do a full evaluation once Hydra settles down and then
*really* freeze it ;)

>> Unfortunately I have not been able to backport the fix to 1.18.4.
>
> Okay, hopefully we can figure it out or copy from another distro.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates frozen!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]

On Fri, Mar 03, 2017 at 07:33:52PM +0100, Marius Bakke wrote:
> Note that 1.19.3 will be released shortly since 1.19.2 had a release bug
> that requires running `autoreconf`. See:
> 
> https://lists.x.org/archives/xorg-announce/2017-March/002780.html

Ooookay :)

> > I've been reconfiguring my GuixSD system on core-updates, so I've fixed
> > and then suffered breakage in a few non-core packages so far :)
> 
> Wow, nice :)

Note that I haven't succeeded yet. But I did reach the build of GRUB,
which failed to build.

> > At some arbitrary point we have to stop changing the branch and just
> > build it. Newer important updates will have to be grafted.
> 
> Agreed. Let's do a full evaluation once Hydra settles down and then
> *really* freeze it ;)

Okay, but Hydra may never settle down!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

core-updates frozen!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 3293 bytes --]

Guix,

The 'core-updates' branch is ready for testing!  This is a very early
stage, so many substitutes are missing.  Consider yourself warned ;-)

I have compiled a summary of the changes below.  The package updates are
too numerous to list here; try `git shortlog -n master..core-updates`
for the scoop.

We are still waiting for armhf to catch up on the CI, as well as
<https://issues.guix.info/issue/36535> before starting the full
rebuild.  I will send another email when that happens, along with a
status update.

* Build system changes
** 'python-build-system' runs tests after installing the package,
   instead of in between the 'build' and 'install' phases, to improve
   reproducibility.
** 'python-build-system' does not wrap already-wrapped executables.
** 'python-build-system' has a (python-version ...) procedure that
   returns the major+minor version of a given python package.
** 'meson-build-system' produces optimized binaries with debugging
   symbols by default, similar to 'cmake-build-system'.
** 'meson-build-system' no longer attempts to run Autotools bootstrap
   scripts.
** 'gnu-build-system' (and those inheriting from it) has deterministic
   behavior in some corner cases (<https://issues.guix.info/35387).
** gnu-build-system copies license files to all outputs instead of just
   "out".  It now also works for out-of-source builds.
** (guix build utils) has a new 'wrap-script' procedure that replaces
   the shebang in scripts in a language-aware manner, as an alternative
   to the shell wrapper created by 'wrap-program'.
** (invoke ...) from the same module now reports errors in a
   human-friendly way, instead displaying a long stack trace.
** There is also a new (invoke/quiet ...) that swallows program output,
   unless it fails.  The return value is unspecified.

* Toolchain changes
** On i686 and x86_64, the "binary seeds" at the root of the dependency
   graph no longer includes GCC, glibc, and binutils.  Instead they are
   built from source using a new GNU Mes based toolchain, reducing the
   set of trusted bootstrap binaries from ~250MiB to ~130MiB.
** GCC 7 became the default compiler.  Consequently, C_INCLUDE_PATH and
   CPLUS_INCLUDE_PATH are no longer set in the build environment.
   Packages that rely on those variables or their CROSS_ counterparts
   will need to be adjusted to use {CROSS_,}CPATH instead.
** GNU/Hurd no longer uses a special glibc variant.
** Guile was updated to 2.2.6, and libgc to 7.6.12.
** Linux-Libre headers was updated to 4.19.57.
** Glibc was updated to 2.29.
** Binutils was updated to 2.32.
** coreutils was updated to 8.31.
** Bash was updated to 5.0.7.
** Grep 3.3, Gawk 5.0.1, Gettext 0.20.1, diffutils 3.7, Bison 3.4.1, ...

Other noteworthy changes:
** GNOME was updated to 3.30.
** OpenSSL 1.1.1 is the default 'openssl' package.
** Many packages were migrated to use Python 3 instead of Python 2.
** cURL and GNU SASL use MIT Kerberos instead of GNU Security Services.
** Python 2 builds reproducibly.
** Python (2 & 3) no longer uses a bundled copy of Expat.
** CMake was updated 3.14.5.
** CMake comes with documentation, too.
** Perl was updated to 5.30.0.
** Python was updated to 3.7.4.
** Boost was updated to 1.70.0.
** SQLite was updated to 3.28.0.
** Pytest was updated to 4.4.2.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates frozen!

[Ludovic Courtès]

Hello!

Marius Bakke <mbakke@fastmail.com> skribis:

> The 'core-updates' branch is ready for testing!  This is a very early
> stage, so many substitutes are missing.  Consider yourself warned ;-)

Yay!

> ** GNU/Hurd no longer uses a special glibc variant.

That’s the case since the previous ‘core-updates’ merge.  :-)

At any rate, thanks for the news items, it’s great to rediscover what’s
actually been done months ago!

Ludo’.

Re: core-updates frozen!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 595 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Marius Bakke <mbakke@fastmail.com> skribis:
>
>> The 'core-updates' branch is ready for testing!  This is a very early
>> stage, so many substitutes are missing.  Consider yourself warned ;-)
>
> Yay!
>
>> ** GNU/Hurd no longer uses a special glibc variant.
>
> That’s the case since the previous ‘core-updates’ merge.  :-)

Whoops!

There is another important change that I forgot to list as well: glibc
no longer provides Sun/ONC RPC support, so many packages need to migrate
to 'libtirpc' and/or 'rpcsvc-proto'.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates frozen!

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 202 bytes --]

I've sent a few patches to fix a couple of build issues on the
core-updates branch [1][2].

1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36641
2: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36643

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

core-updates frozen!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]

Hello Guix!

The 'core-updates' branch is now in a "feature freeze" after a long ...
thawing period.  That means that there will be no more world rebuilding
changes apart from bug fixes[*].

It is expected to start the full rebuild in a few days once the
bootstrap is complete on AArch64 and ARMv7, which are unfortunately
lacking in compute power at the moment (side note: if you have hardware
to spare, consider donating!).

The branch currently represents 676 commits by 26 people.  Some
highlights from this round:

* Guix runs natively on GNU/Hurd.
* Guix System can be cross-compiled for foreign architectures.
* The distribution is built with Guile 3.0.
* GNOME 3.34 (on a separate branch, will get merged shortly).

Significant package updates:

* glibc 2.31
* Python 3.8.2
* Ruby 2.6.5
* TeX Live 2019

Other changes that may require adjusting third-party channels:

* "libjpeg" has been deprecated in favor of "libjpeg-turbo".
* "util-linux" gained a "lib" output for significant size savings.
* Build systems now set C_INCLUDE_PATH and CPLUS_INCLUDE_PATH instead of
  CPATH.
* Any package that uses libcurl now respects SSL_CERT_DIR and
  SSL_CERT_FILE.

Thanks to everyone who contributed to this branch.  Let the rebuilds
begin!

[*] There are problems with OpenSSL 1.1.1e and we might take on 1.1.1f
if it is released before the full rebuild starts:
https://mta.openssl.org/pipermail/openssl-project/2020-March/001919.html


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates frozen!

[Jan Nieuwenhuizen]

Marius Bakke writes:

> The branch currently represents 676 commits by 26 people.  Some
> highlights from this round:
>
> * Guix runs natively on GNU/Hurd.
> * Guix System can be cross-compiled for foreign architectures.
> * The distribution is built with Guile 3.0.
> * GNOME 3.34 (on a separate branch, will get merged shortly).

These are exciting changes!  Let's also not forget:

 * for x86 and x86_64: another reduction of the bootstrap binary seed by
roughly %0%, to ~60MB!

> Thanks to everyone who contributed to this branch.  Let the rebuilds
> begin!

\o/

Thanks,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: core-updates frozen!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 953 bytes --]

Jan Nieuwenhuizen <janneke@gnu.org> writes:

> Marius Bakke writes:
>
>> The branch currently represents 676 commits by 26 people.  Some
>> highlights from this round:
>>
>> * Guix runs natively on GNU/Hurd.
>> * Guix System can be cross-compiled for foreign architectures.
>> * The distribution is built with Guile 3.0.
>> * GNOME 3.34 (on a separate branch, will get merged shortly).
>
> These are exciting changes!  Let's also not forget:
>
>  * for x86 and x86_64: another reduction of the bootstrap binary seed by
> roughly %0%, to ~60MB!

I knew I was forgetting something important!  Thanks for the reminder.

For those unaware, this means that the set of trusted binaries at the
root of the package graph from which everything else derives is only 60
MiB (on i686 and x86_64).  The set no longer includes GCC, binutils, or
glibc!

For more information, see jannekes blog post:

https://www.joyofsource.com/guix-reduces-bootstrap-seed-by-50.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates frozen!

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 266 bytes --]

Jan Nieuwenhuizen <janneke@gnu.org> writes:

> These are exciting changes!  Let's also not forget:
>
>  * for x86 and x86_64: another reduction of the bootstrap binary seed by
> roughly %0%, to ~60MB!

%0%? :p

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates frozen!

[Jan Nieuwenhuizen]

Pierre Neidhardt writes:

> Jan Nieuwenhuizen <janneke@gnu.org> writes:
>
>> These are exciting changes!  Let's also not forget:
>>
>>  * for x86 and x86_64: another reduction of the bootstrap binary seed by
>> roughly %0%, to ~60MB!
>
> %0%? :p

Hah -- should have been yeah, 50%!

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: core-updates frozen!

[Jan Nieuwenhuizen]

Marius Bakke writes:

> Jan Nieuwenhuizen <janneke@gnu.org> writes:
>
>> Marius Bakke writes:
>>
>>> The branch currently represents 676 commits by 26 people.  Some
>>> highlights from this round:
>>>
>>> * Guix runs natively on GNU/Hurd.
>>> * Guix System can be cross-compiled for foreign architectures.
>>> * The distribution is built with Guile 3.0.
>>> * GNOME 3.34 (on a separate branch, will get merged shortly).
>>
>> These are exciting changes!  Let's also not forget:
>>
>>  * for x86 and x86_64: another reduction of the bootstrap binary seed by
>> roughly %0%, to ~60MB!
>
> I knew I was forgetting something important!  Thanks for the reminder.

:-)

> For those unaware, this means that the set of trusted binaries at the
> root of the package graph from which everything else derives is only 60
> MiB (on i686 and x86_64).  The set no longer includes GCC, binutils, or
> glibc!

> For more information, see jannekes blog post:
>
> https://www.joyofsource.com/guix-reduces-bootstrap-seed-by-50.html

For the record...it's even better: The GCC, binutils and glibc
removal mentioned above is already in master; so our next release will
bootstrap from ~135MB like blog post mentions!

The Core-updates bootstrap (code named "Scheme-only bootstrap") removes

  Awk, Bash, the GNU Core Utilities, Grep, Gzip, Sed, and Tar.

and replaces them with Gash and Gash-Utils!

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: core-updates frozen!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 882 bytes --]

Jan Nieuwenhuizen <janneke@gnu.org> writes:

>> For those unaware, this means that the set of trusted binaries at the
>> root of the package graph from which everything else derives is only 60
>> MiB (on i686 and x86_64).  The set no longer includes GCC, binutils, or
>> glibc!
>
>> For more information, see jannekes blog post:
>>
>> https://www.joyofsource.com/guix-reduces-bootstrap-seed-by-50.html
>
> For the record...it's even better: The GCC, binutils and glibc
> removal mentioned above is already in master; so our next release will
> bootstrap from ~135MB like blog post mentions!
>
> The Core-updates bootstrap (code named "Scheme-only bootstrap") removes
>
>   Awk, Bash, the GNU Core Utilities, Grep, Gzip, Sed, and Tar.
>
> and replaces them with Gash and Gash-Utils!

Oh my, that's just incredible, I did not think we were there already!

Thanks for the update.  :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates frozen!

[Ludovic Courtès]

Hi!

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> For the record...it's even better: The GCC, binutils and glibc
> removal mentioned above is already in master; so our next release will
> bootstrap from ~135MB like blog post mentions!
>
> The Core-updates bootstrap (code named "Scheme-only bootstrap") removes
>
>   Awk, Bash, the GNU Core Utilities, Grep, Gzip, Sed, and Tar.
>
> and replaces them with Gash and Gash-Utils!

That’s worth a second blog post!  :-)

Ludo’.

Re: core-updates frozen!

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

>> The Core-updates bootstrap (code named "Scheme-only bootstrap") removes
>>
>>   Awk, Bash, the GNU Core Utilities, Grep, Gzip, Sed, and Tar.
>>
>> and replaces them with Gash and Gash-Utils!
>
> That’s worth a second blog post!  :-)

Yes, let's do that when core-updates is merged into master!  "someone"
is still struggling a bit with the documentation anyway.  ;-)

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com