Releasing 2.2.5?

Releasing 2.2.5?

[Ludovic Courtès]

Hello comrades!

What would you think of releasing ‘stable-2.2’ as 2.2.5?

It’s great if you can do it, Mark, but otherwise I can do it.

Thanks,
Ludo’.

Re: Releasing 2.2.5?

[Nala Ginrut]

I'm looking forward to it!

On Thu, Jun 6, 2019 at 4:51 PM Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hello comrades!
>
> What would you think of releasing ‘stable-2.2’ as 2.2.5?
>
> It’s great if you can do it, Mark, but otherwise I can do it.
>
> Thanks,
> Ludo’.
>

Re: Releasing 2.2.5?

[Mike Gran]

On Thu, Jun 06, 2019 at 10:44:44AM +0200, Ludovic Court?s wrote:
> Hello comrades!
> 
> What would you think of releasing ???stable-2.2??? as 2.2.5?

I did include patches a couple of months ago in stable-2.2 so the
Cygwin build is clean.  Not sure how things are with OpenBSD and
MinGW right now.  But it would be worth it just for the Cygwin
fixes.

Regards,
Mike Gran

Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Mark H Weaver]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:
> What would you think of releasing ‘stable-2.2’ as 2.2.5?

I think it's a fine idea.

> It’s great if you can do it, Mark, but otherwise I can do it.

Regrettably, Guile 2.2 has become too heavy to build on the only machine
in my possession that I have any trust in.  I don't have a machine that
I consider sufficiently trustworthy to produce build outputs for wider
distribution.  I'm not sure that any of us do.

To mitigate the risk that a compromised development machine could be
used to attack others, I propose that we adopt a practice of distributed
verification of release tarballs.  We would publish code that uses Guix
to produce the release tarball deterministically, and put out a call for
volunteers to generate the tarball and post signed declarations
containing the hash of the resulting tarball.  After we have received
several such declarations, we can sign and publish the official tarball.

What do you think?

      Mark

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Ludovic Courtès]

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>> What would you think of releasing ‘stable-2.2’ as 2.2.5?
>
> I think it's a fine idea.

Awesome.  We’ll have to update NEWS; I can give it a go, but if you
could add bullet items for the things you’ve worked on, that’d be great.

>> It’s great if you can do it, Mark, but otherwise I can do it.
>
> Regrettably, Guile 2.2 has become too heavy to build on the only machine
> in my possession that I have any trust in.  I don't have a machine that
> I consider sufficiently trustworthy to produce build outputs for wider
> distribution.  I'm not sure that any of us do.

Note that “make dist” is rather inexpensive; “distcheck” is much more
expensive though, but maybe avoidable for a minor release tarball.

> To mitigate the risk that a compromised development machine could be
> used to attack others, I propose that we adopt a practice of distributed
> verification of release tarballs.  We would publish code that uses Guix
> to produce the release tarball deterministically, and put out a call for
> volunteers to generate the tarball and post signed declarations
> containing the hash of the resulting tarball.  After we have received
> several such declarations, we can sign and publish the official tarball.

I don’t think this should block 2.2.5, but I think it’s an idea we
should explore.

One issue is that “make dist” is non-deterministic because the archive
contains timestamps; I’m sure there of other sources of non-determinism
though, because “make dist” was not designed with that in mind.

The non-source byproducts in release tarballs are: the pre-built .go
files (which are optional), psyntax-pp.scm, and then Info files and all
the autotools machinery.  Are these those you had in mind?

Thoughts?

Ludo’.

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Mark H Weaver]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>> What would you think of releasing ‘stable-2.2’ as 2.2.5?
>>
>> I think it's a fine idea.
>
> Awesome.  We’ll have to update NEWS; I can give it a go, but if you
> could add bullet items for the things you’ve worked on, that’d be great.

Sure, I can take care of updating NEWS in the next day or two.

>>> It’s great if you can do it, Mark, but otherwise I can do it.
>>
>> Regrettably, Guile 2.2 has become too heavy to build on the only machine
>> in my possession that I have any trust in.  I don't have a machine that
>> I consider sufficiently trustworthy to produce build outputs for wider
>> distribution.  I'm not sure that any of us do.
>
> Note that “make dist” is rather inexpensive;

I assume it builds the prebuilt .go files, no?  That involves running
Guile's compiler, which is too heavy to run on my Yeeloong.

> “distcheck” is much more
> expensive though, but maybe avoidable for a minor release tarball.
>
>> To mitigate the risk that a compromised development machine could be
>> used to attack others, I propose that we adopt a practice of distributed
>> verification of release tarballs.  We would publish code that uses Guix
>> to produce the release tarball deterministically, and put out a call for
>> volunteers to generate the tarball and post signed declarations
>> containing the hash of the resulting tarball.  After we have received
>> several such declarations, we can sign and publish the official tarball.
>
> I don’t think this should block 2.2.5, but I think it’s an idea we
> should explore.

If you'd like to produce the 2.2.5 release in the traditional way,
that's fine with me.  I'm not comfortable doing it myself, though.

> One issue is that “make dist” is non-deterministic because the archive
> contains timestamps; I’m sure there of other sources of non-determinism
> though, because “make dist” was not designed with that in mind.

Right.  I suppose the right approach is to start a conversation with the
autotools developers.  In the meantime, I wonder if we could implement
our own deterministic version of "make dist" using Guix, and use that
instead.  Or perhaps it would be easier to use "make dist" and then
canonicalize the timestamps in the resulting tarball in a later step?

Thoughts?

> The non-source byproducts in release tarballs are: the pre-built .go
> files (which are optional), psyntax-pp.scm, and then Info files and all
> the autotools machinery.  Are these those you had in mind?

Yes, all of the above are potential security risks, except possibly for
the info files.

     Thanks!
       Mark

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Ludovic Courtès]

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> Sure, I can take care of updating NEWS in the next day or two.

Awesome, thank you!

>>> Regrettably, Guile 2.2 has become too heavy to build on the only machine
>>> in my possession that I have any trust in.  I don't have a machine that
>>> I consider sufficiently trustworthy to produce build outputs for wider
>>> distribution.  I'm not sure that any of us do.
>>
>> Note that “make dist” is rather inexpensive;
>
> I assume it builds the prebuilt .go files, no?  That involves running
> Guile's compiler, which is too heavy to run on my Yeeloong.

I think it’s not that bad if you already have build tree with Guile’s
compiler.  If you start from scratch, it’s definitely expensive.

> If you'd like to produce the 2.2.5 release in the traditional way,
> that's fine with me.  I'm not comfortable doing it myself, though.

OK, I can do this.

>> One issue is that “make dist” is non-deterministic because the archive
>> contains timestamps; I’m sure there of other sources of non-determinism
>> though, because “make dist” was not designed with that in mind.
>
> Right.  I suppose the right approach is to start a conversation with the
> autotools developers.  In the meantime, I wonder if we could implement
> our own deterministic version of "make dist" using Guix, and use that
> instead.  Or perhaps it would be easier to use "make dist" and then
> canonicalize the timestamps in the resulting tarball in a later step?
>
> Thoughts?

I think you raise valid concerns, but they are to some extent beyond the
scope of Guile.

Regarding “make dist”, there’s the issue of tar timestamps, of
version.texi, and probably others of that sort in the
autotools-generated machinery.

So yes, I think this should be discussed with the Automake/Autotools
developers.  Namely: how can we achieve reproducible “dist” builds?
Which tools should honor SOURCE_DATE_EPOCH and how? etc.

As a PoC and/or interim solution, we could also try to hack a
reproducible “make dist” in Guix.

For Guix, from a bootstrapping viewpoint, the alternative is to do away
with tarballs produced by “make dist” and instead always run
“autoreconf” ourselves, like Debian does.  That would solve a large part
of the problem.

>> The non-source byproducts in release tarballs are: the pre-built .go
>> files (which are optional), psyntax-pp.scm, and then Info files and all
>> the autotools machinery.  Are these those you had in mind?
>
> Yes, all of the above are potential security risks, except possibly for
> the info files.

I think psyntax-pp.scm is the main issue since all the others can be
trivially rebuilt (the package in Guix deletes the pre-built .go files
for instance.)

With regards to bootstrapping in the context of Guile, I believe
psyntax-pp.scm should be our primary concern.

Thanks,
Ludo’.

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Mark H Weaver]

Hi Ludovic,

I've finished my updates to the NEWS file in preparation for the 2.2.5
release.  Feel free to reorganize, edit, or expand on the NEWS entries
as you think best.  I believe that all of the relevant changes are
listed, anyway.

Note that GUILE-VERSION still needs to be updated.  See below for the
changes that I believe should be made to it.

If it all looks okay to you, feel free to run distcheck and upload the
release.

     Thanks,
       Mark


--8<---------------cut here---------------start------------->8---
diff --git a/GUILE-VERSION b/GUILE-VERSION
index 32c124c84..bd2525775 100644
--- a/GUILE-VERSION
+++ b/GUILE-VERSION
@@ -3,7 +3,7 @@
 # Note: `GUILE_VERSION' is defined in `configure.ac' using `git-version-gen'.
 GUILE_MAJOR_VERSION=2
 GUILE_MINOR_VERSION=2
-GUILE_MICRO_VERSION=4
+GUILE_MICRO_VERSION=5
 
 GUILE_EFFECTIVE_VERSION=2.2
 
@@ -16,7 +16,7 @@ GUILE_EFFECTIVE_VERSION=2.2
 # See libtool info pages for more information on how and when to
 # change these.
 
-LIBGUILE_INTERFACE_CURRENT=4
-LIBGUILE_INTERFACE_REVISION=1
-LIBGUILE_INTERFACE_AGE=3
+LIBGUILE_INTERFACE_CURRENT=5
+LIBGUILE_INTERFACE_REVISION=0
+LIBGUILE_INTERFACE_AGE=4
 LIBGUILE_INTERFACE="${LIBGUILE_INTERFACE_CURRENT}:${LIBGUILE_INTERFACE_REVISION}:${LIBGUILE_INTERFACE_AGE}"
--8<---------------cut here---------------end--------------->8---

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Ludovic Courtès]

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> I've finished my updates to the NEWS file in preparation for the 2.2.5
> release.  Feel free to reorganize, edit, or expand on the NEWS entries
> as you think best.  I believe that all of the relevant changes are
> listed, anyway.

Thanks a *lot* for all the preparation work, that made things really
easy for me.

> Note that GUILE-VERSION still needs to be updated.  See below for the
> changes that I believe should be made to it.
>
> If it all looks okay to you, feel free to run distcheck and upload the
> release.

Done!  :-)

Thanks,
Ludo’.

PS: This is the first release that I build and upload while on the
    train, crazy stuff.  ;-)

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Mark H Weaver]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> I've finished my updates to the NEWS file in preparation for the 2.2.5
>> release.  Feel free to reorganize, edit, or expand on the NEWS entries
>> as you think best.  I believe that all of the relevant changes are
>> listed, anyway.
>
> Thanks a *lot* for all the preparation work, that made things really
> easy for me.

Given the enormous amount of important work you've been doing on Guix,
and Andy's exciting work on the JIT compiler, it's entirely appropriate
for me to take care of most of the Guile stable release work.

>> If it all looks okay to you, feel free to run distcheck and upload the
>> release.
>
> Done!  :-)

Thanks very much for taking care of the part that I find uncomfortable.

> PS: This is the first release that I build and upload while on the
>     train, crazy stuff.  ;-)

Heh, nice! :)

      Mark

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Neil Jerram]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

On Thu, 20 Jun 2019 at 22:56, Mark H Weaver <mhw@netris.org> wrote:

> Hi Ludovic,
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
> > Mark H Weaver <mhw@netris.org> skribis:
> >
> >> I've finished my updates to the NEWS file in preparation for the 2.2.5
> >> release.  Feel free to reorganize, edit, or expand on the NEWS entries
> >> as you think best.  I believe that all of the relevant changes are
> >> listed, anyway.
> >
> > Thanks a *lot* for all the preparation work, that made things really
> > easy for me.
>
> Given the enormous amount of important work you've been doing on Guix,
> and Andy's exciting work on the JIT compiler, it's entirely appropriate
> for me to take care of most of the Guile stable release work.
>

Thank you to all of you for continuing to push Guile forward!  It seems
like interesting and rigorous developments are coming all the time now, and
I also love how central it is in Guix.

Best wishes,
    Neil

[-- Attachment #2: Type: text/html, Size: 1492 bytes --]

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Rob Browning]

Ludovic Courtès <ludo@gnu.org> writes:

> One issue is that “make dist” is non-deterministic because the archive
> contains timestamps; I’m sure there of other sources of non-determinism
> though, because “make dist” was not designed with that in mind.
>
> The non-source byproducts in release tarballs are: the pre-built .go
> files (which are optional), psyntax-pp.scm, and then Info files and all
> the autotools machinery.  Are these those you had in mind?

If you haven't already seen it, I'd also suggest consulting
https://reproducible-builds.org.  They've been doing a lot of relevant
heavy-lifting over the past few years (working on the relevant tools,
generating patches or workarounds, etc.).  Their diffoscope tool might
also be of interest: https://reproducible-builds.org/tools/

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Ricardo Wurmus]

Hi Rob,

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> One issue is that “make dist” is non-deterministic because the archive
>> contains timestamps; I’m sure there of other sources of non-determinism
>> though, because “make dist” was not designed with that in mind.
>>
>> The non-source byproducts in release tarballs are: the pre-built .go
>> files (which are optional), psyntax-pp.scm, and then Info files and all
>> the autotools machinery.  Are these those you had in mind?
>
> If you haven't already seen it, I'd also suggest consulting
> https://reproducible-builds.org.

Ludovic and other folks in the Guile and Guix communities are
participating in the reproducible builds effort since the first summit.

--
Ricardo

Re: Distributed verification of release tarballs using Guix? (was Re: Releasing 2.2.5?)

[Rob Browning]

Ricardo Wurmus <rekado@elephly.net> writes:

> Ludovic and other folks in the Guile and Guix communities are
> participating in the reproducible builds effort since the first summit.

Oh, excellent.  (Now that you mention it, I think I remember seeing Guix
mentioned, and had just forgotten.)

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4