Publishing an Official Statement on Self-Hosted Compilers

Publishing an Official Statement on Self-Hosted Compilers

[Alex Griffin]

Hello Guixlings,

One thing I think the Guix project should do is work with the
reproducible builds folks to publish a document explaining the issues
involved with self-hosted compilers. It should encourage language
communities to continuously maintain some way to build their language
starting from hand-written C source code (or another language which can
itself be bootstrapped from C). It could also mention that some members
of our community are exploring ways to bootstrap gcc.

What do you think? It might be a total flop, but it looks like something
we should try anyway! At the moment I do not see other communities
talking about this. Guix is deeply concerned about these issues, but
ultimately we cannot fix everything alone. If we can convince the
Rust/OCaml/Haskell folks that this is important, we may be able to
attract a much larger group of people to bear on the problem.

Thanks for your thoughts,
-- 
Alex Griffin

Re: Publishing an Official Statement on Self-Hosted Compilers

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1646 bytes --]

On Thu, May 05, 2016 at 09:52:32AM -0500, Alex Griffin wrote:
> Hello Guixlings,
> 
> One thing I think the Guix project should do is work with the
> reproducible builds folks to publish a document explaining the issues
> involved with self-hosted compilers. It should encourage language
> communities to continuously maintain some way to build their language
> starting from hand-written C source code (or another language which can
> itself be bootstrapped from C). It could also mention that some members
> of our community are exploring ways to bootstrap gcc.
> 
> What do you think? It might be a total flop, but it looks like something
> we should try anyway! At the moment I do not see other communities
> talking about this. Guix is deeply concerned about these issues, but
> ultimately we cannot fix everything alone. If we can convince the
> Rust/OCaml/Haskell folks that this is important, we may be able to
> attract a much larger group of people to bear on the problem.
> 
> Thanks for your thoughts,
> -- 
> Alex Griffin
> 

It's not something I had really thought of before, with the focus being
on reproducable building of packages. It doesn't take much, though, to
realize that you have to start from somewhere to get reproducable and
trustable binaries. While it's easy to throw up your hands and say "its
turtles all the way down," the more the turtles rest on C's turtles the
better.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Publishing an Official Statement on Self-Hosted Compilers

[Pjotr Prins]

Reproducible builds of D compilers
http://forum.dlang.org/post/fsmdaethvbvcxnunbugb@forum.dlang.org

On Fri, May 06, 2016 at 01:30:18AM +0300, Efraim Flashner wrote:
> On Thu, May 05, 2016 at 09:52:32AM -0500, Alex Griffin wrote:
> > Hello Guixlings,
> > 
> > One thing I think the Guix project should do is work with the
> > reproducible builds folks to publish a document explaining the issues
> > involved with self-hosted compilers. It should encourage language
> > communities to continuously maintain some way to build their language
> > starting from hand-written C source code (or another language which can
> > itself be bootstrapped from C). It could also mention that some members
> > of our community are exploring ways to bootstrap gcc.
> > 
> > What do you think? It might be a total flop, but it looks like something
> > we should try anyway! At the moment I do not see other communities
> > talking about this. Guix is deeply concerned about these issues, but
> > ultimately we cannot fix everything alone. If we can convince the
> > Rust/OCaml/Haskell folks that this is important, we may be able to
> > attract a much larger group of people to bear on the problem.
> > 
> > Thanks for your thoughts,
> > -- 
> > Alex Griffin
> > 
> 
> It's not something I had really thought of before, with the focus being
> on reproducable building of packages. It doesn't take much, though, to
> realize that you have to start from somewhere to get reproducable and
> trustable binaries. While it's easy to throw up your hands and say "its
> turtles all the way down," the more the turtles rest on C's turtles the
> better.
> 
> -- 
> Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
> GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted



-- 

Re: Publishing an Official Statement on Self-Hosted Compilers

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1531 bytes --]

On Thu, May 05, 2016 at 09:52:32AM -0500, Alex Griffin wrote:
     Hello Guixlings,
     
     One thing I think the Guix project should do is work with the
     reproducible builds folks to publish a document explaining the issues
     involved with self-hosted compilers. It should encourage language
     communities to continuously maintain some way to build their language
     starting from hand-written C source code (or another language which can
     itself be bootstrapped from C). It could also mention that some members
     of our community are exploring ways to bootstrap gcc.
     
     What do you think? It might be a total flop, but it looks like something
     we should try anyway! At the moment I do not see other communities
     talking about this. Guix is deeply concerned about these issues, but
     ultimately we cannot fix everything alone. If we can convince the
     Rust/OCaml/Haskell folks that this is important, we may be able to
     attract a much larger group of people to bear on the problem.
     
     Thanks for your thoughts,

I fully agree.  And like you say, bootstrapping gcc also belongs as part of this
exercise.  In fact I would not stop at C  I think it should be possible to
have traceability to a hand crafted assembler.

J'

-- 
Avoid eavesdropping.  Send strong encryted email.
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: Publishing an Official Statement on Self-Hosted Compilers

[Ludovic Courtès]

Hello!

Alex Griffin <a@ajgrf.com> skribis:

> One thing I think the Guix project should do is work with the
> reproducible builds folks to publish a document explaining the issues
> involved with self-hosted compilers. It should encourage language
> communities to continuously maintain some way to build their language
> starting from hand-written C source code (or another language which can
> itself be bootstrapped from C). It could also mention that some members
> of our community are exploring ways to bootstrap gcc.
>
> What do you think? It might be a total flop, but it looks like something
> we should try anyway! At the moment I do not see other communities
> talking about this. Guix is deeply concerned about these issues, but
> ultimately we cannot fix everything alone. If we can convince the
> Rust/OCaml/Haskell folks that this is important, we may be able to
> attract a much larger group of people to bear on the problem.

I think it’s a good idea!  A lot of the work to fix this issue will be
to raise awareness among compiler writers and invite them to have a
bootstrapping story like you describe.

Other people in the reproducible-builds community are interested in this
so yes, it sounds like the right place to discuss it.

Would you like to get it started?  :-)  We could discuss it on
rb-general@lists.reproducible-builds.org¹ and here.

Thanks,
Ludo’.

¹ http://lists.reproducible-builds.org/pipermail/rb-general/

Re: Publishing an Official Statement on Self-Hosted Compilers

[Alex Griffin]

On Fri, May 6, 2016, at 05:09 AM, Ludovic Courtès wrote:
> I think it’s a good idea!  A lot of the work to fix this issue will be
> to raise awareness among compiler writers and invite them to have a
> bootstrapping story like you describe.
> 
> Other people in the reproducible-builds community are interested in this
> so yes, it sounds like the right place to discuss it.
> 
> Would you like to get it started?  :-)  We could discuss it on
> rb-general@lists.reproducible-builds.org¹ and here.
> 
> Thanks,
> Ludo’.

Sure, this weekend I'll put together an outline of everything I think we
should include and then solicit more feedback.

Thanks everybody!
-- 
Alex Griffin

Re: Publishing an Official Statement on Self-Hosted Compilers

[Ludovic Courtès]

Alex Griffin <a@ajgrf.com> skribis:

> On Fri, May 6, 2016, at 05:09 AM, Ludovic Courtès wrote:
>> I think it’s a good idea!  A lot of the work to fix this issue will be
>> to raise awareness among compiler writers and invite them to have a
>> bootstrapping story like you describe.
>> 
>> Other people in the reproducible-builds community are interested in this
>> so yes, it sounds like the right place to discuss it.
>> 
>> Would you like to get it started?  :-)  We could discuss it on
>> rb-general@lists.reproducible-builds.org¹ and here.
>> 
>> Thanks,
>> Ludo’.
>
> Sure, this weekend I'll put together an outline of everything I think we
> should include and then solicit more feedback.

Awesome, thank you!

Ludo’.

Re: Publishing an Official Statement on Self-Hosted Compilers

[Alex Griffin]

I've put my initial notes in a git repository
[here](https://gitlab.com/ajgrf/bootstrapping-compilers/blob/master/notes.org).
They are in a very rough state, but mostly everything is there in some
form. If anyone has any thoughts please let me know! Also, if you want
to contribute changes you can send me patches, GitLab pull requests, or
just ask for commit access.

I think I am going to mull over my notes for a couple more days before I
email the good folks at reproducible builds, though.
-- 
Alex Griffin


On Mon, May 9, 2016, at 03:29 AM, Ludovic Courtès wrote:
> Alex Griffin <a@ajgrf.com> skribis:
> 
> > On Fri, May 6, 2016, at 05:09 AM, Ludovic Courtès wrote:
> >> I think it’s a good idea!  A lot of the work to fix this issue will be
> >> to raise awareness among compiler writers and invite them to have a
> >> bootstrapping story like you describe.
> >> 
> >> Other people in the reproducible-builds community are interested in this
> >> so yes, it sounds like the right place to discuss it.
> >> 
> >> Would you like to get it started?  :-)  We could discuss it on
> >> rb-general@lists.reproducible-builds.org¹ and here.
> >> 
> >> Thanks,
> >> Ludo’.
> >
> > Sure, this weekend I'll put together an outline of everything I think we
> > should include and then solicit more feedback.
> 
> Awesome, thank you!
> 
> Ludo’.

Re: Publishing an Official Statement on Self-Hosted Compilers

[Ludovic Courtès]

Hi!

Alex Griffin <a@ajgrf.com> skribis:

> I've put my initial notes in a git repository
> [here](https://gitlab.com/ajgrf/bootstrapping-compilers/blob/master/notes.org).
> They are in a very rough state, but mostly everything is there in some
> form. If anyone has any thoughts please let me know! 

I like it.  :-)

I think the bit about the “trusting trust” attack should go under “The
Problem”.  Specifically, I would suggest expounding on the software
freedom bit (the fact that users must be provided with the Corresponding
Source), and the reproducibility bit (allow people to build from source
and to ensure the binaries correspond to the source), and then on
security (“trusting trust”.)

WDYT?

We should then discuss it with the repro-builds folks, and probably
contact a bunch of compiler writers to get initial feedback.

Thanks,
Ludo’.

PS: I would suggest wrapping lines in notes.org, which would make it
    easier to read IMO, and also facilitate patch handling.

Re: Publishing an Official Statement on Self-Hosted Compilers

[Ludovic Courtès]

Hi!

Alex Griffin <a@ajgrf.com> skribis:

> I've put my initial notes in a git repository
> [here](https://gitlab.com/ajgrf/bootstrapping-compilers/blob/master/notes.org).
> They are in a very rough state, but mostly everything is there in some
> form. If anyone has any thoughts please let me know! 

I like it.  :-)

I think the bit about the “trusting trust” attack should go under “The
Problem”.  Specifically, I would suggest expounding on the software
freedom bit (the fact that users must be provided with the Corresponding
Source), and the reproducibility bit (allow people to build from source
and to ensure the binaries correspond to the source), and then on
security (“trusting trust”.)

WDYT?

We should then discuss it with the repro-builds folks, and probably
contact a bunch of compiler writers to get initial feedback.

Thanks,
Ludo’.

PS: I would suggest wrapping lines in notes.org, which would make it
    easier to read IMO, and also facilitate patch handling.

Re: Publishing an Official Statement on Self-Hosted Compilers

[Alex Griffin]

On Thu, May 12, 2016, at 05:05 AM, Ludovic Courtès wrote:
> I like it.  :-)
> 
> I think the bit about the “trusting trust” attack should go under “The
> Problem”. [...]
> 
> WDYT?
> 
> We should then discuss it with the repro-builds folks, and probably
> contact a bunch of compiler writers to get initial feedback.

Thanks for the feedback, I agree!

Unfortunately I can't continue working on this right now. Even though
this is not such a large task, it still requires more attention than I
can afford at the moment. I intend to come back to it later, once I am
confident that I can do a good job without neglecting anything more
important. In the meantime, everyone should feel free to work on this
without me if they want.

Thanks for understanding,
-- 
Alex Griffin