Reproducible builds: a means to an end

Reproducible builds: a means to an end

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 249 bytes --]

Hello!

I published a note on how I think reproducible builds fit in the bigger
picture of user freedom and user autonomy, and what role Guix can play:

  https://savannah.gnu.org/forum/forum.php?forum_id=8407

Feedback welcome!

Ludo’.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Reproducible builds: a means to an end

[Jan Synáček]

[-- Attachment #1: Type: text/plain, Size: 411 bytes --]

On Wed, Nov 11, 2015 at 3:55 PM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hello!
>
> I published a note on how I think reproducible builds fit in the bigger
> picture of user freedom and user autonomy, and what role Guix can play:
>
>   https://savannah.gnu.org/forum/forum.php?forum_id=8407
>
> Feedback welcome!
>
> Ludo’.
>

The talk mentioned in that post was great!

-- 
Jan Synáček

[-- Attachment #2: Type: text/html, Size: 935 bytes --]

Re: Reproducible builds: a means to an end

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> I published a note on how I think reproducible builds fit in the bigger
> picture of user freedom and user autonomy, and what role Guix can play:
>
>   https://savannah.gnu.org/forum/forum.php?forum_id=8407

That was a very nice read!

I wonder how we as a project could help the reproducible builds project
and/or directly benefit from their findings.  Are there ready-made
patches we could apply to our package recipes?  Or should we just wait
for upstream projects to be fixed?

The utility of “guix challenge” is much reduced when for so many
packages we do not actually have reproducible builds.

(Maybe we could have a page that lists packages that “guix challenge”
suggests as having non-reproducible builds.)

Can we automate some fixes, such as disabling timestamps?  (I see, for
example, that the Python REPL tells me when it was built.)

~~ Ricardo

Re: Reproducible builds: a means to an end

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> I wonder how we as a project could help the reproducible builds project
> and/or directly benefit from their findings.

I was invited to their first Reproducible World Summit in December,
along with people from many different projects.  I guess the main focus
will be on collaboration and knowledge sharing.  We’ll see!

> Are there ready-made patches we could apply to our package recipes?
> Or should we just wait for upstream projects to be fixed?

Sometimes there are ready-made patches that can be found in Debian or
other distros, sometimes not.  Often they’re hard to find though (for
instance, patch-tracker.debian.org seems to be off-line.)

> The utility of “guix challenge” is much reduced when for so many
> packages we do not actually have reproducible builds.
>
> (Maybe we could have a page that lists packages that “guix challenge”
> suggests as having non-reproducible builds.)

“guix challenge” is a simple way to find out which packages are non
deterministic.  That’s how I found about those that can be seen at
<http://bugs.gnu.org/guix> for example.

We could also have a second build farm, probably x86_64-only,
specifically for the purpose of doing independent builds.

The ability to publish the hash of local builds in a peer-to-peer
fashion would be even better.

I also want to merge
<https://github.com/NixOS/nix/commit/8fdd156a650f9b2ce9ae8cd74edcf16225478292>.
There are some issues that this approach cannot catch, but it’s good
enough for all the timestamp or randomness related issues.

> Can we automate some fixes, such as disabling timestamps?  (I see, for
> example, that the Python REPL tells me when it was built.)

I fixed that one in ‘tk-update’.  This particular one could have been
avoided by having GCC return zero for __DATE__ and __TIME__.

However, most other timestamp issues (like Python, Emacs, and Groff
adding timestamps in their byproducts) cannot be addressed
automatically.  That’s why reproducible-build.org as a cross-distro
project is so important.

Ludo’.

Re: Reproducible builds: a means to an end

[Alex Vong]

Hi,

On 16/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:
> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>
>> I wonder how we as a project could help the reproducible builds project
>> and/or directly benefit from their findings.
>
> I was invited to their first Reproducible World Summit in December,
> along with people from many different projects.  I guess the main focus
> will be on collaboration and knowledge sharing.  We’ll see!
>
>> Are there ready-made patches we could apply to our package recipes?
>> Or should we just wait for upstream projects to be fixed?
>
> Sometimes there are ready-made patches that can be found in Debian or
> other distros, sometimes not.  Often they’re hard to find though (for
> instance, patch-tracker.debian.org seems to be off-line.)
>
Yes, according to
<https://lists.debian.org/debian-devel/2014/05/msg00889.html>, the
maintainer of patch-tracker.debian.org has been missing in action
until now. I think the website will be off-line in the near future.

>> The utility of “guix challenge” is much reduced when for so many
>> packages we do not actually have reproducible builds.
>>
>> (Maybe we could have a page that lists packages that “guix challenge”
>> suggests as having non-reproducible builds.)
>
> “guix challenge” is a simple way to find out which packages are non
> deterministic.  That’s how I found about those that can be seen at
> <http://bugs.gnu.org/guix> for example.
>
Does that mean we should have a bug report for every non-reproducible
packages? Or should we only have bug reports for popular packages?

> We could also have a second build farm, probably x86_64-only,
> specifically for the purpose of doing independent builds.
>
> The ability to publish the hash of local builds in a peer-to-peer
> fashion would be even better.
>
> I also want to merge
> <https://github.com/NixOS/nix/commit/8fdd156a650f9b2ce9ae8cd74edcf16225478292>.
> There are some issues that this approach cannot catch, but it’s good
> enough for all the timestamp or randomness related issues.
>
>> Can we automate some fixes, such as disabling timestamps?  (I see, for
>> example, that the Python REPL tells me when it was built.)
>
> I fixed that one in ‘tk-update’.  This particular one could have been
> avoided by having GCC return zero for __DATE__ and __TIME__.
>
> However, most other timestamp issues (like Python, Emacs, and Groff
> adding timestamps in their byproducts) cannot be addressed
> automatically.  That’s why reproducible-build.org as a cross-distro
> project is so important.
>
> Ludo’.
>
>

Cheers,
Alex

Re: Reproducible builds: a means to an end

[Ludovic Courtès]

Alex Vong <alexvong1995@gmail.com> skribis:

> On 16/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

>> Sometimes there are ready-made patches that can be found in Debian or
>> other distros, sometimes not.  Often they’re hard to find though (for
>> instance, patch-tracker.debian.org seems to be off-line.)
>>
> Yes, according to
> <https://lists.debian.org/debian-devel/2014/05/msg00889.html>, the
> maintainer of patch-tracker.debian.org has been missing in action
> until now. I think the website will be off-line in the near future.

OK.

>> “guix challenge” is a simple way to find out which packages are non
>> deterministic.  That’s how I found about those that can be seen at
>> <http://bugs.gnu.org/guix> for example.
>>
> Does that mean we should have a bug report for every non-reproducible
> packages? Or should we only have bug reports for popular packages?

It’s OK to have bug reports for any package, as long as people volunteer
to fix the bugs.  Often it’s a trivial timestamp issue; sometimes it’s
more involved, like <http://bugs.gnu.org/21918>.

Does Debbugs allow us to add custom tags?  We could have a “determinism”
tag to facilitate triage.

Ludo’.

Re: Reproducible builds: a means to an end

[Alex Vong]

# Hi,
#
# I wasn't expecting this, but the Debian BTS actually provides
# mechanism for adding custom tags. I will use #21918 as an example.
#
# Limit the command to the guix package only
package guix
#
# Set an address, it can be the mailing list address for a team
user alexvong1995@gmail.com
#
# Set usertag, the character set [A-Za-z0-9_-] should be safe
usertag 21918 + determinism
#
# Add usercategory unreproducible-build and friends
usercategory unreproducible-build
 * Non-derterministic-build [tag=]
 + Determinism [determinism]
 + Uncategorized []
usercategory normal
 * status
 * unreproducible-build
 * severity
usercategory <pkg>-sort
 * status
 * unreproducible-build
 * severity
usercategory old
 * status
 * severity
 * classification
#
# End of commands
thanks

Now, if you visit
<https://debbugs.gnu.org/cgi/pkgreport.cgi?package=guix;users=alexvong1995@gmail.com>,
you should see `Outstanding bugs -- Determinism; Normal bugs (1 bug)`.
Of course, this is a rough example, the exact wordings can be changed.
Does this work for you?

References:
 https://wiki.debian.org/bugs.debian.org/usertags
 https://www.debian.org/Bugs/server-control
 https://www.debian.org/Bugs/server-request
 https://lists.debian.org/debian-devel-announce/2005/09/msg00002.html

Cheers,
Alex

On 18/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:
> Alex Vong <alexvong1995@gmail.com> skribis:
>
>> On 16/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:
>
> [...]
>
>>> Sometimes there are ready-made patches that can be found in Debian or
>>> other distros, sometimes not.  Often they’re hard to find though (for
>>> instance, patch-tracker.debian.org seems to be off-line.)
>>>
>> Yes, according to
>> <https://lists.debian.org/debian-devel/2014/05/msg00889.html>, the
>> maintainer of patch-tracker.debian.org has been missing in action
>> until now. I think the website will be off-line in the near future.
>
> OK.
>
>>> “guix challenge” is a simple way to find out which packages are non
>>> deterministic.  That’s how I found about those that can be seen at
>>> <http://bugs.gnu.org/guix> for example.
>>>
>> Does that mean we should have a bug report for every non-reproducible
>> packages? Or should we only have bug reports for popular packages?
>
> It’s OK to have bug reports for any package, as long as people volunteer
> to fix the bugs.  Often it’s a trivial timestamp issue; sometimes it’s
> more involved, like <http://bugs.gnu.org/21918>.
>
> Does Debbugs allow us to add custom tags?  We could have a “determinism”
> tag to facilitate triage.
>
> Ludo’.
>

Re: Reproducible builds: a means to an end

[Ludovic Courtès]

Alex Vong <alexvong1995@gmail.com> skribis:

> usercategory unreproducible-build
>  * Non-derterministic-build [tag=]
          ^^
Typo.  :-)

> Now, if you visit
> <https://debbugs.gnu.org/cgi/pkgreport.cgi?package=guix;users=alexvong1995@gmail.com>,
> you should see `Outstanding bugs -- Determinism; Normal bugs (1 bug)`.
> Of course, this is a rough example, the exact wordings can be changed.
> Does this work for you?

It seems that this is a per-user tag, right?  It would be perfect if
this was a global tag that everyone would see, even without passing
“users=” in the URL above.

Does that seem doable?

Thanks for your help!

Ludo’.

Re: Reproducible builds: a means to an end

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1287 bytes --]

On Wed, 18 Nov 2015 19:20:39 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Alex Vong <alexvong1995@gmail.com> skribis:
> 
> > usercategory unreproducible-build
> >  * Non-derterministic-build [tag=]  
>           ^^
> Typo.  :-)
> 
> > Now, if you visit
> > <https://debbugs.gnu.org/cgi/pkgreport.cgi?package=guix;users=alexvong1995@gmail.com>,
> > you should see `Outstanding bugs -- Determinism; Normal bugs (1 bug)`.
> > Of course, this is a rough example, the exact wordings can be changed.
> > Does this work for you?  
> 
> It seems that this is a per-user tag, right?  It would be perfect if
> this was a global tag that everyone would see, even without passing
> “users=” in the URL above.

Looking at https://www.debian.org/Bugs/server-control#tag it looks like we
might have to change the builtin tags to support tags that we want. without
doing that, we could change the title to with something like retitle, and add
[determinism] to the subject line
 
> Does that seem doable?
> 
> Thanks for your help!
> 
> Ludo’.
> 



-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Reproducible builds: a means to an end

[Alex Vong]

Hi all,

On 19/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:
> Alex Vong <alexvong1995@gmail.com> skribis:
>
>> usercategory unreproducible-build
>>  * Non-derterministic-build [tag=]
>           ^^
> Typo.  :-)
>
>> Now, if you visit
>> <https://debbugs.gnu.org/cgi/pkgreport.cgi?package=guix;users=alexvong1995@gmail.com>,
>> you should see `Outstanding bugs -- Determinism; Normal bugs (1 bug)`.
>> Of course, this is a rough example, the exact wordings can be changed.
>> Does this work for you?
>
> It seems that this is a per-user tag, right?  It would be perfect if
> this was a global tag that everyone would see, even without passing
> “users=” in the URL above.
>
> Does that seem doable?
>
I think it is doable since Debian add a release tag for every release
but I can't find the documentation to it.

Beside the technical side, I think it is also a conceptual problem. If
we were to add the `determinism` tag globally, then all of the GNU
projects using <debbugs.debian.org> will be able to use this new tag.
I am conservative towards this, and would rather make the
`determinism` tag be the usertag of `guix-devel@gnu.org`.

What is your idea?

> Thanks for your help!
>
> Ludo’.
>

Cheers,
Alex

Re: Reproducible builds: a means to an end

[Ludovic Courtès]

Alex Vong <alexvong1995@gmail.com> skribis:

> Beside the technical side, I think it is also a conceptual problem. If
> we were to add the `determinism` tag globally, then all of the GNU
> projects using <debbugs.debian.org> will be able to use this new tag.
> I am conservative towards this, and would rather make the
> `determinism` tag be the usertag of `guix-devel@gnu.org`.

If it’s a user tag of guix-devel@gnu.org, does that mean that it would
automatically show up on bugs.gnu.org/guix, even when not explicitly
specifying a user?  Would M-x debbugs show it too?  :-)

If the answers are yes, then go for it!

Ludo’.

Re: Reproducible builds: a means to an end

[Alex Vong]

Hi again,

On 20/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:
> Alex Vong <alexvong1995@gmail.com> skribis:
>
>> Beside the technical side, I think it is also a conceptual problem. If
>> we were to add the `determinism` tag globally, then all of the GNU
>> projects using <debbugs.debian.org> will be able to use this new tag.
>> I am conservative towards this, and would rather make the
>> `determinism` tag be the usertag of `guix-devel@gnu.org`.
>
> If it’s a user tag of guix-devel@gnu.org, does that mean that it would
> automatically show up on bugs.gnu.org/guix, even when not explicitly
> specifying a user?  Would M-x debbugs show it too?  :-)
>
For the web interface, I think no. You will need to pass the
`users=foo` parameter to let it show up.. Otherwise,  the website
interface could become very messy, if everyone adds him/her favourite
tag.

For emacs-debbugs, I think yes! When you type `M-x
debbugs-gnu-usertags`, you get a list of usertags used by the user
with name "emacs", so apparently emacs is using usertags! To list
usertag for a specific user (take "alexvong1995@gmail.com" as an
example), the debbugs info page suggests to evaluate the elisp
function `(debbugs-gnu-usertags "alexvong1995@gmail.com")` and it
works.

> If the answers are yes, then go for it!
>
The answer is half yes and half no. What is your idea?

> Ludo’.
>

Cheers,
Alex

Re: Reproducible builds: a means to an end

[Ludovic Courtès]

Alex Vong <alexvong1995@gmail.com> skribis:

> On 20/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:
>> Alex Vong <alexvong1995@gmail.com> skribis:
>>
>>> Beside the technical side, I think it is also a conceptual problem. If
>>> we were to add the `determinism` tag globally, then all of the GNU
>>> projects using <debbugs.debian.org> will be able to use this new tag.
>>> I am conservative towards this, and would rather make the
>>> `determinism` tag be the usertag of `guix-devel@gnu.org`.
>>
>> If it’s a user tag of guix-devel@gnu.org, does that mean that it would
>> automatically show up on bugs.gnu.org/guix, even when not explicitly
>> specifying a user?  Would M-x debbugs show it too?  :-)
>>
> For the web interface, I think no. You will need to pass the
> `users=foo` parameter to let it show up.. Otherwise,  the website
> interface could become very messy, if everyone adds him/her favourite
> tag.
>
> For emacs-debbugs, I think yes! When you type `M-x
> debbugs-gnu-usertags`, you get a list of usertags used by the user
> with name "emacs", so apparently emacs is using usertags! To list
> usertag for a specific user (take "alexvong1995@gmail.com" as an
> example), the debbugs info page suggests to evaluate the elisp
> function `(debbugs-gnu-usertags "alexvong1995@gmail.com")` and it
> works.

So, what about adding this user tag for user “guix,” and then
recommending using this user?

I tried C-u M-x debbugs-gnu and passing it the “tagged” severity, as
suggested in debbugs.el, and then the “determinism” user tag, but that
fails as of debbugs 0.7:

--8<---------------cut here---------------start------------->8---
Debugger entered--Lisp error: (error "Unknown key: :status")
  signal(error ("Unknown key: :status"))
  error("Unknown key: %s" :status)
  debbugs-get-usertag(:tag "determinism" :status "forwarded" :status "open" :user "guix" :severity "normal" :severity "important" :severity "serious")
  apply(debbugs-get-usertag (:tag "determinism" :status "forwarded" :status "open" :user "guix" :severity "normal" :severity "important" :severity "serious"))
  debbugs-gnu-get-bugs(((tag . "determinism") (status . "forwarded") (status . "open") (package . "guix") (severity . "tagged") (severity . "normal") (severity . "important") (severity . "serious")))
  debbugs-gnu(("serious" "important" "normal" "tagged") ("guix") nil t ("determinism"))
  call-interactively(debbugs-gnu record nil)
  command-execute(debbugs-gnu record)
  execute-extended-command((4) "debbugs-gnu")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)
--8<---------------cut here---------------end--------------->8---

Am I missing something?

Ludo’.

Re: Reproducible builds: a means to an end

[Alex Vong]

Hi,

On 21/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:
> Alex Vong <alexvong1995@gmail.com> skribis:
>
>> On 20/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:
>>> Alex Vong <alexvong1995@gmail.com> skribis:
>>>
>>>> Beside the technical side, I think it is also a conceptual problem. If
>>>> we were to add the `determinism` tag globally, then all of the GNU
>>>> projects using <debbugs.debian.org> will be able to use this new tag.
>>>> I am conservative towards this, and would rather make the
>>>> `determinism` tag be the usertag of `guix-devel@gnu.org`.
>>>
>>> If it’s a user tag of guix-devel@gnu.org, does that mean that it would
>>> automatically show up on bugs.gnu.org/guix, even when not explicitly
>>> specifying a user?  Would M-x debbugs show it too?  :-)
>>>
>> For the web interface, I think no. You will need to pass the
>> `users=foo` parameter to let it show up.. Otherwise,  the website
>> interface could become very messy, if everyone adds him/her favourite
>> tag.
>>
>> For emacs-debbugs, I think yes! When you type `M-x
>> debbugs-gnu-usertags`, you get a list of usertags used by the user
>> with name "emacs", so apparently emacs is using usertags! To list
>> usertag for a specific user (take "alexvong1995@gmail.com" as an
>> example), the debbugs info page suggests to evaluate the elisp
>> function `(debbugs-gnu-usertags "alexvong1995@gmail.com")` and it
>> works.
>
> So, what about adding this user tag for user “guix,” and then
> recommending using this user?
>
Sure! I guess this is how the emacs devs does it. This can help
maintaining a list of unreproducible packages.

> I tried C-u M-x debbugs-gnu and passing it the “tagged” severity, as
> suggested in debbugs.el, and then the “determinism” user tag, but that
> fails as of debbugs 0.7:
>
> --8<---------------cut here---------------start------------->8---
> Debugger entered--Lisp error: (error "Unknown key: :status")
>   signal(error ("Unknown key: :status"))
>   error("Unknown key: %s" :status)
>   debbugs-get-usertag(:tag "determinism" :status "forwarded" :status "open"
> :user "guix" :severity "normal" :severity "important" :severity "serious")
>   apply(debbugs-get-usertag (:tag "determinism" :status "forwarded" :status
> "open" :user "guix" :severity "normal" :severity "important" :severity
> "serious"))
>   debbugs-gnu-get-bugs(((tag . "determinism") (status . "forwarded") (status
> . "open") (package . "guix") (severity . "tagged") (severity . "normal")
> (severity . "important") (severity . "serious")))
>   debbugs-gnu(("serious" "important" "normal" "tagged") ("guix") nil t
> ("determinism"))
>   call-interactively(debbugs-gnu record nil)
>   command-execute(debbugs-gnu record)
>   execute-extended-command((4) "debbugs-gnu")
>   call-interactively(execute-extended-command nil nil)
>   command-execute(execute-extended-command)
> --8<---------------cut here---------------end--------------->8---
>
> Am I missing something?
>
> Ludo’.
>

I think entering `C-u M-x debbugs-gnu-usertags` and then USER should
show all the usertags of user USER, for example USER could be `emacs`
or `alexxvong1995@gmail.com`. Does this work for you?

Cheers,
Alex

Re: Reproducible builds: a means to an end

[Ludovic Courtès]

Alex Vong <alexvong1995@gmail.com> skribis:

> On 21/11/2015, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

>> So, what about adding this user tag for user “guix,” and then
>> recommending using this user?
>>
> Sure! I guess this is how the emacs devs does it. This can help
> maintaining a list of unreproducible packages.

Great.

> I think entering `C-u M-x debbugs-gnu-usertags` and then USER should
> show all the usertags of user USER, for example USER could be `emacs`
> or `alexxvong1995@gmail.com`. Does this work for you?

It does.  Thanks for the tip!

Ludo’.