Re: reproducible builds and debugging information

["Tomáš Čech" <sleep_walker@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 5381 bytes --]

On Tue, Mar 24, 2015 at 10:09:50PM +0100, Ludovic Courtès wrote:
>Tomáš Čech <sleep_walker@gnu.org> skribis:
>
>> As I was tracing curl code, I needed to rebuild the package with
>> "-ggdb" in CFLAGS and enable debug among outputs.
>>
>> The later doesn't change the hash (and the generated code), but the
>> first does.
>
>Both approaches change the output hash.  (As soon as a bit changes in
>the build process, the output hash changes.)
>
>Adding a “debug” output is nice because we have support to automatically
>DTRT (info "(guix) Installing Debugging Files").

For once I have read that and understood that before sending an email.

Adding debug output is not nice because it changes the hash.

>> So I'd like to propose to put "-ggdb" to generally applied CFLAGS for
>> whole distribution.
>
>Packages that have an autoconf-based build system, and I suppose most
>others, are built with -g.

I can't confirm this statement.

I added "debug" output to curl package:

diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index 821a957..996342a 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -74,6 +74,7 @@
          ;; verbose.
          (zero? (system* "make" "-C" "tests" "test")))
        %standard-phases)))
+   (outputs '("out" "debug"))
    (synopsis "Command line tool for transferring data with URL syntax")
    (description
     "curl is a command line tool for transferring data with URL syntax,
				 
Rebuilt:
/gnu/store/c2x7r38zkzf60vz02j7az7r847vy2sng-curl-7.40.0
/gnu/store/9gh88mgh68kxl38vyd42yc8i9v8fa449-curl-7.40.0-debug

And GDB is complaining on the provided files:
Reading symbols from /gnu/store/c2x7r38zkzf60vz02j7az7r847vy2sng-curl-7.40.0/bin/curl...Reading symbols from /gnu/store/9gh88mgh68kxl38vyd42yc8i9v8fa449-curl-7.40.0-debug/lib/debug//gnu/store/c2x7r38zkzf60vz02j7az7r847vy2sng-curl-7.40.0/bin/curl.debug...(no debugging symbols found)...done.
(no debugging symbols found)...done.

We have (hopefully) reproducible builds, you can see for yourself.

After adding `-g' to CFLAGS:
diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index 821a957..d2bf4d9 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -60,7 +60,8 @@
        ("pkg-config" ,pkg-config)
        ("python" ,python-2)))
    (arguments
-    `(#:configure-flags '("--with-gnutls" "--with-gssapi")
+    `(#:configure-flags '("--with-gnutls" "--with-gssapi" "CFLAGS=-g")
+      #:make-flags '("CFLAGS=-g")
       ;; Add a phase to patch '/bin/sh' occurances in tests/runtests.pl
       #:phases
       (alist-replace
@@ -74,6 +75,7 @@
          ;; verbose.
          (zero? (system* "make" "-C" "tests" "test")))
        %standard-phases)))
+   (outputs '("out" "debug"))
    (synopsis "Command line tool for transferring data with URL syntax")
    (description
     "curl is a command line tool for transferring data with URL syntax,

it worked as expected:
(gdb) file /gnu/store/lz28mjhnddb91by1mq4bili35fm1dfyk-curl-7.40.0/bin/curl
Reading symbols from /gnu/store/lz28mjhnddb91by1mq4bili35fm1dfyk-curl-7.40.0/bin/curl...Reading symbols from /gnu/store/ml1dlaqrc8kkrka6xbp7y85iwcjd3p6k-curl-7.40.0-debug/lib/debug///gnu/store/lz28mjhnddb91by1mq4bili35fm1dfyk-curl-7.40.0/bin/curl.debug...done.


>The binaries get stripped by default and
>debugging info is lost unless the package has a “debug” output.

OK, the difference -g and -ggdb is slight, but there is the problem
with "debug" output.

When package has output "debug" always - there is no problem.

When package doesn't have "debug" output and I need it, mere adding
output "debug" into package receipt will change the hash so I'll get
different package.

Better solution for this from my POV would be to keep information
whether keep or drop debug output outside package receipt.

Then `guix package -i curl:debug' could mean request to keep generated
debug informations on disk.

>Currently a few key packages have that, but most don’t (I think Debian
>does something similar, not sure about other distros.)

On openSUSE you have available all the subpackage providing stripped
debug informations and subpackage providing source code from the
moment of build (so DWARF information in debug part can match the source).

On Gentoo it's up to you. It's not in the default configuration, it's
mentioned as recommended for debugging and QA purposes, but that is
not exactly good example as every distribution is different so sharing
corer dumps doesn't make sence.

Gentoo gather all stripped files and IIRC it was posible to keep it
compressed on disk and GDB still could use them...  (Very nice, but yeah, it
won't help hydra with disk space problem.)

>We could make it opt-out rather than opt-in, but the issue is disk usage
>on build machine (including end-user machines.)  See
><http://lists.gnu.org/archive/html/bug-guix/2013-07/msg00015.html>.
>
>Thoughts?

If we have distribution of reproducible packages, we can keep it
opt-in and generate debug information next time (by not dropping
it).

The only problematic packages will be the big ones like Webkit, Libre
Office and similar because generating debug increases memory usage
during the build significantly and may not be suitable for average
personal computer.


S_W

[-- Attachment #2: Type: application/pgp-signature, Size: 181 bytes --]