Debugging and source code

Debugging and source code

[Pjotr Prins]

Since we allow for debug symbols using :debug (great feature!), can we
also add the unpacked source tree to the package by default? Say in
~/.guix-profile/lib/debug/ or some place. Anyone who wants to debug
has to fetch that anyway and that raises the problem of what source
code to fetch.

Re: Debugging and source code

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> Since we allow for debug symbols using :debug (great feature!), can we
> also add the unpacked source tree to the package by default? Say in
> ~/.guix-profile/lib/debug/ or some place. Anyone who wants to debug
> has to fetch that anyway and that raises the problem of what source
> code to fetch.

You can always get the right source code by running “guix build -S
package”.

OTOH having the source directly in the “debug” (or “source”?) output as
you suggest would be more convenient.

The only downside is the extra size of the “debug” output.  Most of the
time that’s a price people are happy to pay when they are installing the
“debug” output.  But sometimes maybe not.

For example, I always have gcc-toolchain:debug, libgc:debug, and a few
others in my profile, “just in case.”  My goal is to have libc and libgc
symbols when I something goes wrong, but I typically already have
checkouts of these libraries elsewhere.  If the source was in the
“debug” output, I’d pay a pretty high cost (esp. for libc) that I could
have avoided.  Maybe this is a corner case, though.

What do people think?

Ludo’.

Re: Debugging and source code

[Pjotr Prins]

On Wed, Jan 25, 2017 at 02:22:48PM +0100, Ludovic Courtès wrote:
> You can always get the right source code by running “guix build -S
> package”.
> 
> OTOH having the source directly in the “debug” (or “source”?) output as
> you suggest would be more convenient.
> 
> The only downside is the extra size of the “debug” output.  Most of the
> time that’s a price people are happy to pay when they are installing the
> “debug” output.  But sometimes maybe not.

How about a :debug-full switch?

Pj.

Re: Debugging and source code

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Wed, Jan 25, 2017 at 02:22:48PM +0100, Ludovic Courtès wrote:
>> You can always get the right source code by running “guix build -S
>> package”.
>> 
>> OTOH having the source directly in the “debug” (or “source”?) output as
>> you suggest would be more convenient.
>> 
>> The only downside is the extra size of the “debug” output.  Most of the
>> time that’s a price people are happy to pay when they are installing the
>> “debug” output.  But sometimes maybe not.
>
> How about a :debug-full switch?

There’s no switch, only a “debug” output.

Ludo’.

Re: Debugging and source code

[Danny Milosavljevic]

Hi,

On Wed, 25 Jan 2017 14:22:48 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> 
> > Since we allow for debug symbols using :debug (great feature!), 

Aha? Why's that not available in our icecat package? I have to find out how it's done elsewhere and copy it :)

> You can always get the right source code by running “guix build -S
> package”.
> 
> OTOH having the source directly in the “debug” (or “source”?) output as
> you suggest would be more convenient.
> 
> The only downside is the extra size of the “debug” output.  Most of the
> time that’s a price people are happy to pay when they are installing the
> “debug” output.  But sometimes maybe not.

Ahhh that's a difficult call. I think it would be best if we collected some statistics before we decided that. For example how big is the source code of a package versus the binary? On average? Depending on the language? How many % of the total size is the source code?

Re: Debugging and source code

[Pjotr Prins]

On Wed, Jan 25, 2017 at 06:54:30PM +0100, Ludovic Courtès wrote:
> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> 
> > On Wed, Jan 25, 2017 at 02:22:48PM +0100, Ludovic Courtès wrote:
> >> You can always get the right source code by running “guix build -S
> >> package”.
> >> 
> >> OTOH having the source directly in the “debug” (or “source”?) output as
> >> you suggest would be more convenient.
> >> 
> >> The only downside is the extra size of the “debug” output.  Most of the
> >> time that’s a price people are happy to pay when they are installing the
> >> “debug” output.  But sometimes maybe not.
> >
> > How about a :debug-full switch?
> 
> There’s no switch, only a “debug” output.

yes, yes. I meant a "debug-full" output.

-- 

Re: Debugging and source code

[Pjotr Prins]

On Wed, Jan 25, 2017 at 09:41:09PM +0100, Danny Milosavljevic wrote:
> Hi,
> 
> On Wed, 25 Jan 2017 14:22:48 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
> 
> > Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> > 
> > > Since we allow for debug symbols using :debug (great feature!), 
> 
> Aha? Why's that not available in our icecat package? I have to find out how it's done elsewhere and copy it :)

The beauty of it that it is already there - at least for the gnu build
system. Just add the debug output.

> > You can always get the right source code by running “guix build -S
> > package”.
> > 
> > OTOH having the source directly in the “debug” (or “source”?) output as
> > you suggest would be more convenient.
> > 
> > The only downside is the extra size of the “debug” output.  Most of the
> > time that’s a price people are happy to pay when they are installing the
> > “debug” output.  But sometimes maybe not.
> 
> Ahhh that's a difficult call. I think it would be best if we collected some statistics before we decided that. For example how big is the source code of a package versus the binary? On average? Depending on the language? How many % of the total size is the source code?

If we add debug output by default I suppose we don't have to
distribute binary packages. Likewise an output with source code
included. That means there would be no cost involved other than
building and testing.
-- 

Re: Debugging and source code

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On Wed, Jan 25, 2017 at 06:54:30PM +0100, Ludovic Courtès wrote:
>> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>> 
>> > On Wed, Jan 25, 2017 at 02:22:48PM +0100, Ludovic Courtès wrote:
>> >> You can always get the right source code by running “guix build -S
>> >> package”.
>> >> 
>> >> OTOH having the source directly in the “debug” (or “source”?) output as
>> >> you suggest would be more convenient.
>> >> 
>> >> The only downside is the extra size of the “debug” output.  Most of the
>> >> time that’s a price people are happy to pay when they are installing the
>> >> “debug” output.  But sometimes maybe not.
>> >
>> > How about a :debug-full switch?
>> 
>> There’s no switch, only a “debug” output.
>
> yes, yes. I meant a "debug-full" output.

We have "guix build -S foo" which fetches the source.  It sounds like
you're suggesting that we remove this and replace it with a "source" (or
"debug-full") output for each package, so to get the source you would
run "guix build foo:source" instead.  Is my understanding correct?

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Debugging and source code

[Pjotr Prins]

On Thu, Jan 26, 2017 at 12:11:19AM -0800, Chris Marusich wrote:
> Pjotr Prins <pjotr.public12@thebird.nl> writes:
> 
> > On Wed, Jan 25, 2017 at 06:54:30PM +0100, Ludovic Courtès wrote:
> >> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> >> 
> >> > On Wed, Jan 25, 2017 at 02:22:48PM +0100, Ludovic Courtès wrote:
> >> >> You can always get the right source code by running “guix build -S
> >> >> package”.
> >> >> 
> >> >> OTOH having the source directly in the “debug” (or “source”?) output as
> >> >> you suggest would be more convenient.
> >> >> 
> >> >> The only downside is the extra size of the “debug” output.  Most of the
> >> >> time that’s a price people are happy to pay when they are installing the
> >> >> “debug” output.  But sometimes maybe not.
> >> >
> >> > How about a :debug-full switch?
> >> 
> >> There’s no switch, only a “debug” output.
> >
> > yes, yes. I meant a "debug-full" output.
> 
> We have "guix build -S foo" which fetches the source.  It sounds like
> you're suggesting that we remove this and replace it with a "source" (or
> "debug-full") output for each package, so to get the source you would
> run "guix build foo:source" instead.  Is my understanding correct?

With binaries and debug info. Yes. The current :debug does not
include the sources which makes debugging hard(er).

Pj.

Re: Debugging and source code

[Tobias Geerinckx-Rice]

On 2017-01-26 09:11, Chris Marusich wrote:
> We have "guix build -S foo" which fetches the source.  It sounds like
> you're suggesting that we remove this and replace it with a "source" 
> (or
> "debug-full") output for each package, so to get the source you would
> run "guix build foo:source" instead.  Is my understanding correct?

Interesting idea (:source, not :debug-full)! Two minor points:

Won't that mean that the package has to be built to completion at least 
once, somewhere, to output its own :source? If so, adding special-case 
code to avoid that would not be a net win.

We'd also have to choose between keeping a misleading name, 'outputs', 
and a global bike-sed.

Kind regards,

T G-R

-- 
Sent from a web browser. Excuse/enjoy my brevity.

Re: Debugging and source code

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Wed, Jan 25, 2017 at 06:54:30PM +0100, Ludovic Courtès wrote:
>> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>> 
>> > On Wed, Jan 25, 2017 at 02:22:48PM +0100, Ludovic Courtès wrote:
>> >> You can always get the right source code by running “guix build -S
>> >> package”.
>> >> 
>> >> OTOH having the source directly in the “debug” (or “source”?) output as
>> >> you suggest would be more convenient.
>> >> 
>> >> The only downside is the extra size of the “debug” output.  Most of the
>> >> time that’s a price people are happy to pay when they are installing the
>> >> “debug” output.  But sometimes maybe not.
>> >
>> > How about a :debug-full switch?
>> 
>> There’s no switch, only a “debug” output.
>
> yes, yes. I meant a "debug-full" output.

I don’t think we should have both “debug” and “debug-full”.

Instead, the possible options I see are:

  0. Status quo: people need to get the source by themselves with “guix
     build -S”.

  1. The “debug” output, when it exists, always includes the source.

  2. When a “source” output exists, the source code is copied there.
     The “debug” output contains nothing more than debugging info, as is
     currently the case.

We’ve already discussed #1.

With #2, we’d have to manually opt-in in package recipes.  So we could
have:

  (package
    (output '("out" "debug"))  ;like now
    …)

or:

  (package
    (output '("out" "source" "debug"))
    …)

Maybe that’s too inconvenient though.

Thoughts?

Ludo’.

Re: Debugging and source code

[Ludovic Courtès]

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

> On 2017-01-26 09:11, Chris Marusich wrote:
>> We have "guix build -S foo" which fetches the source.  It sounds like
>> you're suggesting that we remove this and replace it with a "source"
>> (or
>> "debug-full") output for each package, so to get the source you would
>> run "guix build foo:source" instead.  Is my understanding correct?
>
> Interesting idea (:source, not :debug-full)! Two minor points:
>
> Won't that mean that the package has to be built to completion at
> least once, somewhere, to output its own :source? If so, adding
> special-case code to avoid that would not be a net win.

Right, for this reason a “source” output cannot supersede ‘guix build
-S’ (and internally, origins need to exist.)

Ludo’.

Re: Debugging and source code

[Pjotr Prins]

On Thu, Jan 26, 2017 at 10:54:56AM +0100, Ludovic Courtès wrote:
>   0. Status quo: people need to get the source by themselves with “guix
>      build -S”.
> 
>   1. The “debug” output, when it exists, always includes the source.
> 
>   2. When a “source” output exists, the source code is copied there.
>      The “debug” output contains nothing more than debugging info, as is
>      currently the case.
> 
> We’ve already discussed #1.
> 
> With #2, we’d have to manually opt-in in package recipes.  So we could
> have:
> 
>   (package
>     (output '("out" "debug"))  ;like now
>     …)
> 
> or:
> 
>   (package
>     (output '("out" "source" "debug"))
>     …)
> 
> Maybe that’s too inconvenient though.
> 
> Thoughts?

I think we are missing something here. What I want to achieve is that
someone can do 'guix package -i gdb foo:debug-with-sources' so he/she can 
simply start the debugger with

   gdb ~/.guix-profile/bin/foo

and have full debugging information to test drive.

The problem I am facing now is that I have to explain non-programmers
to fetch the source code, unpack it from a *named* store path, go to
the new directory and then run gdb. Not only that, this package
fetches multiple source trees from multiple origins (a common thing in
programming languages). So I'll have to explain how to fetch each
individual tree and make sure they play well together.

And that is exactly what the build system does!!

So, the next idea is to use -K and tell someone to move into the
temporary directory. But then again I miss the dependencies with their
trees.

The :debug target is awesome, it wouuld be even more awesome to point
to source trees from the profile - so they are easy to find and in
sync with the built target. It would mean you can ask anyone to
debug software like we are doing here:
https://github.com/lomereiter/sambamba/issues/219#issuecomment-275615245

Note that this feature is to debug complex software on architectures I
have no access for - like some HPC environments. In the next step I'll
make it relocatable too. I can probably work it using multiple -K's,
but it would be a one-off for every software package. 

That is my request. 

I have a feeling that if I have a use case there will be more takers,
what to think of icecat ;)

-- 

Re: Debugging and source code

[Ludovic Courtès]

Hi Pjotr!

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Thu, Jan 26, 2017 at 10:54:56AM +0100, Ludovic Courtès wrote:
>>   0. Status quo: people need to get the source by themselves with “guix
>>      build -S”.
>> 
>>   1. The “debug” output, when it exists, always includes the source.
>> 
>>   2. When a “source” output exists, the source code is copied there.
>>      The “debug” output contains nothing more than debugging info, as is
>>      currently the case.
>> 
>> We’ve already discussed #1.
>> 
>> With #2, we’d have to manually opt-in in package recipes.  So we could
>> have:
>> 
>>   (package
>>     (output '("out" "debug"))  ;like now
>>     …)
>> 
>> or:
>> 
>>   (package
>>     (output '("out" "source" "debug"))
>>     …)
>> 
>> Maybe that’s too inconvenient though.
>> 
>> Thoughts?
>
> I think we are missing something here. What I want to achieve is that
> someone can do 'guix package -i gdb foo:debug-with-sources' so he/she can 
> simply start the debugger with
>
>    gdb ~/.guix-profile/bin/foo
>
> and have full debugging information to test drive.

That’s already what happens, though one needs to tell GDB about the
location of debug info as well as the location of the source:

  https://www.gnu.org/software/guix/manual/html_node/Installing-Debugging-Files.html

> The problem I am facing now is that I have to explain non-programmers
> to fetch the source code, unpack it from a *named* store path, go to
> the new directory and then run gdb. Not only that, this package
> fetches multiple source trees from multiple origins (a common thing in
> programming languages). So I'll have to explain how to fetch each
> individual tree and make sure they play well together.

I don’t think someone who runs GDB can still be called a
“non-programmer”.  :-)

Regardless, as I wrote, I do agree that shipping source in the “debug”
output or similar would facilitate things, the question is about how to
do that in practice (the list of proposals above.)

Ludo’.

Re: Debugging and source code

[Danny Milosavljevic]

Hi,

for example when you try 

  guix build -S ldc

it will not put phobos in the correct place (which is unpacked to the correct place in a phase).

It would be nice if there was a way to run gdb after the build phase ran.

[I guess easiest would be if one could insert something after the 'build phase (and/or after the 'unpack phase) from the command line. Sounds hacky, though.
 Or maybe a phase that's always there in the build systems which doesn't do anything if no command line "enable" flag is given. That sounds hacky.]

Or maybe something like --keep-failed just that it always keeps (also when not failed). Huh, that would go a long way. What would you think about that?