Hwo to debug and teacking builds?

Hwo to debug and teacking builds?

[Hartmut Goebel]

Hi,

when defining a new package, I often find myself spending *a lot* of
time debugging and tweaking the build. E.g. the if the Makefile needs to
be modified, or some test-cases adjusted.

How do I "get into" the build container, so I can debug, modify files,
rebuild and run tests there - in a closed environment (nearly) like the
build-daemon has?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Hwo to debug and teacking builds?

[Ludovic Courtès]

Howdy!

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> when defining a new package, I often find myself spending *a lot* of
> time debugging and tweaking the build. E.g. the if the Makefile needs to
> be modified, or some test-cases adjusted.
>
> How do I "get into" the build container, so I can debug, modify files,
> rebuild and run tests there - in a closed environment (nearly) like the
> build-daemon has?

Most of the time, you don’t need to get into a container.  In those
cases, it’s enough to do:

  guix build -K foo
  # build fails…
  cd /tmp/guix-build-foo.drv-0
  source ./environment-variables
  cd foo-1.2
  …

In some cases (for example when you have tests that fail in the build
environment but succeed once you’ve followed the steps above), you
really need a container similar to that created by guix-daemon.  In that
case, do:

  guix build -K foo
  # build fails…
  cd /tmp/guix-build-foo.drv-0
  guix environment -C foo --ad-hoc strace gdb
  rm /bin/sh           # to be really like in the guix-daemon environment
  source ./environment-variables
  cd foo-1.2
  $GUIX_ENVIRONMENT/bin/strace -f -o log make check
  …

That would probably make a good “Debugging Build Failures” section.

Thoughts?

Ludo’.

Re: Hwo to debug and teacking builds?

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 1668 bytes --]

2017-03-28 11:52 GMT+02:00 Ludovic Courtès <ludo@gnu.org>:

> Howdy!
>
> Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:
>
> > when defining a new package, I often find myself spending *a lot* of
> > time debugging and tweaking the build. E.g. the if the Makefile needs to
> > be modified, or some test-cases adjusted.
> >
> > How do I "get into" the build container, so I can debug, modify files,
> > rebuild and run tests there - in a closed environment (nearly) like the
> > build-daemon has?
>
> Most of the time, you don’t need to get into a container.  In those
> cases, it’s enough to do:
>
>   guix build -K foo
>   # build fails…
>   cd /tmp/guix-build-foo.drv-0
>   source ./environment-variables
>   cd foo-1.2
>   …
>
> In some cases (for example when you have tests that fail in the build
> environment but succeed once you’ve followed the steps above), you
> really need a container similar to that created by guix-daemon.  In that
> case, do:
>
>   guix build -K foo
>   # build fails…
>   cd /tmp/guix-build-foo.drv-0
>   guix environment -C foo --ad-hoc strace gdb
>   rm /bin/sh           # to be really like in the guix-daemon environment
>   source ./environment-variables
>   cd foo-1.2
>   $GUIX_ENVIRONMENT/bin/strace -f -o log make check
>   …
>
> That would probably make a good “Debugging Build Failures” section.
>
>
>
This is important information and it's more detailed than what Pjotr wrote
in his wrap up

I'd do it myself but I don't know the first thing about the info system

Which is the source file ?
How do I compile it ?
Which Emacs mode should I use ?

[-- Attachment #2: Type: text/html, Size: 2266 bytes --]

Re: Hwo to debug and teacking builds?

[ng0]

Catonano transcribed 4.3K bytes:
> 2017-03-28 11:52 GMT+02:00 Ludovic Courtès <ludo@gnu.org>:
> 
> > Howdy!
> >
> > Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:
> >
> > > when defining a new package, I often find myself spending *a lot* of
> > > time debugging and tweaking the build. E.g. the if the Makefile needs to
> > > be modified, or some test-cases adjusted.
> > >
> > > How do I "get into" the build container, so I can debug, modify files,
> > > rebuild and run tests there - in a closed environment (nearly) like the
> > > build-daemon has?
> >
> > Most of the time, you don’t need to get into a container.  In those
> > cases, it’s enough to do:
> >
> >   guix build -K foo
> >   # build fails…
> >   cd /tmp/guix-build-foo.drv-0
> >   source ./environment-variables
> >   cd foo-1.2
> >   …
> >
> > In some cases (for example when you have tests that fail in the build
> > environment but succeed once you’ve followed the steps above), you
> > really need a container similar to that created by guix-daemon.  In that
> > case, do:
> >
> >   guix build -K foo
> >   # build fails…
> >   cd /tmp/guix-build-foo.drv-0
> >   guix environment -C foo --ad-hoc strace gdb
> >   rm /bin/sh           # to be really like in the guix-daemon environment
> >   source ./environment-variables
> >   cd foo-1.2
> >   $GUIX_ENVIRONMENT/bin/strace -f -o log make check
> >   …
> >
> > That would probably make a good “Debugging Build Failures” section.
> >
> >
> >
> This is important information and it's more detailed than what Pjotr wrote
> in his wrap up
> 
> I'd do it myself but I don't know the first thing about the info system
> 
> Which is the source file ?

doc/guix.texi

> How do I compile it ?

I never got anything else than html to build with make of guix, (make
doc/guix.html).. it's not super obvious. I know how texi2pdf etc works,
but this should at least be listed somewhere other than the Makefile.

> Which Emacs mode should I use ?

Re: Hwo to debug and teacking builds?

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]

2017-03-28 18:07 GMT+02:00 ng0 <contact.ng0@cryptolab.net>:

> Catonano transcribed 4.3K bytes:
>
>
>
> > Which is the source file ?
>
> doc/guix.texi
>
>
Thank you.

I'd like to ask something about Emacs. A bit off topic but not that much.

I remember seeing an Emacs package that displays a preview of the whole
document (buffer ?) in a small region of the window (frame ?)

So that you can know which portion of the document is currently displayed

Gedit, like many editors, does this, it's showed here (the small square on
the right)
ctrlv.in/938108

Too bad that I can't remember how it was called and I couldn't find
anything related on line

If anyone has a hint, I'd appreciate that

It'd be helpful in browsing the guix.texi file that's quite large.

> How do I compile it ?
>
> I never got anything else than html to build with make of guix, (make
> doc/guix.html)..
>
it's not super obvious. I know how texi2pdf etc works,
> but this should at least be listed somewhere other than the Makefile.
>

It definitely should be. I had no idea.
And I wouldn't know what to look for in the Makefile

Thanks again

[-- Attachment #2: Type: text/html, Size: 2230 bytes --]

Re: Hwo to debug and teacking builds?

[ng0]

Actually you've sent this to the list and not off-list. So all is okay

I wouldn't know how to help with the emacs question.

I'm currently fixing lots of drupal to texinfo exports and after 70.000+
lines you just stop wanting a preview for every change.


Catonano transcribed 3.6K bytes:
> 2017-03-28 18:07 GMT+02:00 ng0 <contact.ng0@cryptolab.net>:
> 
> > Catonano transcribed 4.3K bytes:
> >
> >
> >
> > > Which is the source file ?
> >
> > doc/guix.texi
> >
> >
> Thank you.
> 
> I'd like to ask something about Emacs. A bit off topic but not that much.
> 
> I remember seeing an Emacs package that displays a preview of the whole
> document (buffer ?) in a small region of the window (frame ?)
> 
> So that you can know which portion of the document is currently displayed
> 
> Gedit, like many editors, does this, it's showed here (the small square on
> the right)
> ctrlv.in/938108
> 
> Too bad that I can't remember how it was called and I couldn't find
> anything related on line
> 
> If anyone has a hint, I'd appreciate that
> 
> It'd be helpful in browsing the guix.texi file that's quite large.
> 
> > How do I compile it ?
> >
> > I never got anything else than html to build with make of guix, (make
> > doc/guix.html)..
> >
> it's not super obvious. I know how texi2pdf etc works,
> > but this should at least be listed somewhere other than the Makefile.
> >
> 
> It definitely should be. I had no idea.
> And I wouldn't know what to look for in the Makefile
> 
> Thanks again

Re: Hwo to debug and teacking builds?

[Ricardo Wurmus]

Catonano <catonano@gmail.com> writes:

> I remember seeing an Emacs package that displays a preview of the whole
> document (buffer ?) in a small region of the window (frame ?)
>
> So that you can know which portion of the document is currently displayed
>
> Gedit, like many editors, does this, it's showed here (the small square on
> the right)
> ctrlv.in/938108

I cannot see the picture, but I think you mean a feature provided by
both “minimap” and “sublimity”.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Hwo to debug and teacking builds?

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 264 bytes --]

2017-04-10 8:39 GMT+02:00 Ricardo Wurmus <rekado@elephly.net>:


> I cannot see the picture, but I think you mean a feature provided by
> both “minimap” and “sublimity”.
>

Yes, the one I saw is Sublimity. And I didn't know about minimap
Thanks !

[-- Attachment #2: Type: text/html, Size: 626 bytes --]

Re: Hwo to debug and teacking builds?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1433 bytes --]

Catonano <catonano@gmail.com> writes:

> This is important information and it's more detailed than what Pjotr wrote in his wrap up 
>
> I'd do it myself but I don't know the first thing about the info system
>
> Which is the source file ? 

I'm sure your contribution would be welcome!  The source is in guix/doc,
mostly in guix.texi.

> How do I compile it ?

Get the dependencies:

  guix environment guix --ad-hoc texlive

Make everything:

  make

Make just the documentation, in Info format:

  make info

If you're curious to understand why these targets exist, please refer to
the Automake manual for details:

https://www.gnu.org/software/automake/manual/automake.html#Documentation

After making changes to the guix.texi file, just re-run the 'make'
invocation, and it will quickly recompile just the documentation.

To view your changes, it's helpful to add the 'guix/doc' directory from
your local Guix checkout to your INFOPATH.  I've put the following line
in my ~/.bash_profile to do this automatically:

  export INFOPATH="$HOME/guix/doc${INFOPATH:+:}$INFOPATH"

That way, when you invoke 'guix info' (or 'C-h i' in Emacs), you'll see
the updated version.  I don't know how to preview changes in real-time
as you modify guix.texi, but if you learn how, please let me know!

> Which Emacs mode should I use ?

Try texinfo-mode (see: (texinfo) Emacs Editing).

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Hwo to debug and teacking builds?

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 483 bytes --]

2017-03-29 8:10 GMT+02:00 Chris Marusich <cmmarusich@gmail.com>:

> Catonano <catonano@gmail.com> writes:
>
> > This is important information and it's more detailed than what Pjotr
> wrote in his wrap up
> >
> > I'd do it myself but I don't know the first thing about the info system
> >
> > Which is the source file ?
>
> I'm sure your contribution would be welcome!


Thank you Chris. Very much appreciated :-)
https://lists.gnu.org/archive/html/guix-patches/2017-03/msg00858.html

[-- Attachment #2: Type: text/html, Size: 1021 bytes --]

Re: Hwo to debug and teacking builds?

[myglc2]

On 03/28/2017 at 23:10 Chris Marusich writes:

[...]

> To view your changes, it's helpful to add the 'guix/doc' directory from
> your local Guix checkout to your INFOPATH.  I've put the following line
> in my ~/.bash_profile to do this automatically:
>
>   export INFOPATH="$HOME/guix/doc${INFOPATH:+:}$INFOPATH"

Hi Chris, don't you mean ...

export INFOPATH="$HOME/guix/doc${INFOPATH:+:$INFOPATH}"

- George

Re: Hwo to debug and teacking builds?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

myglc2 <myglc2@gmail.com> writes:

> On 03/28/2017 at 23:10 Chris Marusich writes:
>
> [...]
>
>> To view your changes, it's helpful to add the 'guix/doc' directory from
>> your local Guix checkout to your INFOPATH.  I've put the following line
>> in my ~/.bash_profile to do this automatically:
>>
>>   export INFOPATH="$HOME/guix/doc${INFOPATH:+:}$INFOPATH"
>
> Hi Chris, don't you mean ...
>
> export INFOPATH="$HOME/guix/doc${INFOPATH:+:$INFOPATH}"
>
> - George

Yes, you're correct.  I mis-typed that.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Hwo to debug and teacking builds?

[Hartmut Goebel]

Am 11.04.2017 um 17:40 schrieb Chris Marusich:
> myglc2 <myglc2@gmail.com> writes:
>
>> On 03/28/2017 at 23:10 Chris Marusich writes:
>>
>> [...]
>>
>>>   export INFOPATH="$HOME/guix/doc${INFOPATH:+:}$INFOPATH"
>> export INFOPATH="$HOME/guix/doc${INFOPATH:+:$INFOPATH}"
> Yes, you're correct.  I mis-typed that.

But isn't the result the same?


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Hwo to debug and teacking builds?

[myglc2]

On 04/11/2017 at 18:36 Hartmut Goebel writes:

> Am 11.04.2017 um 17:40 schrieb Chris Marusich:
>> myglc2 <myglc2@gmail.com> writes:
>>
>>> On 03/28/2017 at 23:10 Chris Marusich writes:
>>>
>>> [...]
>>>
>>>>   export INFOPATH="$HOME/guix/doc${INFOPATH:+:}$INFOPATH"
>>> export INFOPATH="$HOME/guix/doc${INFOPATH:+:$INFOPATH}"
>> Yes, you're correct.  I mis-typed that.
>
> But isn't the result the same?

Duh... Yes. 

Re: Hwo to debug and teacking builds?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 11.04.2017 um 17:40 schrieb Chris Marusich:
>> myglc2 <myglc2@gmail.com> writes:
>>
>>> On 03/28/2017 at 23:10 Chris Marusich writes:
>>>
>>> [...]
>>>
>>>>   export INFOPATH="$HOME/guix/doc${INFOPATH:+:}$INFOPATH"
>>> export INFOPATH="$HOME/guix/doc${INFOPATH:+:$INFOPATH}"
>> Yes, you're correct.  I mis-typed that.
>
> But isn't the result the same?

Maybe?  I typed this manually, and I meant to type what George
mentioned.  If what I typed was correct, then it was correct on
accident.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Hwo to debug and teacking builds?

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 748 bytes --]

Hello,

thank you very much for these snippets. I'm a lazy guy :-) so I wrapped
this into a script, which starts a new bash with this environment (see
attachment). Maybe this could be enhanced using guix mechanisms?
Suggestions are welcome.

Usage:

./pkg-env  # go into latest failed build - very convenient :-)
./pkg-env  some-package # go into latest failed build of this package
./pkg-env /tmp/guix-build-some-package-1.4.01.drv-1 # go into this directory

> That would probably make a good “Debugging Build Failures” section.

Absolutely!

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: pkg-env --]
[-- Type: text/plain, Size: 1059 bytes --]

#!/bin/bash
# Copyright © 2017 Hartmut Goebel <h.goebel@crazy-compilers.com>
# License: GNU Public License v3.0 or (at your choice) later

PACKAGE="${1}"
if [ -z "${PACKAGE}" ] ; then
    PACKAGE=$(ls -td /var/tmp/guix-build-*.drv-* /tmp/guix-build-"$PACKAGE"-* 2>/dev/null | head -1)
    echo >&2 "No package name given, using $PACKAGE"
elif [ "${PACKAGE#/var/tmp/}" = "$PACKAGE" ] ; then
    PACKAGE=$(ls -td /var/tmp/guix-build-"$PACKAGE"-* /tmp/guix-build-"$PACKAGE"-* 2>/dev/null | head -1)
fi

if [ ! -d "$PACKAGE" ] ; then
    echo >&2 "Not found: $PACKAGE"
    exit 10
fi

#echo "Using $PACKAGE"

# TODO: Support containers:
#  guix environment -C foo --ad-hoc strace gdb
#  rm /bin/sh           # to be really like in the guix-daemon environment

inifile=$(mktemp)
cat > $inifile <<EOF
cd $PACKAGE
source $PACKAGE/environment-variables 
set +x
cd \$(find . -maxdepth 1 -mindepth 1 -type d 2>/dev/null) .
echo \$PWD
EOF

#echo '------------' ; cat $inifile ; echo '------------'
env -i HOME=/tmp bash --noprofile --init-file $inifile

rm -f $inifile