Testing isolated builds

Testing isolated builds

[Pjotr Prins]

I am facing some issues try to upgrade Elixir. The problem is that the
build fails on tests, but when I build by hand inside
/tmp/guix-build-elixir-1.5.1.drv-0/ the build passes.

Is there an easy way to create a similarly restricted environment when
doing a build outside Guix? I.e., no $HOME, no /tmp access, no
network? Any hints on an approach?

Pj.

Re: Testing isolated builds

[Pjotr Prins]

On Wed, Sep 20, 2017 at 02:18:01PM +0200, Pjotr Prins wrote:
> I am facing some issues try to upgrade Elixir. The problem is that the
> build fails on tests, but when I build by hand inside
> /tmp/guix-build-elixir-1.5.1.drv-0/ the build passes.
> 
> Is there an easy way to create a similarly restricted environment when
> doing a build outside Guix? I.e., no $HOME, no /tmp access, no
> network? Any hints on an approach?

When I build with Guix I get an error 

       test/elixir/kernel/dialyzer_test.exs:95
       ** (File.CopyError) could not copy from "/tmp/guix-build-elixir-1.5.1.drv-0/elixir-1.5.1/lib/elixir/tmp/dialyzer/Elixir.Dialyzer.ForBooleanCheck.beam" to "/tmp/guix-build-elixir-1.5.1.drv-0/elixir-1.5.1/lib/elixir/tmp/dialyzer/line95/Elixir.Dialyzer.ForBooleanCheck.beam": no such file or directory
       code: copy_beam! context, Dialyzer.ForBooleanCheck
       stacktrace:
         (elixir) lib/file.ex:631: File.cp!/3
         test/elixir/kernel/dialyzer_test.exs:96: (test)


When I try:

  ./pre-inst-env guix environment erlang -C --ad-hoc erlang vim make \
    glibc-utf8-locales git --pure \
    --share=/tmp/guix-build-elixir-1.5.1.drv-12

That gives me a clean environment, but now all effing tests pass!

The Guix build environment does not create
/tmp/guix-build-elixir-1.5.1.drv-0/elixir-1.5.1/lib/elixir/tmp for some reason.
Because when then test fails it does not exist. The Elixir.Dialyzer.ForBooleanCheck.beam is never created.

It is weird.

Could it be file permissions? I think mkdir_p would throw a descriptive error if it fails. 

If no one has an idea here I'll try and plug in informative errors. But it is a
royal pain I can't reproduce the error in guix environment.

Pj.
-- 

Re: Testing isolated builds

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 457 bytes --]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> I am facing some issues try to upgrade Elixir. The problem is that the
> build fails on tests, but when I build by hand inside
> /tmp/guix-build-elixir-1.5.1.drv-0/ the build passes.
>
> Is there an easy way to create a similarly restricted environment when
> doing a build outside Guix? I.e., no $HOME, no /tmp access, no
> network? Any hints on an approach?

Maybe `guix environment --container elixir`?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Testing isolated builds

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2004 bytes --]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On Wed, Sep 20, 2017 at 02:18:01PM +0200, Pjotr Prins wrote:
>> I am facing some issues try to upgrade Elixir. The problem is that the
>> build fails on tests, but when I build by hand inside
>> /tmp/guix-build-elixir-1.5.1.drv-0/ the build passes.
>> 
>> Is there an easy way to create a similarly restricted environment when
>> doing a build outside Guix? I.e., no $HOME, no /tmp access, no
>> network? Any hints on an approach?
>
> When I build with Guix I get an error 
>
>        test/elixir/kernel/dialyzer_test.exs:95
>        ** (File.CopyError) could not copy from "/tmp/guix-build-elixir-1.5.1.drv-0/elixir-1.5.1/lib/elixir/tmp/dialyzer/Elixir.Dialyzer.ForBooleanCheck.beam" to "/tmp/guix-build-elixir-1.5.1.drv-0/elixir-1.5.1/lib/elixir/tmp/dialyzer/line95/Elixir.Dialyzer.ForBooleanCheck.beam": no such file or directory
>        code: copy_beam! context, Dialyzer.ForBooleanCheck
>        stacktrace:
>          (elixir) lib/file.ex:631: File.cp!/3
>          test/elixir/kernel/dialyzer_test.exs:96: (test)
>
>
> When I try:
>
>   ./pre-inst-env guix environment erlang -C --ad-hoc erlang vim make \
>     glibc-utf8-locales git --pure \
>     --share=/tmp/guix-build-elixir-1.5.1.drv-12
>
> That gives me a clean environment, but now all effing tests pass!
>
> The Guix build environment does not create
> /tmp/guix-build-elixir-1.5.1.drv-0/elixir-1.5.1/lib/elixir/tmp for some reason.
> Because when then test fails it does not exist. The Elixir.Dialyzer.ForBooleanCheck.beam is never created.
>
> It is weird.
>
> Could it be file permissions? I think mkdir_p would throw a descriptive error if it fails. 
>
> If no one has an idea here I'll try and plug in informative errors. But it is a
> royal pain I can't reproduce the error in guix environment.

IIRC the main difference between `guix environment -C` and the build
container is that /bin/sh is present in the former.  Maybe grep for it?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Testing isolated builds

[Hartmut Goebel]

Hi,

Am 20.09.2017 um 14:18 schrieb Pjotr Prins:
> I am facing some issues try to upgrade Elixir. The problem is that the
> build fails on tests, but when I build by hand inside
> /tmp/guix-build-elixir-1.5.1.drv-0/ the build passes.

I've been facing such issue, too. Enclosed please find the script I'm
using to get an isolated environment. The container is still beta.
Feedback is welcome. (I can set up a repo at github or gitlab if more
people are interested.)

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Testing isolated builds (now w/ attachment)

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 642 bytes --]

Hi,

Am 20.09.2017 um 14:18 schrieb Pjotr Prins:
> I am facing some issues try to upgrade Elixir. The problem is that the
> build fails on tests, but when I build by hand inside
> /tmp/guix-build-elixir-1.5.1.drv-0/ the build passes.

I've been facing such issue, too. Enclosed please find the script I'm
using to get an isolated environment. The container is still beta.
Feedback is welcome. (I can set up a repo at github or gitlab if more
people are interested.)

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: pkg-env --]
[-- Type: text/plain, Size: 2720 bytes --]

#!/bin/bash
# Copyright © 2017 Hartmut Goebel <h.goebel@crazy-compilers.com>
# License: GNU Public License v3.0 or (at your choice) later

ARGS=$(getopt -o "C" -l "pure,container" -- "$@")
if [ $? -ne 0 ] ; then exit 1 ; fi # Bad arguments
eval set -- "$ARGS"

USE_CONTAINER=
PURE_ENV=

CONTAINER_PACKAGES="strace"  # gdb

while true ; do
    case "$1" in
	-C|--container)
	    shift;
	    USE_CONTAINER="yes" ;;
	--pure)
	    shift;
	    PURE_ENV="--pure" ;;
	--) shift ; break ;;
    esac
done


function package_from_path () {
    local pkg name version
    pkg=$(basename "$1")
    pkg=${pkg#guix-build-}
    pkg=${pkg%.drv-*}
    version=${pkg##*-}
    name=${pkg%-*}
    echo $name"@"$version
}


function oldest_dir () {
    find "$1/" -maxdepth 1 -mindepth 1 -type d 2>/dev/null | \
	xargs ls -tdr | head -1
}

function get_build_dir () {
    BUILD_DIR="${1}"
    if [ -d "${BUILD_DIR}" ] ; then
	echo $BUILD_DIR
    elif [ -z "${BUILD_DIR}" ] ; then
	ls -td {,/var}/tmp/guix-build-*.drv-* 2>/dev/null | head -1
	echo >&2 "No package name given, using $BUILD_DIR"
    elif [ "${BUILD_DIR#{,/var}/tmp/}" = "$BUILD_DIR" ] ; then
	ls -td {,/var}/tmp/guix-build-"$BUILD_DIR"-* 2>/dev/null | head -1
    else
	ls -td {,/var}/tmp/"$BUILD_DIR" 2>/dev/null | head -1
    fi
}

BUILD_DIR=$(get_build_dir "$1")
if [ ! -d "$BUILD_DIR" ] ; then
    echo >&2 "Not found: $1"
    exit 10
fi

# hG special: create a symlink from /var/tmp to /tmp
if [ "${BUILD_DIR#/var/tmp/}" != "$BUILD_DIR" ] ; then
    ln -s "$BUILD_DIR" /tmp 2>/dev/null
    CONTAINER_OPTIONS="CONTAINER_OPTIONS --share=$BUILD_DIR=${BUILD_DIR#/var}"
    BUILD_DIR=${BUILD_DIR#/var}
    #BUILD_DIR=/tmp/$(basename $BUILD_DIR)   
fi
echo "Using $BUILD_DIR"

pkg=$(package_from_path "$BUILD_DIR")


inifile=$BUILD_DIR/.guix-env-inifile
cat > $inifile <<EOF
echo "----------------- Starting init-file --------------"
cd $BUILD_DIR
source $BUILD_DIR/environment-variables
#set -x
PS1="[$pkg \W]\$ "
export PS1
cd $(oldest_dir $BUILD_DIR)
echo \$PWD
EOF

if [ -n "$USE_CONTAINER" ] ; then
    cat >> $inifile <<EOF
if [ "${BUILD_DIR#/var/tmp/}" != "$BUILD_DIR" ] ; then
    ln -s "$BUILD_DIR" /tmp
fi
#rm /bin/sh
echo "You may no want to run e.g.:"
echo "strace -f -o log make check"
PATH=\$GUIX_ENVIRONMENT/bin:\$PATH
exec /bin/sh
EOF
fi
#echo '------------' ; cat $inifile ; echo '------------'

if [ -n "$USE_CONTAINER" ] ; then
    if [ -x "$PWD/pre-inst-env" ] ; then
	d="$PWD"
    else
	d=$(dirname "$0")
    fi
    cd $BUILD_DIR
    $d/pre-inst-env guix environment --fallback -C $pkg \
	--no-grafts $CONTAINER_OPTIONS \
	--ad-hoc $CONTAINER_PACKAGES # -- /bin/sh $inifile
else
    env -i HOME=/tmp bash --noprofile --init-file $inifile
fi
rm -f $inifile