[PATCH 1/2] fix out of tree build for ruby bindings

[PATCH 1/2] fix out of tree build for ruby bindings

[Tomi Ollila]

From: Tomi Ollila <tomi.ollila@iki.fi>

Ruby bindings source files had to be copied to the out-of-tree
target directory -- final obstacle avoiding that was that if
there were built object files in source directory, new target object
files were not built (caused link failure, fortunately).
---

The ruby-part fix of id:1479146030-17083-1-git-send-email-tomi.ollila@iki.fi

 bindings/Makefile.local  |  2 +-
 bindings/ruby/extconf.rb | 12 +++++++++++-
 configure                |  6 ++++++
 3 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/bindings/Makefile.local b/bindings/Makefile.local
index 11d11d4..61fe412 100644
--- a/bindings/Makefile.local
+++ b/bindings/Makefile.local
@@ -8,7 +8,7 @@ ifeq ($(HAVE_RUBY_DEV),1)
 	cd $(dir)/ruby && \
 		EXTRA_LDFLAGS="$(NO_UNDEFINED_LDFLAGS)" \
 		LIBNOTMUCH="../../lib/$(LINKER_NAME)" \
-		ruby extconf.rb --vendor
+		SRCDIR=$(srcdir) ruby extconf.rb --vendor
 	$(MAKE) -C $(dir)/ruby
 endif
 
diff --git a/bindings/ruby/extconf.rb b/bindings/ruby/extconf.rb
index ddaa684..ed57d33 100644
--- a/bindings/ruby/extconf.rb
+++ b/bindings/ruby/extconf.rb
@@ -1,11 +1,21 @@
 #!/usr/bin/env ruby
 # coding: utf-8
 # Copyright 2010, 2011, 2012 Ali Polatel <alip@exherbo.org>
+# Out of tree build update: 2016 Tomi Ollila
 # Distributed under the terms of the GNU General Public License v3
 
 require 'mkmf'
 
-dir = File.join('..', '..', 'lib')
+# ENV['SRCDIR'] is build system's vision of source dir...
+srcroot = ENV['SRCDIR']
+# ...so fix it; expect hardcoded ../.. to work
+if srcroot == '.'
+  srcroot = File.join('..', '..')
+elsif srcroot[0] != '/'
+  srcroot = File.join('..', '..', srcroot)
+end
+
+dir = File.join(srcroot, 'lib')
 
 # includes
 $INCFLAGS = "-I#{dir} #{$INCFLAGS}"
diff --git a/configure b/configure
index f0fc466..22767c5 100755
--- a/configure
+++ b/configure
@@ -42,6 +42,12 @@ if [ "$srcdir" != "." ]; then
     # Emacs only likes to generate compiled files next to the .el files
     # by default so copy these as well (which is not ideal).
     cp -a "$srcdir"/emacs/*.el emacs
+
+    # We were not able to create fully working Makefile using ruby mkmf.rb
+    # so ruby bindings source files are copied as well (ditto -- not ideal).
+    mkdir bindings/ruby
+    cp -a "$srcdir"/bindings/ruby/*.[ch] bindings/ruby
+    cp -a "$srcdir"/bindings/ruby/extconf.rb bindings/ruby
 fi
 
 # Set several defaults (optionally specified by the user in
-- 
2.10.0

[PATCH 2/2] fix (rest of the) out of tree build

[Tomi Ollila]

From: Tomi Ollila <tomi.ollila@iki.fi>

With working tests (on various Linux systems).
---

V3 of id:1479146030-17083-1-git-send-email-tomi.ollila@iki.fi

 configure               | 6 ++++--
 test/test-lib-common.sh | 7 +++++++
 test/test-lib.sh        | 4 ++--
 3 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/configure b/configure
index 22767c5..03dddce 100755
--- a/configure
+++ b/configure
@@ -348,8 +348,8 @@ int main(void) {
     return 0;
 }
 EOF
-if ${CC} ${CFLAGS} _libversion.c -o _libversion > /dev/null 2>&1 && \
-       ./_libversion > _libversion.sh && . ./_libversion.sh
+if ${CC} ${CFLAGS} -I"$srcdir" _libversion.c -o _libversion > /dev/null 2>&1 \
+       && ./_libversion > _libversion.sh && . ./_libversion.sh
 then
     printf "OK.\n"
 else
@@ -1165,6 +1165,8 @@ cat > sh.config <<EOF
 # This sh.config was automatically generated by the ./configure
 # script of notmuch.
 
+NOTMUCH_SRCDIR='${srcdir}'
+
 # Whether the Xapian version in use supports compaction
 NOTMUCH_HAVE_XAPIAN_COMPACT=${have_xapian_compact}
 
diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index 03ef1d2..9b05b61 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -66,6 +66,13 @@ export LD_LIBRARY_PATH
 # configure output
 . $notmuch_path/sh.config || exit 1
 
+# Like $TEST_DIRECTORY this needs to be absolute as working directories change.
+SOURCE_DIRECTORY=$(
+	case $NOTMUCH_SRCDIR in /*) cd "$NOTMUCH_SRCDIR" ;;
+				 *) cd "../$NOTMUCH_SRCDIR"
+	esac
+	pwd -P)
+
 if test -n "$valgrind"
 then
 	make_symlink () {
diff --git a/test/test-lib.sh b/test/test-lib.sh
index a12c6d0..a7a2664 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -1205,7 +1205,7 @@ test_emacs () {
 test_python() {
     # Note: if there is need to print debug information from python program,
     # use stdout = os.fdopen(6, 'w') or stderr = os.fdopen(7, 'w')
-    PYTHONPATH="$TEST_DIRECTORY/../bindings/python${PYTHONPATH:+:$PYTHONPATH}" \
+    PYTHONPATH="$SOURCE_DIRECTORY/bindings/python${PYTHONPATH:+:$PYTHONPATH}" \
 	$NOTMUCH_PYTHON -B - > OUTPUT
 }
 
@@ -1217,7 +1217,7 @@ test_C () {
     exec_file="test${test_count}"
     test_file="${exec_file}.c"
     cat > ${test_file}
-    ${TEST_CC} ${TEST_CFLAGS} -I${TEST_DIRECTORY} -I${TEST_DIRECTORY}/../lib -o ${exec_file} ${test_file} -L${TEST_DIRECTORY}/../lib/ -lnotmuch -ltalloc
+    ${TEST_CC} ${TEST_CFLAGS} -I${TEST_DIRECTORY} -I${SOURCE_DIRECTORY}/lib -o ${exec_file} ${test_file} -L${TEST_DIRECTORY}/../lib/ -lnotmuch -ltalloc
     echo "== stdout ==" > OUTPUT.stdout
     echo "== stderr ==" > OUTPUT.stderr
     ./${exec_file} "$@" 1>>OUTPUT.stdout 2>>OUTPUT.stderr
-- 
2.10.0

Re: [PATCH 2/2] fix (rest of the) out of tree build

[David Bremner]

Tomi Ollila <too@guru.guru-group.fi> writes:


> +# Like $TEST_DIRECTORY this needs to be absolute as working directories change.
> +SOURCE_DIRECTORY=$(
> +	case $NOTMUCH_SRCDIR in /*) cd "$NOTMUCH_SRCDIR" ;;
> +				 *) cd "../$NOTMUCH_SRCDIR"
> +	esac
> +	pwd -P)
> +

I don't understand the second case here.

Re: [PATCH 1/2] fix out of tree build for ruby bindings

[David Bremner]

Tomi Ollila <too@guru.guru-group.fi> writes:

> From: Tomi Ollila <tomi.ollila@iki.fi>
>
> Ruby bindings source files had to be copied to the out-of-tree
> target directory -- final obstacle avoiding that was that if
> there were built object files in source directory, new target object
> files were not built (caused link failure, fortunately).
> ---
>
> The ruby-part fix of id:1479146030-17083-1-git-send-email-tomi.ollila@iki.fi
>
>  bindings/Makefile.local  |  2 +-
>  bindings/ruby/extconf.rb | 12 +++++++++++-
>  configure                |  6 ++++++
>  3 files changed, 18 insertions(+), 2 deletions(-)
>
> diff --git a/bindings/Makefile.local b/bindings/Makefile.local
> index 11d11d4..61fe412 100644
> --- a/bindings/Makefile.local
> +++ b/bindings/Makefile.local
> @@ -8,7 +8,7 @@ ifeq ($(HAVE_RUBY_DEV),1)
>  	cd $(dir)/ruby && \
>  		EXTRA_LDFLAGS="$(NO_UNDEFINED_LDFLAGS)" \
>  		LIBNOTMUCH="../../lib/$(LINKER_NAME)" \
> -		ruby extconf.rb --vendor
> +		SRCDIR=$(srcdir) ruby extconf.rb --vendor
>  	$(MAKE) -C $(dir)/ruby
>  endif
>  
> diff --git a/bindings/ruby/extconf.rb b/bindings/ruby/extconf.rb
> index ddaa684..ed57d33 100644
> --- a/bindings/ruby/extconf.rb
> +++ b/bindings/ruby/extconf.rb
> @@ -1,11 +1,21 @@
>  #!/usr/bin/env ruby
>  # coding: utf-8
>  # Copyright 2010, 2011, 2012 Ali Polatel <alip@exherbo.org>
> +# Out of tree build update: 2016 Tomi Ollila
>  # Distributed under the terms of the GNU General Public License v3
>  
>  require 'mkmf'
>  
> -dir = File.join('..', '..', 'lib')
> +# ENV['SRCDIR'] is build system's vision of source dir...
> +srcroot = ENV['SRCDIR']
> +# ...so fix it; expect hardcoded ../.. to work
> +if srcroot == '.'
> +  srcroot = File.join('..', '..')
> +elsif srcroot[0] != '/'
> +  srcroot = File.join('..', '..', srcroot)
> +end

I guess I have the same question about the non-absolute path case
here. Are we assuming some convention?

Re: [PATCH 1/2] fix out of tree build for ruby bindings

[Tomi Ollila]

On Tue, Nov 29 2016, David Bremner <david@tethera.net> wrote:

> Tomi Ollila <too@guru.guru-group.fi> writes:
>
>> From: Tomi Ollila <tomi.ollila@iki.fi>
>>
>> Ruby bindings source files had to be copied to the out-of-tree
>> target directory -- final obstacle avoiding that was that if
>> there were built object files in source directory, new target object
>> files were not built (caused link failure, fortunately).
>> ---
>>
>> The ruby-part fix of id:1479146030-17083-1-git-send-email-tomi.ollila@iki.fi
>>
>>  bindings/Makefile.local  |  2 +-
>>  bindings/ruby/extconf.rb | 12 +++++++++++-
>>  configure                |  6 ++++++
>>  3 files changed, 18 insertions(+), 2 deletions(-)
>>
>> diff --git a/bindings/Makefile.local b/bindings/Makefile.local
>> index 11d11d4..61fe412 100644
>> --- a/bindings/Makefile.local
>> +++ b/bindings/Makefile.local
>> @@ -8,7 +8,7 @@ ifeq ($(HAVE_RUBY_DEV),1)
>>  	cd $(dir)/ruby && \
>>  		EXTRA_LDFLAGS="$(NO_UNDEFINED_LDFLAGS)" \
>>  		LIBNOTMUCH="../../lib/$(LINKER_NAME)" \
>> -		ruby extconf.rb --vendor
>> +		SRCDIR=$(srcdir) ruby extconf.rb --vendor
>>  	$(MAKE) -C $(dir)/ruby
>>  endif
>>  
>> diff --git a/bindings/ruby/extconf.rb b/bindings/ruby/extconf.rb
>> index ddaa684..ed57d33 100644
>> --- a/bindings/ruby/extconf.rb
>> +++ b/bindings/ruby/extconf.rb
>> @@ -1,11 +1,21 @@
>>  #!/usr/bin/env ruby
>>  # coding: utf-8
>>  # Copyright 2010, 2011, 2012 Ali Polatel <alip@exherbo.org>
>> +# Out of tree build update: 2016 Tomi Ollila
>>  # Distributed under the terms of the GNU General Public License v3
>>  
>>  require 'mkmf'
>>  
>> -dir = File.join('..', '..', 'lib')
>> +# ENV['SRCDIR'] is build system's vision of source dir...
>> +srcroot = ENV['SRCDIR']
>> +# ...so fix it; expect hardcoded ../.. to work
>> +if srcroot == '.'
>> +  srcroot = File.join('..', '..')
>> +elsif srcroot[0] != '/'
>> +  srcroot = File.join('..', '..', srcroot)
>> +end
>
> I guess I have the same question about the non-absolute path case
> here. Are we assuming some convention?


The case here is that $srcdir can be either relative or absolute.
In normat in-tree build it (usually is '.') (./configure is executed,
but in case of in-tree build also $PWD/configure could also be executed).

In out-of-tree build $srcdir can be e.g. ../notmuch (../notmuch/configure
was executed) or absolute /path/to/notmuch.

In case of $TEST_DIRECTORY. When $srcdir is relative, the real it is to be
referenced from $TEST_DIRECTORY/.. (as current dir when tests are executed
is (path/to/)notmuch/test ) is $srcdir is absolute, then that is the source
directory (and prefixing that with ../* is just plain wrong ;) (in 1st patch
I used cd and then pwd -P to resolve real physical path, to be in
consistency with TEST_DIRECTORY=$(pwd -P) (*)) 

In case of ruby bindings (notmuch/ruby/bindings), same story, just ../..
(and no physical path resolvation).

There are now 2 alternatives

1) write srcdir as absolute path (only or additionally) to the *.config
files. this would make these things simpler. Here the hard problem is off^H^H^H
name the variable...

2) I write some more comments to the code -- it took a second or two for me
to re-orient this change so I could write this reply.


Tomi



(*) here it does not matter, but in generic case this pwd -P would not be
such a good idea, if TEST_DIRECTORY were a symlink in notmuch directory,
the parent dir of $TEST_DIRECTORY could point elsewhere. But also note that
if x=$(pwd) and y=$(pwd -P) of any directory differs, then also "$x/.." may
point to different location than  "${x%/*}" -- I got hit by this couple of
weeks ago... 

Re: [PATCH 1/2] fix out of tree build for ruby bindings

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

>
> There are now 2 alternatives
>
> 1) write srcdir as absolute path (only or additionally) to the *.config
> files. this would make these things simpler. Here the hard problem is off^H^H^H
> name the variable...

I'm generally in favour of centralizing these kinds of calculations in
configure if possible.

d