[PATCH] Have to configure and build inside the source directory.

[PATCH] Have to configure and build inside the source directory.

[Thomas Schwinge]

From: Thomas Schwinge <thomas@schwinge.name>

Signed-off-by: Thomas Schwinge <thomas@schwinge.name>

---
 configure |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/configure b/configure
index c58dd0f..b5e446c 100755
--- a/configure
+++ b/configure
@@ -188,6 +188,17 @@ developers. Then, please email those details to the Notmuch list
 (notmuch@notmuchmail.org) so that we can hopefully make future
 versions of notmuch easier for you to use.
 
+EOF
+
+if ! { : < configure; } 2> /dev/null; then
+    cat <<EOF
+*** Error: You have to configure and build in the source directory.
+
+EOF
+    exit 1
+fi
+
+cat <<EOF
 We'll now investigate your system to verify that all required
 dependencies are available:
 
-- 
tg: (b3caef1..) srcdir (depends on: master)

Re: [PATCH] Have to configure and build inside the source directory.

[Michal Sojka]

On Tue, 11 Jan 2011, Thomas Schwinge wrote:
> From: Thomas Schwinge <thomas@schwinge.name>
> 
> Signed-off-by: Thomas Schwinge <thomas@schwinge.name>
> 
> ---
>  configure |   11 +++++++++++
>  1 files changed, 11 insertions(+), 0 deletions(-)
> 
> diff --git a/configure b/configure
> index c58dd0f..b5e446c 100755
> --- a/configure
> +++ b/configure
> @@ -188,6 +188,17 @@ developers. Then, please email those details to the Notmuch list
>  (notmuch@notmuchmail.org) so that we can hopefully make future
>  versions of notmuch easier for you to use.
>  
> +EOF
> +
> +if ! { : < configure; } 2> /dev/null; then

I think that "if ! test -f configure; then" be more readable.

-Michal

Re: [PATCH] Have to configure and build inside the source directory.

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 552 bytes --]

> +if ! { : < configure; } 2> /dev/null; then
> +    cat <<EOF
> +*** Error: You have to configure and build in the source directory.
> +
> +EOF
> +    exit 1
> +fi

Rather than documenting a limitation here, why don't we do what people
actually want.

What do other build systems generally do when running configure from
some other directory? Copy/link the Makefiles and then construct them
carefully such that they can find all the source files?

That doesn't sound like it would be that hard.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] Have to configure and build inside the source directory.

[Rob Browning]

Carl Worth <cworth@cworth.org> writes:

> What do other build systems generally do when running configure from
> some other directory? Copy/link the Makefiles and then construct them
> carefully such that they can find all the source files?

I think the use of GNU make VPATH is fairly common.  For example, I
believe the autotools build a tree that matches the srcdir structure and
add Makefiles that contain something like this:

  VPATH = ../../wherever/notmuch/thisdir
  ...

Of course, given that, the build tree doesn't include any source files.

FWIW
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

Build system (was: [PATCH] Have to configure and build inside the source directory.)

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 1040 bytes --]

Hallo!

On Sat, 29 Jan 2011 06:58:59 +1000, Carl Worth <cworth@cworth.org> wrote:
> Rather than documenting a limitation here, why don't we do what people
> actually want.

Well, not a big deal for me -- I'm just used to building projects
containing a configure file outside of the source tree, and the notmuch
build machinery at first accepted this, but then fell over.


> What do other build systems generally do when running configure from
> some other directory?

What Rob said -- but: why re-invent Autoconf / Automake if it's already
there?  I totally admit that the GNU Autotools have their ugly corners
(and indeed a lot of these), but on the other hand they do solve some
issues quite nicely.

You surely had reasons to not use these tools, and that's fine with me.
(And I'm not especially interested in working on build systems -- done
that enough in the past.)  Have you looked at other tools before going
for the straight-forward (and that is very fine!) Makefile-based
solution?


Grüße,
 Thomas

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: Build system

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 1589 bytes --]

Hallo!

On Sun, 30 Jan 2011 11:12:42 +0100, I <thomas@schwinge.name> wrote:
> On Sat, 29 Jan 2011 06:58:59 +1000, Carl Worth <cworth@cworth.org> wrote:
> > Rather than documenting a limitation here, why don't we do what people
> > actually want.
> 
> Well, not a big deal for me -- I'm just used to building projects
> containing a configure file outside of the source tree, and the notmuch
> build machinery at first accepted this, but then fell over.
> 
> 
> > What do other build systems generally do when running configure from
> > some other directory?
> 
> What Rob said -- but: why re-invent Autoconf / Automake if it's already
> there?  I totally admit that the GNU Autotools have their ugly corners
> (and indeed a lot of these), but on the other hand they do solve some
> issues quite nicely.
> 
> You surely had reasons to not use these tools, and that's fine with me.
> (And I'm not especially interested in working on build systems -- done
> that enough in the past.)  Have you looked at other tools before going
> for the straight-forward (and that is very fine!) Makefile-based
> solution?

OK -- I found the thread starting at
id:"1258897630-22282-1-git-send-email-jeff@ocjtech.us", where this has
been discussed already (as I should have expected).  ;-)

Still, my point holds that (unless someone is willing to spend time on
this, of course) we shouldn't try to replicate the Autotools, but instead
keep our system as simple as it currently is, and thus just have it fail
if configured outside of the source tree.


Grüße,
 Thomas

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: Build system

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 935 bytes --]

On Sun, 30 Jan 2011 11:28:08 +0100, Thomas Schwinge <thomas@schwinge.name> wrote:
> OK -- I found the thread starting at
> id:"1258897630-22282-1-git-send-email-jeff@ocjtech.us", where this has
> been discussed already (as I should have expected).  ;-)

OK. Then I won't go through those answers again.

> Still, my point holds that (unless someone is willing to spend time on
> this, of course) we shouldn't try to replicate the Autotools, but instead
> keep our system as simple as it currently is, and thus just have it fail
> if configured outside of the source tree.

Oh, I agree that if we don't support this then we should give the user a
nice error message. But I think it will actually be very easy to add
support for this. (And at this point, I think the notmuch build system
is something that other projects could emulate if they want. I don't
think it's too crazt).

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Build system

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 1813 bytes --]

Hallo!

On Sun, 30 Jan 2011 21:14:39 +1000, Carl Worth <cworth@cworth.org> wrote:
> On Sun, 30 Jan 2011 11:28:08 +0100, Thomas Schwinge <thomas@schwinge.name> wrote:
> > Still, my point holds that (unless someone is willing to spend time on
> > this, of course) we shouldn't try to replicate the Autotools, but instead
> > keep our system as simple as it currently is, and thus just have it fail
> > if configured outside of the source tree.
> 
> Oh, I agree that if we don't support this then we should give the user a
> nice error message. But I think it will actually be very easy to add
> support for this. (And at this point, I think the notmuch build system
> is something that other projects could emulate if they want. I don't
> think it's too crazt).

Is the testsuite also easy to convert to VPATH style builds?  (I don't
know, but would expect some difficulties.)


And, another thing I just noticed: I had the source tree configured with
--prefix=[something].  Now I updated the sources, re-ran make, and saw
this:

    $ make
    
    Note: Calling ./configure with no command-line arguments. This is often fine,
          but if you want to specify any arguments (such as an alternate prefix
          into which to install), call ./configure explicitly and then make again.
          See "./configure --help" for more details.
    
    ./configure
    Welcome to Notmuch, a system for indexing, searching and tagging your email.
    [...]

My --prefix=[something] is gone.  (At least the build system warns about
this.)  But it's another issue that other build systems already have
solved.  (And, I probably wouldn't have expected that to do the right
thing if the configure script / build system wouldn't try to be like GNU
Autoconf.)


Grüße,
 Thomas

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: Build system

[Michal Sojka]

On Tue, 08 Feb 2011, Thomas Schwinge wrote:
> On Sun, 30 Jan 2011 21:14:39 +1000, Carl Worth <cworth@cworth.org> wrote:
> > On Sun, 30 Jan 2011 11:28:08 +0100, Thomas Schwinge <thomas@schwinge.name> wrote:
> > > Still, my point holds that (unless someone is willing to spend time on
> > > this, of course) we shouldn't try to replicate the Autotools, but instead
> > > keep our system as simple as it currently is, and thus just have it fail
> > > if configured outside of the source tree.
> > 
> > Oh, I agree that if we don't support this then we should give the user a
> > nice error message. But I think it will actually be very easy to add
> > support for this. (And at this point, I think the notmuch build system
> > is something that other projects could emulate if they want. I don't
> > think it's too crazt).
> 
> Is the testsuite also easy to convert to VPATH style builds?  (I don't
> know, but would expect some difficulties.)

It won't be easy to run the tests in different directory, but the
following approach might be sufficient: When notmuch is configured and
compiled in a different directory, then running `make test` there could
run the tests in the original directory but the tests will invoke the
notmuch binary from where it was compiled.

If you send the patch for out-of-source compilation, I'll modify the
test suite to work with it.

-Michal

Re: [PATCH] Have to configure and build inside the source directory.

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1598 bytes --]

On Fri, 28 Jan 2011 17:18:16 -0600, Rob Browning <rlb@defaultvalue.org> wrote:
> I think the use of GNU make VPATH is fairly common.  For example, I
> believe the autotools build a tree that matches the srcdir structure and
> add Makefiles that contain something like this:
> 
>   VPATH = ../../wherever/notmuch/thisdir
>   ...
> 
> Of course, given that, the build tree doesn't include any source
> files.

Thanks for the pointer to VPATH. I had some code half-written to use
this sitting around since we last talked about it, and I've just cleaned
it up and pushed it out now.

It turns out that VPATH itself isn't very useful, (since it matches on
targets as well as prerequisites), but the similar "vpath" directive
matches only on prerequisites so it does what we want here.

So non-source-directory builds of notmuch should work now. Along the
lines of:

	git clone git.notmuchmail.org:/git/notmuch
        cd notmuch      
	mkdir build
        cd build
        ../configure
        make

Everyone, please feel free to test this and report back if you see any
problems.

We should also fix the "make release" target to exercise this code. Does
anyone know what else a GNU configure script typically does in "make
distcheck". I recall that it untars the built tar file and does a
non-srcdir build of it, (and runs the test suite). Then I think it also
does an install and perhaps checks that no extra files are leftover
after "make clean"? Something like that anyway---and we should probably
do something similar.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Build system

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1974 bytes --]

On Tue, 08 Feb 2011 09:55:57 +0100, Thomas Schwinge <thomas@schwinge.name> wrote:
> Is the testsuite also easy to convert to VPATH style builds?  (I don't
> know, but would expect some difficulties.)

In what I just pushed I rather punted on this issue. I made the
configure script simply copy the entire test suite into the build
directory. (I also did the same for .el files since emacs really wants
to put the compiled .elc file right next to the .el file).

This isn't ideal since it means that a non-srcdir build won't get access
to updated emacs nor test-suite bits without re-running configure. If
someone would like to do work to fix either or both of these cases, that
would be helpful.

> And, another thing I just noticed: I had the source tree configured with
> --prefix=[something].  Now I updated the sources, re-ran make, and saw
> this:
> 
>     $ make
>     
>     Note: Calling ./configure with no command-line arguments.

Thanks for pointing out this problem. It was easy to fix, and I've
pushed a change for that now. In testing it, I also realized that my
support for non-srcdir builds also broke the automatic running of
configure on a simple "make" invocation from a source tree that had
never been configured. I've fixed that now as well.

> My --prefix=[something] is gone.  (At least the build system warns about
> this.)  But it's another issue that other build systems already have
> solved.  (And, I probably wouldn't have expected that to do the right
> thing if the configure script / build system wouldn't try to be like GNU
> Autoconf.)

I agree that trying to be similar and being slightly different can be
more surprising than being obviously different. For the build system, I
actually do want compatibility with the features of
GNU-autoconf-generated build system that people actually care about. So
please keep letting me know about any annoying surprises.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Build system

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 964 bytes --]

On Tue, 08 Feb 2011 22:13:09 +0100, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> It won't be easy to run the tests in different directory, but the
> following approach might be sufficient: When notmuch is configured and
> compiled in a different directory, then running `make test` there could
> run the tests in the original directory but the tests will invoke the
> notmuch binary from where it was compiled.

That sounds like it could be a reasonable plan. Note, however, that
output generated by the test suite should go to the build directory, and
not the original source directory.

> If you send the patch for out-of-source compilation, I'll modify the
> test suite to work with it.

I've pushed a patch for out-of-source compilation now. I made the test
suite work by simply copying the whole thing. If you would like to fix
the test suite to work in this case without copying, that would be
great.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Build system

[Rob Browning]

Carl Worth <cworth@cworth.org> writes:

> This isn't ideal since it means that a non-srcdir build won't get access
> to updated emacs nor test-suite bits without re-running configure. If
> someone would like to do work to fix either or both of these cases, that
> would be helpful.

If you're willing to depend on GNU coreutils, would "cp -rl ..." work?

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

Re: Build system

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1094 bytes --]

On Thu, 10 Mar 2011 17:42:16 -0600, Rob Browning <rlb@defaultvalue.org> wrote:
> > This isn't ideal since it means that a non-srcdir build won't get access
> > to updated emacs nor test-suite bits without re-running configure. If
> > someone would like to do work to fix either or both of these cases, that
> > would be helpful.
> 
> If you're willing to depend on GNU coreutils, would "cp -rl ..." work?

I'd be willing to, but it wouldn't then also get access to new files
that appear in the test suite, (where the Makefiles could actually find
new, uncopied source files).

But wait, what am I thinking... the whole approach of the non-srcdir
builds as currently implemented also copies the Makefiles. So nothing at
all is going to work without re-running configure for non-srcdir builds.

I suppose that what we want is for configure to automatically add
dependencies for every file that it copies. Then if any of these copied
files get changed in the srcdir, configure would be run again.

That should be pretty slick, I think.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]