unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* [RFC PATCH] python-cffi out-of-tree build
@ 2021-02-17 21:36 Tomi Ollila
       [not found] ` <87a6m015ys.fsf@tethera.net>
  2021-10-31 11:40 ` David Bremner
  0 siblings, 2 replies; 5+ messages in thread
From: Tomi Ollila @ 2021-02-17 21:36 UTC (permalink / raw)
  To: notmuch; +Cc: tomi.ollila

setup.py and _build.py to refer some other files based on directory
where setup.py is located (os.path.dirname(sys.argv[0]).

Dropped bindings/python-cffi/version.txt and refer ../../version.txt
instead -- _build.py already refers ../../lib so why have version.txt
twice (with identical content).

Dropped copying of bindings/python-cffi source files to the build
directory bindings/python-cffi in configure.

---

an RFC alternative to id:20210215215410.28668-1-tomi.ollila@iki.fi

I really don't know how this should be done... do they even in e.g.
https://github.com/pypa/pip/issues/7555 ?

Tested to work in in-tree build, and out-of-tree builds referencing
.../notmuch/configure with absolute and relative paths, and all
3 builds worked.

When doing out-of-tree build, only _capi.abi3.so appeared in
bindings/python-cffi/build/stage/notmuch2/ -- in case of in-tree
build all the notmuch2/*.py source files also got copied there...

... I tried this changei in setup.py:
.  - packages=setuptools.find_packages(exclude=['tests'])
.  + packages=setuptools.find_packages(dn0, exclude=['tests'])
but then build broke.

 bindings/Makefile.local                 | 5 +++--
 bindings/python-cffi/notmuch2/_build.py | 6 +++++-
 bindings/python-cffi/setup.py           | 9 +++++++--
 bindings/python-cffi/version.txt        | 1 -
 configure                               | 4 ----
 5 files changed, 15 insertions(+), 10 deletions(-)
 delete mode 100644 bindings/python-cffi/version.txt

diff --git a/bindings/Makefile.local b/bindings/Makefile.local
index bc960bbc..d5c70bff 100644
--- a/bindings/Makefile.local
+++ b/bindings/Makefile.local
@@ -15,9 +15,10 @@ endif
 
 python-cffi-bindings: lib/$(LINKER_NAME)
 ifeq ($(HAVE_PYTHON3_CFFI),1)
+	test '$(srcdir)' = . && bdir=. || bdir='$(NOTMUCH_SRCDIR)/$(dir)/python-cffi'; \
 	cd $(dir)/python-cffi && \
-		${PYTHON} setup.py build --build-lib build/stage && \
-		mkdir -p build/stage/tests && cp tests/*.py build/stage/tests
+		${PYTHON} "$$bdir"/setup.py build --build-lib build/stage && \
+		mkdir -p build/stage/tests && cp "$$bdir"/tests/*.py build/stage/tests
 endif
 
 CLEAN += $(patsubst %,$(dir)/ruby/%, \
diff --git a/bindings/python-cffi/notmuch2/_build.py b/bindings/python-cffi/notmuch2/_build.py
index f67b4de6..ad585d21 100644
--- a/bindings/python-cffi/notmuch2/_build.py
+++ b/bindings/python-cffi/notmuch2/_build.py
@@ -1,5 +1,9 @@
 import cffi
 
+import os
+import sys
+
+dn0 = os.path.dirname(sys.argv[0])
 
 ffibuilder = cffi.FFI()
 ffibuilder.set_source(
@@ -16,7 +20,7 @@ ffibuilder.set_source(
         #ERROR libnotmuch  version < 5.1 not supported
     #endif
     """,
-    include_dirs=['../../lib'],
+    include_dirs=[dn0 + '/../../lib'],
     library_dirs=['../../lib'],
     libraries=['notmuch'],
 )
diff --git a/bindings/python-cffi/setup.py b/bindings/python-cffi/setup.py
index cda52338..5884944b 100644
--- a/bindings/python-cffi/setup.py
+++ b/bindings/python-cffi/setup.py
@@ -1,6 +1,11 @@
 import setuptools
 
-with open('version.txt') as fp:
+import os
+import sys
+
+dn0 = os.path.dirname(sys.argv[0])
+
+with open(dn0 + '/../../version.txt') as fp:
     VERSION = fp.read().strip()
 
 setuptools.setup(
@@ -12,7 +17,7 @@ setuptools.setup(
     setup_requires=['cffi>=1.0.0'],
     install_requires=['cffi>=1.0.0'],
     packages=setuptools.find_packages(exclude=['tests']),
-    cffi_modules=['notmuch2/_build.py:ffibuilder'],
+    cffi_modules=[dn0 + '/notmuch2/_build.py:ffibuilder'],
     classifiers=[
         'Development Status :: 3 - Alpha',
         'Intended Audience :: Developers',
diff --git a/bindings/python-cffi/version.txt b/bindings/python-cffi/version.txt
deleted file mode 100644
index 8239f42d..00000000
--- a/bindings/python-cffi/version.txt
+++ /dev/null
@@ -1 +0,0 @@
-0.31.3
diff --git a/configure b/configure
index cfa9c09b..7bdd7a13 100755
--- a/configure
+++ b/configure
@@ -74,10 +74,6 @@ if [ "$srcdir" != "." ]; then
     # Use the same hack to replicate python-cffi source for
     # out-of-tree builds (again, not ideal).
     mkdir bindings/python-cffi
-    cp -a "$srcdir"/bindings/python-cffi/tests \
-       "$srcdir"/bindings/python-cffi/notmuch2 \
-       "$srcdir"/bindings/python-cffi/setup.py \
-       bindings/python-cffi/
 fi
 
 # Set several defaults (optionally specified by the user in
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [Tomi Ollila] [RFC PATCH] python-cffi out-of-tree build
       [not found] ` <87a6m015ys.fsf@tethera.net>
@ 2021-08-10 20:39   ` Floris Bruynooghe
  2021-10-05 12:22     ` David Bremner
  0 siblings, 1 reply; 5+ messages in thread
From: Floris Bruynooghe @ 2021-08-10 20:39 UTC (permalink / raw)
  To: David Bremner, notmuch

Hi David, everyone,

I had a vague recollection of reading from a single version file before
and indeed, looking through annotations it seems commit
3a42abb456893b71b530f099a1467400f2b0ea71 changed this.  Seems the
concern was that "pip install ." didn't work.  So perhaps that's
something else to check?

And err... thanks for your patience.  I know this is really slow!

Cheers,
Floris


On Mon 02 Aug 2021 at 07:44 -0300, David Bremner wrote:

> Any thoughts on this one?
> From: Tomi Ollila <tomi.ollila@iki.fi>
> Subject: [RFC PATCH] python-cffi out-of-tree build
> To: notmuch@notmuchmail.org
> Cc: tomi.ollila@iki.fi
> Date: Wed, 17 Feb 2021 23:36:00 +0200
>
> setup.py and _build.py to refer some other files based on directory
> where setup.py is located (os.path.dirname(sys.argv[0]).
>
> Dropped bindings/python-cffi/version.txt and refer ../../version.txt
> instead -- _build.py already refers ../../lib so why have version.txt
> twice (with identical content).
>
> Dropped copying of bindings/python-cffi source files to the build
> directory bindings/python-cffi in configure.
>
> ---
>
> an RFC alternative to id:20210215215410.28668-1-tomi.ollila@iki.fi
>
> I really don't know how this should be done... do they even in e.g.
> https://github.com/pypa/pip/issues/7555 ?
>
> Tested to work in in-tree build, and out-of-tree builds referencing
> .../notmuch/configure with absolute and relative paths, and all
> 3 builds worked.
>
> When doing out-of-tree build, only _capi.abi3.so appeared in
> bindings/python-cffi/build/stage/notmuch2/ -- in case of in-tree
> build all the notmuch2/*.py source files also got copied there...
>
> ... I tried this changei in setup.py:
> .  - packages=setuptools.find_packages(exclude=['tests'])
> .  + packages=setuptools.find_packages(dn0, exclude=['tests'])
> but then build broke.
>
>  bindings/Makefile.local                 | 5 +++--
>  bindings/python-cffi/notmuch2/_build.py | 6 +++++-
>  bindings/python-cffi/setup.py           | 9 +++++++--
>  bindings/python-cffi/version.txt        | 1 -
>  configure                               | 4 ----
>  5 files changed, 15 insertions(+), 10 deletions(-)
>  delete mode 100644 bindings/python-cffi/version.txt
>
> diff --git a/bindings/Makefile.local b/bindings/Makefile.local
> index bc960bbc..d5c70bff 100644
> --- a/bindings/Makefile.local
> +++ b/bindings/Makefile.local
> @@ -15,9 +15,10 @@ endif
>  
>  python-cffi-bindings: lib/$(LINKER_NAME)
>  ifeq ($(HAVE_PYTHON3_CFFI),1)
> +	test '$(srcdir)' = . && bdir=. || bdir='$(NOTMUCH_SRCDIR)/$(dir)/python-cffi'; \
>  	cd $(dir)/python-cffi && \
> -		${PYTHON} setup.py build --build-lib build/stage && \
> -		mkdir -p build/stage/tests && cp tests/*.py build/stage/tests
> +		${PYTHON} "$$bdir"/setup.py build --build-lib build/stage && \
> +		mkdir -p build/stage/tests && cp "$$bdir"/tests/*.py build/stage/tests
>  endif
>  
>  CLEAN += $(patsubst %,$(dir)/ruby/%, \
> diff --git a/bindings/python-cffi/notmuch2/_build.py b/bindings/python-cffi/notmuch2/_build.py
> index f67b4de6..ad585d21 100644
> --- a/bindings/python-cffi/notmuch2/_build.py
> +++ b/bindings/python-cffi/notmuch2/_build.py
> @@ -1,5 +1,9 @@
>  import cffi
>  
> +import os
> +import sys
> +
> +dn0 = os.path.dirname(sys.argv[0])
>  
>  ffibuilder = cffi.FFI()
>  ffibuilder.set_source(
> @@ -16,7 +20,7 @@ ffibuilder.set_source(
>          #ERROR libnotmuch  version < 5.1 not supported
>      #endif
>      """,
> -    include_dirs=['../../lib'],
> +    include_dirs=[dn0 + '/../../lib'],
>      library_dirs=['../../lib'],
>      libraries=['notmuch'],
>  )
> diff --git a/bindings/python-cffi/setup.py b/bindings/python-cffi/setup.py
> index cda52338..5884944b 100644
> --- a/bindings/python-cffi/setup.py
> +++ b/bindings/python-cffi/setup.py
> @@ -1,6 +1,11 @@
>  import setuptools
>  
> -with open('version.txt') as fp:
> +import os
> +import sys
> +
> +dn0 = os.path.dirname(sys.argv[0])
> +
> +with open(dn0 + '/../../version.txt') as fp:
>      VERSION = fp.read().strip()
>  
>  setuptools.setup(
> @@ -12,7 +17,7 @@ setuptools.setup(
>      setup_requires=['cffi>=1.0.0'],
>      install_requires=['cffi>=1.0.0'],
>      packages=setuptools.find_packages(exclude=['tests']),
> -    cffi_modules=['notmuch2/_build.py:ffibuilder'],
> +    cffi_modules=[dn0 + '/notmuch2/_build.py:ffibuilder'],
>      classifiers=[
>          'Development Status :: 3 - Alpha',
>          'Intended Audience :: Developers',
> diff --git a/bindings/python-cffi/version.txt b/bindings/python-cffi/version.txt
> deleted file mode 100644
> index 8239f42d..00000000
> --- a/bindings/python-cffi/version.txt
> +++ /dev/null
> @@ -1 +0,0 @@
> -0.31.3
> diff --git a/configure b/configure
> index cfa9c09b..7bdd7a13 100755
> --- a/configure
> +++ b/configure
> @@ -74,10 +74,6 @@ if [ "$srcdir" != "." ]; then
>      # Use the same hack to replicate python-cffi source for
>      # out-of-tree builds (again, not ideal).
>      mkdir bindings/python-cffi
> -    cp -a "$srcdir"/bindings/python-cffi/tests \
> -       "$srcdir"/bindings/python-cffi/notmuch2 \
> -       "$srcdir"/bindings/python-cffi/setup.py \
> -       bindings/python-cffi/
>  fi
>  
>  # Set several defaults (optionally specified by the user in

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [Tomi Ollila] [RFC PATCH] python-cffi out-of-tree build
  2021-08-10 20:39   ` [Tomi Ollila] " Floris Bruynooghe
@ 2021-10-05 12:22     ` David Bremner
  0 siblings, 0 replies; 5+ messages in thread
From: David Bremner @ 2021-10-05 12:22 UTC (permalink / raw)
  To: Floris Bruynooghe, notmuch; +Cc: Tomi Ollila

Floris Bruynooghe <flub@devork.be> writes:

> Hi David, everyone,
>
> I had a vague recollection of reading from a single version file before
> and indeed, looking through annotations it seems commit
> 3a42abb456893b71b530f099a1467400f2b0ea71 changed this.  Seems the
> concern was that "pip install ." didn't work.  So perhaps that's
> something else to check?
>

Yes, that's correct. There could be a different way of doing this, but I
don't understand pip very well, so this is the best I could come up with
to make "pip install ." work.

d

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [RFC PATCH] python-cffi out-of-tree build
  2021-02-17 21:36 [RFC PATCH] python-cffi out-of-tree build Tomi Ollila
       [not found] ` <87a6m015ys.fsf@tethera.net>
@ 2021-10-31 11:40 ` David Bremner
  2021-10-31 19:28   ` Tomi Ollila
  1 sibling, 1 reply; 5+ messages in thread
From: David Bremner @ 2021-10-31 11:40 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:

> setup.py and _build.py to refer some other files based on directory
> where setup.py is located (os.path.dirname(sys.argv[0]).
>
> Dropped bindings/python-cffi/version.txt and refer ../../version.txt
> instead -- _build.py already refers ../../lib so why have version.txt
> twice (with identical content).
>
> Dropped copying of bindings/python-cffi source files to the build
> directory bindings/python-cffi in configure.

Apologies, I think I did not read this discussion very carefully
before. I guess [1] is an alternative to the first two points. I'm not
sure which approach is better. Mine is a bigger hammer, but maybe that
could be useful in the future?

The patch at [1] does not (by itself) allow the third change, but
probably some similar hacking of setup.py, but using the newly created
_notmuch_config.py would work.

Tomi, did you happen to test "pip install ." with your approach? I'm
afraid your patch doesn't (easily) apply anymore so I was too lazy to
test myself.

Cheers,

db

[1]: id:20211031100354.1697379-1-david@tethera.net

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [RFC PATCH] python-cffi out-of-tree build
  2021-10-31 11:40 ` David Bremner
@ 2021-10-31 19:28   ` Tomi Ollila
  0 siblings, 0 replies; 5+ messages in thread
From: Tomi Ollila @ 2021-10-31 19:28 UTC (permalink / raw)
  To: David Bremner, notmuch

On Sun, Oct 31 2021, David Bremner wrote:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>
>> setup.py and _build.py to refer some other files based on directory
>> where setup.py is located (os.path.dirname(sys.argv[0]).
>>
>> Dropped bindings/python-cffi/version.txt and refer ../../version.txt
>> instead -- _build.py already refers ../../lib so why have version.txt
>> twice (with identical content).
>>
>> Dropped copying of bindings/python-cffi source files to the build
>> directory bindings/python-cffi in configure.
>
> Apologies, I think I did not read this discussion very carefully
> before. I guess [1] is an alternative to the first two points. I'm not
> sure which approach is better. Mine is a bigger hammer, but maybe that
> could be useful in the future?
>
> The patch at [1] does not (by itself) allow the third change, but
> probably some similar hacking of setup.py, but using the newly created
> _notmuch_config.py would work.
>
> Tomi, did you happen to test "pip install ." with your approach? I'm
> afraid your patch doesn't (easily) apply anymore so I was too lazy to
> test myself.

IIRC my approaches had quite a few problems which I thought soon after
emailing so probablhy your versions are better. Have to look...

Tomi

>
> Cheers,
>
> db
>
> [1]: id:20211031100354.1697379-1-david@tethera.net
> _______________________________________________
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-leave@notmuchmail.org

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-10-31 19:28 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-17 21:36 [RFC PATCH] python-cffi out-of-tree build Tomi Ollila
     [not found] ` <87a6m015ys.fsf@tethera.net>
2021-08-10 20:39   ` [Tomi Ollila] " Floris Bruynooghe
2021-10-05 12:22     ` David Bremner
2021-10-31 11:40 ` David Bremner
2021-10-31 19:28   ` Tomi Ollila

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).