unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* python CFFI bindings integration into notmuch build/test
@ 2019-11-04  1:39 David Bremner
  2019-11-04  1:39 ` [PATCH 1/5] bindings/python-cffi: preserve environment for tests David Bremner
                   ` (6 more replies)
  0 siblings, 7 replies; 32+ messages in thread
From: David Bremner @ 2019-11-04  1:39 UTC (permalink / raw)
  To: notmuch

This series is on top of a slightly modified version [1] of

     id:20191008210312.20685-2-flub@devork.be

If you want to try out the new bindings, you can try the branch wip/cffi in the repo
https://git.notmuchmail.org/git/notmuch

One current annoyance is that the version of pytest on travis is too
old for what is specified in tox.ini.

[1]: the diff is as follows.

diff --git a/bindings/python-cffi/notdb/_build.py b/bindings/python-cffi/notdb/_build.py
index 97ad7808..6be7e5b1 100644
--- a/bindings/python-cffi/notdb/_build.py
+++ b/bindings/python-cffi/notdb/_build.py
@@ -13,6 +13,8 @@ ffibuilder.set_source(
         #error libnotmuch version not supported by notdb
     #endif
     """,
+    include_dirs=['../../lib'],
+    library_dirs=['../../lib'],
     libraries=['notmuch'],
 )
 ffibuilder.cdef(
diff --git a/bindings/python-cffi/tests/conftest.py b/bindings/python-cffi/tests/conftest.py
index 1b7bbc35..aa940947 100644
--- a/bindings/python-cffi/tests/conftest.py
+++ b/bindings/python-cffi/tests/conftest.py
@@ -5,6 +5,7 @@ import socket
 import subprocess
 import textwrap
 import time
+import os
 
 import pytest
 
@@ -32,10 +33,11 @@ def notmuch(maildir):
         """
         cfg_fname = maildir.path / 'notmuch-config'
         cmd = ['notmuch'] + list(args)
-        print('Invoking: {}'.format(' '.join(cmd)))
+        env = os.environ.copy()
+        env['NOTMUCH_CONFIG'] = str(cfg_fname)
         proc = subprocess.run(cmd,
                               timeout=5,
-                              env={'NOTMUCH_CONFIG': str(cfg_fname)})
+                              env=env)
         proc.check_returncode()
     return run

t

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH 1/5] bindings/python-cffi: preserve environment for tests
  2019-11-04  1:39 python CFFI bindings integration into notmuch build/test David Bremner
@ 2019-11-04  1:39 ` David Bremner
  2019-11-04  1:39 ` [PATCH 2/5] configure: check for python cffi module David Bremner
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 32+ messages in thread
From: David Bremner @ 2019-11-04  1:39 UTC (permalink / raw)
  To: notmuch

We'll need this e.g. to pass PATH to the pytest tests

Based on the suggested approach in id:87d0eljggj.fsf@powell.devork.be
---
 bindings/python-cffi/tests/conftest.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/bindings/python-cffi/tests/conftest.py b/bindings/python-cffi/tests/conftest.py
index 1b7bbc35..aa940947 100644
--- a/bindings/python-cffi/tests/conftest.py
+++ b/bindings/python-cffi/tests/conftest.py
@@ -5,6 +5,7 @@ import socket
 import subprocess
 import textwrap
 import time
+import os
 
 import pytest
 
@@ -32,10 +33,11 @@ def notmuch(maildir):
         """
         cfg_fname = maildir.path / 'notmuch-config'
         cmd = ['notmuch'] + list(args)
-        print('Invoking: {}'.format(' '.join(cmd)))
+        env = os.environ.copy()
+        env['NOTMUCH_CONFIG'] = str(cfg_fname)
         proc = subprocess.run(cmd,
                               timeout=5,
-                              env={'NOTMUCH_CONFIG': str(cfg_fname)})
+                              env=env)
         proc.check_returncode()
     return run
 
-- 
2.24.0.rc1

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH 2/5] configure: check for python cffi module
  2019-11-04  1:39 python CFFI bindings integration into notmuch build/test David Bremner
  2019-11-04  1:39 ` [PATCH 1/5] bindings/python-cffi: preserve environment for tests David Bremner
@ 2019-11-04  1:39 ` David Bremner
  2019-11-04 21:24   ` Tomi Ollila
  2019-11-04  1:39 ` [PATCH 3/5] build: optionally build python-cffi bindings David Bremner
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 32+ messages in thread
From: David Bremner @ 2019-11-04  1:39 UTC (permalink / raw)
  To: notmuch

This is needed to build the new python bindings
---
 configure | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/configure b/configure
index 3c148e12..300591fe 100755
--- a/configure
+++ b/configure
@@ -671,6 +671,15 @@ if [ $have_python -eq 0 ]; then
     errors=$((errors + 1))
 fi
 
+printf "Checking for python cffi... "
+if "$python" -c 'import cffi' >/dev/null 2>&1; then
+    printf "Yes.\n"
+    have_python_cffi=1;
+else
+    printf "No.\n"
+    have_python_cffi=0;
+fi
+
 printf "Checking for valgrind development files... "
 if pkg-config --exists valgrind; then
     printf "Yes.\n"
@@ -1223,6 +1232,9 @@ HAVE_GETLINE = ${have_getline}
 # building/testing ruby bindings.
 HAVE_RUBY_DEV = ${have_ruby_dev}
 
+# Is the python cffi package available?
+HAVE_PYTHON_CFFI = ${have_python_cffi}
+
 # Whether the strcasestr function is available (if not, then notmuch will
 # build its own version)
 HAVE_STRCASESTR = ${have_strcasestr}
@@ -1376,6 +1388,9 @@ NOTMUCH_RUBY=${RUBY}
 # building/testing ruby bindings.
 NOTMUCH_HAVE_RUBY_DEV=${have_ruby_dev}
 
+# Is the python cffi package available?
+NOTMUCH_HAVE_PYTHON_CFFI=${have_python_cffi}
+
 # Platform we are run on
 PLATFORM=${platform}
 EOF
-- 
2.24.0.rc1

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH 3/5] build: optionally build python-cffi bindings
  2019-11-04  1:39 python CFFI bindings integration into notmuch build/test David Bremner
  2019-11-04  1:39 ` [PATCH 1/5] bindings/python-cffi: preserve environment for tests David Bremner
  2019-11-04  1:39 ` [PATCH 2/5] configure: check for python cffi module David Bremner
@ 2019-11-04  1:39 ` David Bremner
  2019-11-04 21:26   ` Tomi Ollila
  2019-11-04  1:39 ` [PATCH 4/5] tests: run python-cffi tests David Bremner
                   ` (3 subsequent siblings)
  6 siblings, 1 reply; 32+ messages in thread
From: David Bremner @ 2019-11-04  1:39 UTC (permalink / raw)
  To: notmuch

Put the build product (and tests) in a well known location so that we
can find them e.g. from the tests.
---
 Makefile.local          | 2 +-
 bindings/Makefile.local | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/Makefile.local b/Makefile.local
index 3c6dacbc..7c12612d 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -1,7 +1,7 @@
 # -*- makefile -*-
 
 .PHONY: all
-all: notmuch notmuch-shared build-man build-info ruby-bindings
+all: notmuch notmuch-shared build-man build-info ruby-bindings python-cffi-bindings
 ifeq ($(MAKECMDGOALS),)
 ifeq ($(shell cat .first-build-message 2>/dev/null),)
 	@NOTMUCH_FIRST_BUILD=1 $(MAKE) --no-print-directory all
diff --git a/bindings/Makefile.local b/bindings/Makefile.local
index 18f95835..b8e18c92 100644
--- a/bindings/Makefile.local
+++ b/bindings/Makefile.local
@@ -13,6 +13,13 @@ ifeq ($(HAVE_RUBY_DEV),1)
 	$(MAKE) -C $(dir)/ruby
 endif
 
+python-cffi-bindings: lib/$(LINKER_NAME)
+ifeq ($(HAVE_PYTHON_CFFI),1)
+	cd $(dir)/python-cffi && \
+		${PYTHON} setup.py build --build-lib build/stage && \
+		mkdir -p build/stage/tests && cp tests/*.py build/stage/tests
+endif
+
 CLEAN += $(patsubst %,$(dir)/ruby/%, \
 	.RUBYARCHDIR.time \
 	Makefile database.o directory.o filenames.o\
@@ -20,3 +27,5 @@ CLEAN += $(patsubst %,$(dir)/ruby/%, \
 	status.o tags.o thread.o threads.o)
 
 CLEAN += bindings/ruby/.vendorarchdir.time
+
+CLEAN += bindings/python-cffi/build
-- 
2.24.0.rc1

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH 4/5] tests: run python-cffi tests
  2019-11-04  1:39 python CFFI bindings integration into notmuch build/test David Bremner
                   ` (2 preceding siblings ...)
  2019-11-04  1:39 ` [PATCH 3/5] build: optionally build python-cffi bindings David Bremner
@ 2019-11-04  1:39 ` David Bremner
  2019-11-04 16:18   ` Tomi Ollila
  2019-11-04 21:45   ` Tomi Ollila
  2019-11-04  1:39 ` [PATCH 5/5] travis: add python3-{cffi,pytest,setuptools} David Bremner
                   ` (2 subsequent siblings)
  6 siblings, 2 replies; 32+ messages in thread
From: David Bremner @ 2019-11-04  1:39 UTC (permalink / raw)
  To: notmuch

The entire python-cffi test suite is considered as a single test at
the level of the notmuch test suite. This might or might not be ideal,
but it gets them run.
---
 test/T391-python-cffi.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100755 test/T391-python-cffi.sh

diff --git a/test/T391-python-cffi.sh b/test/T391-python-cffi.sh
new file mode 100755
index 00000000..a03f9e0e
--- /dev/null
+++ b/test/T391-python-cffi.sh
@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+test_description="python bindings (pytest)"
+. $(dirname "$0")/test-lib.sh || exit 1
+
+if [ $NOTMUCH_HAVE_PYTHON_CFFI -eq 0 ]; then
+    test_done
+fi
+
+
+test_begin_subtest "python cffi tests"
+test_expect_success "(cd $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage && ${NOTMUCH_PYTHON} -m pytest --log-file=$TMP_DIRECTORY/test.output)"
+test_done
-- 
2.24.0.rc1

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH 5/5] travis: add python3-{cffi,pytest,setuptools}
  2019-11-04  1:39 python CFFI bindings integration into notmuch build/test David Bremner
                   ` (3 preceding siblings ...)
  2019-11-04  1:39 ` [PATCH 4/5] tests: run python-cffi tests David Bremner
@ 2019-11-04  1:39 ` David Bremner
  2019-11-04 11:10 ` python CFFI bindings integration into notmuch build/test David Bremner
  2019-11-05 20:47 ` Tomi Ollila
  6 siblings, 0 replies; 32+ messages in thread
From: David Bremner @ 2019-11-04  1:39 UTC (permalink / raw)
  To: notmuch

These are needed for building and testing the new python bindings.
---
 .travis.yml | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index f9516bde..cfd89a28 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -13,6 +13,9 @@ addons:
     - libgmime-3.0-dev
     - libtalloc-dev
     - python3-sphinx
+    - python3-cffi
+    - python3-pytest
+    - python3-setuptools
     - gpgsm
 
 script:
-- 
2.24.0.rc1

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-04  1:39 python CFFI bindings integration into notmuch build/test David Bremner
                   ` (4 preceding siblings ...)
  2019-11-04  1:39 ` [PATCH 5/5] travis: add python3-{cffi,pytest,setuptools} David Bremner
@ 2019-11-04 11:10 ` David Bremner
  2019-11-05 20:47 ` Tomi Ollila
  6 siblings, 0 replies; 32+ messages in thread
From: David Bremner @ 2019-11-04 11:10 UTC (permalink / raw)
  To: notmuch; +Cc: Daniel Kahn Gillmor

David Bremner <david@tethera.net> writes:

> This series is on top of a slightly modified version [1] of
>
>      id:20191008210312.20685-2-flub@devork.be
>
> If you want to try out the new bindings, you can try the branch wip/cffi in the repo
> https://git.notmuchmail.org/git/notmuch
>
> One current annoyance is that the version of pytest on travis is too
> old for what is specified in tox.ini.
>

I fixed this by moving the travis env to bionic (2018.x).  This involved
dropping dkg's notmuch ppa. Daniel, do you happen to know if there is
anything in that repo that we should still keep when building in ubuntu
bionic? FWIW, the build succeeds without that PPA.

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-04  1:39 ` [PATCH 4/5] tests: run python-cffi tests David Bremner
@ 2019-11-04 16:18   ` Tomi Ollila
  2019-11-04 16:57     ` David Bremner
  2019-11-04 21:45   ` Tomi Ollila
  1 sibling, 1 reply; 32+ messages in thread
From: Tomi Ollila @ 2019-11-04 16:18 UTC (permalink / raw)
  To: David Bremner, notmuch

On Sun, Nov 03 2019, David Bremner wrote:

> The entire python-cffi test suite is considered as a single test at
> the level of the notmuch test suite. This might or might not be ideal,
> but it gets them run.

IMO this is good enough approach -- provided that pytest is prereq
for this test to be run =D

(i.e. one may have cffi but decided not to install pytest)

Tomi

> ---
>  test/T391-python-cffi.sh | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>  create mode 100755 test/T391-python-cffi.sh
>
> diff --git a/test/T391-python-cffi.sh b/test/T391-python-cffi.sh
> new file mode 100755
> index 00000000..a03f9e0e
> --- /dev/null
> +++ b/test/T391-python-cffi.sh
> @@ -0,0 +1,12 @@
> +#!/usr/bin/env bash
> +test_description="python bindings (pytest)"
> +. $(dirname "$0")/test-lib.sh || exit 1
> +
> +if [ $NOTMUCH_HAVE_PYTHON_CFFI -eq 0 ]; then
> +    test_done
> +fi
> +
> +
> +test_begin_subtest "python cffi tests"
> +test_expect_success "(cd $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage && ${NOTMUCH_PYTHON} -m pytest --log-file=$TMP_DIRECTORY/test.output)"
> +test_done
> -- 
> 2.24.0.rc1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-04 16:18   ` Tomi Ollila
@ 2019-11-04 16:57     ` David Bremner
  0 siblings, 0 replies; 32+ messages in thread
From: David Bremner @ 2019-11-04 16:57 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Sun, Nov 03 2019, David Bremner wrote:
>
>> The entire python-cffi test suite is considered as a single test at
>> the level of the notmuch test suite. This might or might not be ideal,
>> but it gets them run.
>
> IMO this is good enough approach -- provided that pytest is prereq
> for this test to be run =D
>
> (i.e. one may have cffi but decided not to install pytest)
>
> Tomi

Yes, that's a good point, that actually caused some failures. I guess
I'll also need to check for pytest-cov, or disable coverage testing,
which is apparently enabled by default in tox.ini. At the moment I don't
know the best way to do that, but "python3 -m pytest -c /dev/null" seems
to work.

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 2/5] configure: check for python cffi module
  2019-11-04  1:39 ` [PATCH 2/5] configure: check for python cffi module David Bremner
@ 2019-11-04 21:24   ` Tomi Ollila
  0 siblings, 0 replies; 32+ messages in thread
From: Tomi Ollila @ 2019-11-04 21:24 UTC (permalink / raw)
  To: David Bremner, notmuch

On Sun, Nov 03 2019, David Bremner wrote:

> This is needed to build the new python bindings
> ---
>  configure | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
>
> diff --git a/configure b/configure
> index 3c148e12..300591fe 100755
> --- a/configure
> +++ b/configure
> @@ -671,6 +671,15 @@ if [ $have_python -eq 0 ]; then
>      errors=$((errors + 1))
>  fi
>  
> +printf "Checking for python cffi... "
> +if "$python" -c 'import cffi' >/dev/null 2>&1; then
> +    printf "Yes.\n"
> +    have_python_cffi=1;

trailing ;

> +else
> +    printf "No.\n"
> +    have_python_cffi=0;

ditto :)


otherwise tolerable =D

> +fi
> +
>  printf "Checking for valgrind development files... "
>  if pkg-config --exists valgrind; then
>      printf "Yes.\n"
> @@ -1223,6 +1232,9 @@ HAVE_GETLINE = ${have_getline}
>  # building/testing ruby bindings.
>  HAVE_RUBY_DEV = ${have_ruby_dev}
>  
> +# Is the python cffi package available?
> +HAVE_PYTHON_CFFI = ${have_python_cffi}
> +
>  # Whether the strcasestr function is available (if not, then notmuch will
>  # build its own version)
>  HAVE_STRCASESTR = ${have_strcasestr}
> @@ -1376,6 +1388,9 @@ NOTMUCH_RUBY=${RUBY}
>  # building/testing ruby bindings.
>  NOTMUCH_HAVE_RUBY_DEV=${have_ruby_dev}
>  
> +# Is the python cffi package available?
> +NOTMUCH_HAVE_PYTHON_CFFI=${have_python_cffi}
> +
>  # Platform we are run on
>  PLATFORM=${platform}
>  EOF
> -- 
> 2.24.0.rc1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 3/5] build: optionally build python-cffi bindings
  2019-11-04  1:39 ` [PATCH 3/5] build: optionally build python-cffi bindings David Bremner
@ 2019-11-04 21:26   ` Tomi Ollila
  2019-11-05  0:32     ` David Bremner
  2020-05-22  1:06     ` Daniel Kahn Gillmor
  0 siblings, 2 replies; 32+ messages in thread
From: Tomi Ollila @ 2019-11-04 21:26 UTC (permalink / raw)
  To: David Bremner, notmuch

On Sun, Nov 03 2019, David Bremner wrote:

> Put the build product (and tests) in a well known location so that we
> can find them e.g. from the tests.
> ---
>  Makefile.local          | 2 +-
>  bindings/Makefile.local | 9 +++++++++
>  2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile.local b/Makefile.local
> index 3c6dacbc..7c12612d 100644
> --- a/Makefile.local
> +++ b/Makefile.local
> @@ -1,7 +1,7 @@
>  # -*- makefile -*-
>  
>  .PHONY: all
> -all: notmuch notmuch-shared build-man build-info ruby-bindings
> +all: notmuch notmuch-shared build-man build-info ruby-bindings python-cffi-bindings
>  ifeq ($(MAKECMDGOALS),)
>  ifeq ($(shell cat .first-build-message 2>/dev/null),)
>  	@NOTMUCH_FIRST_BUILD=1 $(MAKE) --no-print-directory all
> diff --git a/bindings/Makefile.local b/bindings/Makefile.local
> index 18f95835..b8e18c92 100644
> --- a/bindings/Makefile.local
> +++ b/bindings/Makefile.local
> @@ -13,6 +13,13 @@ ifeq ($(HAVE_RUBY_DEV),1)
>  	$(MAKE) -C $(dir)/ruby
>  endif
>  
> +python-cffi-bindings: lib/$(LINKER_NAME)
> +ifeq ($(HAVE_PYTHON_CFFI),1)
> +	cd $(dir)/python-cffi && \

how bad does out-of-tree build break with this -- do we need to do
the same as with ruby bindings (copy sources -- do we still do so)? or does
python provide better alternative..?

> +		${PYTHON} setup.py build --build-lib build/stage && \
> +		mkdir -p build/stage/tests && cp tests/*.py build/stage/tests
> +endif
> +
>  CLEAN += $(patsubst %,$(dir)/ruby/%, \
>  	.RUBYARCHDIR.time \
>  	Makefile database.o directory.o filenames.o\
> @@ -20,3 +27,5 @@ CLEAN += $(patsubst %,$(dir)/ruby/%, \
>  	status.o tags.o thread.o threads.o)
>  
>  CLEAN += bindings/ruby/.vendorarchdir.time
> +
> +CLEAN += bindings/python-cffi/build
> -- 
> 2.24.0.rc1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-04  1:39 ` [PATCH 4/5] tests: run python-cffi tests David Bremner
  2019-11-04 16:18   ` Tomi Ollila
@ 2019-11-04 21:45   ` Tomi Ollila
  2019-11-04 22:04     ` Tomi Ollila
  2019-11-06  2:19     ` David Bremner
  1 sibling, 2 replies; 32+ messages in thread
From: Tomi Ollila @ 2019-11-04 21:45 UTC (permalink / raw)
  To: David Bremner, notmuch

On Sun, Nov 03 2019, David Bremner wrote:

> The entire python-cffi test suite is considered as a single test at
> the level of the notmuch test suite. This might or might not be ideal,
> but it gets them run.
> ---
>  test/T391-python-cffi.sh | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>  create mode 100755 test/T391-python-cffi.sh
>
> diff --git a/test/T391-python-cffi.sh b/test/T391-python-cffi.sh
> new file mode 100755
> index 00000000..a03f9e0e
> --- /dev/null
> +++ b/test/T391-python-cffi.sh
> @@ -0,0 +1,12 @@
> +#!/usr/bin/env bash
> +test_description="python bindings (pytest)"
> +. $(dirname "$0")/test-lib.sh || exit 1
> +
> +if [ $NOTMUCH_HAVE_PYTHON_CFFI -eq 0 ]; then
> +    test_done
> +fi
> +
> +
> +test_begin_subtest "python cffi tests"
> +test_expect_success "(cd $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage && ${NOTMUCH_PYTHON} -m pytest --log-file=$TMP_DIRECTORY/test.output)"

probably

test_expect_success "${NOTMUCH_PYTHON} -m pytest \
                    --log-file=$TMP_DIRECTORY/test.output \
                    $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage" 

worked (which reminds me out-of-tree... ;)

as for checking pytest existence, the following works somewhat:

   ~/test-venv/bin/python3 -m pytest --version

It prints the version, then exists. Could not figure out how to request
specific version in quick look (10 min search/trial/error session),
parsing version output is so... :/ 


> +test_done
> -- 
> 2.24.0.rc1

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-04 21:45   ` Tomi Ollila
@ 2019-11-04 22:04     ` Tomi Ollila
  2019-11-06  2:19     ` David Bremner
  1 sibling, 0 replies; 32+ messages in thread
From: Tomi Ollila @ 2019-11-04 22:04 UTC (permalink / raw)
  To: David Bremner, notmuch

On Mon, Nov 04 2019, Tomi Ollila wrote:

>
> as for checking pytest existence, the following works somewhat:
>
>    ~/test-venv/bin/python3 -m pytest --version
>
> It prints the version, then exists. Could not figure out how to request
> specific version in quick look (10 min search/trial/error session),
> parsing version output is so... :/ 

Hah, dumped ~/test-venv/. 

so retry...

$ python3 -m venv ~/test-venv
$ ~/test-venv/bin/pip3 install pytest
$ ~/test-venv/bin/python3
>>> import pytest
>>> dir(pytest)
>>> pytest.__version__

prints out '5.2.2'. argh, string again (in addition to resorting 
to __...__ names/attributes of whatever these are called in python ;).
anyway, even parsing this version string in python than doing it in
shell script would be better...(*)

anything better ?

Tomi

(*) got exhausted when looking through this:

https://stackoverflow.com/questions/11887762/how-do-i-compare-version-numbers-in-python

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 3/5] build: optionally build python-cffi bindings
  2019-11-04 21:26   ` Tomi Ollila
@ 2019-11-05  0:32     ` David Bremner
  2020-05-22  1:06     ` Daniel Kahn Gillmor
  1 sibling, 0 replies; 32+ messages in thread
From: David Bremner @ 2019-11-05  0:32 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:

> how bad does out-of-tree build break with this -- do we need to do
> the same as with ruby bindings (copy sources -- do we still do so)? or does
> python provide better alternative..?

Good question.  What is here will break with out of tree builds, just
from trivial things like missing $(srcdir). At a deeper level someone
would probably have to modify the build bits (python-cffi/setup.py,
python-cffi/notb/_build) to work out of tree.  I don't know how well any
of this is supported by the setuptools module. It seems more friendly to
the idea of building inside the source tree with destination outside,
but I'm really not a setuptools expert.

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-04  1:39 python CFFI bindings integration into notmuch build/test David Bremner
                   ` (5 preceding siblings ...)
  2019-11-04 11:10 ` python CFFI bindings integration into notmuch build/test David Bremner
@ 2019-11-05 20:47 ` Tomi Ollila
  2019-11-06  2:22   ` David Bremner
  6 siblings, 1 reply; 32+ messages in thread
From: Tomi Ollila @ 2019-11-05 20:47 UTC (permalink / raw)
  To: David Bremner, notmuch

On Sun, Nov 03 2019, David Bremner wrote:

>  
> @@ -32,10 +33,11 @@ def notmuch(maildir):
>          """
>          cfg_fname = maildir.path / 'notmuch-config'
>          cmd = ['notmuch'] + list(args)
> -        print('Invoking: {}'.format(' '.join(cmd)))
> +        env = os.environ.copy()
> +        env['NOTMUCH_CONFIG'] = str(cfg_fname)
>          proc = subprocess.run(cmd,
>                                timeout=5,
> -                              env={'NOTMUCH_CONFIG': str(cfg_fname)})
> +                              env=env)

Tried lambda (to have preexe_fn have cfg_fname in context), did not work,
nested function also seems to hold context:

alternative:

       ...
       print('Invoking: {}'.format(' '.join(cmd)))
        
       def preexec_fn(): os.environ['NOTMUCH_CONFIG'] = str(cfg_fname) 

       proc = subprocess.run(cmd, timeout=5, preexec_fn=preexec_fn)
       ...

The unix fork ... (here preexec_fn called in child) ... exec model is
superior to any other alternative ! =D


Tomi

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-04 21:45   ` Tomi Ollila
  2019-11-04 22:04     ` Tomi Ollila
@ 2019-11-06  2:19     ` David Bremner
  2019-11-06 20:51       ` Tomi Ollila
  1 sibling, 1 reply; 32+ messages in thread
From: David Bremner @ 2019-11-06  2:19 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:


>
> probably
>
> test_expect_success "${NOTMUCH_PYTHON} -m pytest \
>                     --log-file=$TMP_DIRECTORY/test.output \
>                     $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage" 
>
> worked (which reminds me out-of-tree... ;)
>

despite looking prettier this does not work because it looks for the
"notdb" module in the current directory. It could work (maybe) if we set
PYTHONPATH. Would that be better than the 'cd'?

> as for checking pytest existence, the following works somewhat:
>
>    ~/test-venv/bin/python3 -m pytest --version
>
> It prints the version, then exists. Could not figure out how to request
> specific version in quick look (10 min search/trial/error session),
> parsing version output is so... :/ 

I think something like

conf=$(mktemp)
printf "[pytest]\nminversion=3.0\n" > $conf
if pytest-3 -c $conf --version >/dev/null; then
   have_pytest=1
else
   have_pytest=0
fi
rm -f $(conf)

should work.

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-05 20:47 ` Tomi Ollila
@ 2019-11-06  2:22   ` David Bremner
  2019-11-14 18:52     ` Floris Bruynooghe
  0 siblings, 1 reply; 32+ messages in thread
From: David Bremner @ 2019-11-06  2:22 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:


> alternative:
>
>        ...
>        print('Invoking: {}'.format(' '.join(cmd)))
>         
>        def preexec_fn(): os.environ['NOTMUCH_CONFIG'] = str(cfg_fname) 
>
>        proc = subprocess.run(cmd, timeout=5, preexec_fn=preexec_fn)
>        ...
>
> The unix fork ... (here preexec_fn called in child) ... exec model is
> superior to any other alternative ! =D

I don't consider myself a good judge of python style, so I'll defer to
Floris on that one.

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-06  2:19     ` David Bremner
@ 2019-11-06 20:51       ` Tomi Ollila
  2019-11-09  1:36         ` David Bremner
  0 siblings, 1 reply; 32+ messages in thread
From: Tomi Ollila @ 2019-11-06 20:51 UTC (permalink / raw)
  To: David Bremner, notmuch

On Tue, Nov 05 2019, David Bremner wrote:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>
>
>>
>> probably
>>
>> test_expect_success "${NOTMUCH_PYTHON} -m pytest \
>>                     --log-file=$TMP_DIRECTORY/test.output \
>>                     $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage" 
>>
>> worked (which reminds me out-of-tree... ;)
>>
>
> despite looking prettier this does not work because it looks for the
> "notdb" module in the current directory. It could work (maybe) if we set
> PYTHONPATH. Would that be better than the 'cd'?

So that would basically be:

  test_expect_success "PYTHONPATH=$PWD${PYTHONPATH:+:$PYTHONPATH} \
                       ${NOTMUCH_PYTHON} -m pytest \
                       --log-file=$TMP_DIRECTORY/test.output \
                       $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage"
(if it worked)

perhaps 'cd' in subshell is clearer

(i also wonder what --rootdir= does (and how new pytest is needed for that)

>> It prints the version, then exists. Could not figure out how to request
>> specific version in quick look (10 min search/trial/error session),
>> parsing version output is so... :/ 
>
> I think something like
>
> conf=$(mktemp)
> printf "[pytest]\nminversion=3.0\n" > $conf
> if pytest-3 -c $conf --version >/dev/null; then
>    have_pytest=1
> else
>    have_pytest=0
> fi
> rm -f $(conf)
>
> should work.

Right -- just that pytest-3 may not be available -- to iterate (tested)...

if ${NOTMUCH_PYTHON} -m pytest -c $conf --version >/dev/null 2>&1; then

(in this particular case redirecting both stdout and stderr is needed
 to silence noise in all 'ordinary' cases, being 
   1) pytest module not available -- silence stderr
   2) pytest version not new enough -- silence stderr
   3) pytest exists and is new enought -- silence stdout )

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-06 20:51       ` Tomi Ollila
@ 2019-11-09  1:36         ` David Bremner
  2019-11-09 21:59           ` Tomi Ollila
  0 siblings, 1 reply; 32+ messages in thread
From: David Bremner @ 2019-11-09  1:36 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:

>
> Right -- just that pytest-3 may not be available -- to iterate (tested)...
>
> if ${NOTMUCH_PYTHON} -m pytest -c $conf --version >/dev/null 2>&1; then
>

The problem with this is that it might pass if pytest is only installed
for python2. 

> (in this particular case redirecting both stdout and stderr is needed
>  to silence noise in all 'ordinary' cases, being 
>    1) pytest module not available -- silence stderr
>    2) pytest version not new enough -- silence stderr
>    3) pytest exists and is new enought -- silence stdout )

agreed

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-09  1:36         ` David Bremner
@ 2019-11-09 21:59           ` Tomi Ollila
  2019-11-22  3:03             ` David Bremner
  0 siblings, 1 reply; 32+ messages in thread
From: Tomi Ollila @ 2019-11-09 21:59 UTC (permalink / raw)
  To: David Bremner, notmuch

On Fri, Nov 08 2019, David Bremner wrote:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>
>>
>> Right -- just that pytest-3 may not be available -- to iterate (tested)...
>>
>> if ${NOTMUCH_PYTHON} -m pytest -c $conf --version >/dev/null 2>&1; then
>>
>
> The problem with this is that it might pass if pytest is only installed
> for python2. 

What is the minimum python3 version supported -- In the previous email thread all
references show python 3.7....

Anyway, in that wip/cffi "branch" commit 97caf16c15d (check for python cffi module)

could also include check for minimum python version, e.g.

+printf "Checking for python 3 cffi... "
+if "${NOTMUCH_PYTHON}" -c 'import sys,cffi; assert sys.version_info >= (3,7)' 2>/dev/null; then
+    printf "Yes.\n"
+    have_python3_cffi=1
+else
+    printf "No.\n"
+    have_python3_cffi=0
+fi

(adjust (3,7) above to any suitable smaller value =D)



Tomi

>
>> (in this particular case redirecting both stdout and stderr is needed
>>  to silence noise in all 'ordinary' cases, being 
>>    1) pytest module not available -- silence stderr
>>    2) pytest version not new enough -- silence stderr
>>    3) pytest exists and is new enought -- silence stdout )
>
> agreed
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-06  2:22   ` David Bremner
@ 2019-11-14 18:52     ` Floris Bruynooghe
  2019-11-14 20:20       ` Tomi Ollila
  0 siblings, 1 reply; 32+ messages in thread
From: Floris Bruynooghe @ 2019-11-14 18:52 UTC (permalink / raw)
  To: David Bremner, Tomi Ollila, notmuch

Hi,

Thanks for carrying on with this!

I'm a little confused with how to follow this, is the current state of
the code somewhere in a repo/branch where I can try things out and make
changes from?

On Tue 05 Nov 2019 at 22:22 -0400, David Bremner wrote:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>
>
>> alternative:
>>
>>        ...
>>        print('Invoking: {}'.format(' '.join(cmd)))
>>         
>>        def preexec_fn(): os.environ['NOTMUCH_CONFIG'] = str(cfg_fname) 
>>
>>        proc = subprocess.run(cmd, timeout=5, preexec_fn=preexec_fn)
>>        ...
>>
>> The unix fork ... (here preexec_fn called in child) ... exec model is
>> superior to any other alternative ! =D
>
> I don't consider myself a good judge of python style, so I'll defer to
> Floris on that one.

I'd have gone with what David wrote to be fair, mostly because it's
pretty simple to see what's going on where for the preexec_fn I'd have
to look at the docs.  Not sure if this has much to do with Python style
though, more with how much you like Unix tricks.


Cheers,
Floris

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-14 18:52     ` Floris Bruynooghe
@ 2019-11-14 20:20       ` Tomi Ollila
  2019-11-14 22:24         ` Floris Bruynooghe
  2019-11-16 15:43         ` David Bremner
  0 siblings, 2 replies; 32+ messages in thread
From: Tomi Ollila @ 2019-11-14 20:20 UTC (permalink / raw)
  To: notmuch

On Thu, Nov 14 2019, Floris Bruynooghe wrote:

> Hi,
>
> Thanks for carrying on with this!
>
> I'm a little confused with how to follow this, is the current state of
> the code somewhere in a repo/branch where I can try things out and make
> changes from?

In git://notmuchmail.org/git/notmuch David has ref origin/wip/cffi
which contains related changes -- You can fetch the code while waiting
for more collaboration instructions from David.

> On Tue 05 Nov 2019 at 22:22 -0400, David Bremner wrote:
>
>> Tomi Ollila <tomi.ollila@iki.fi> writes:
>>
>>
>>> alternative:
>>>
>>>        ...
>>>        print('Invoking: {}'.format(' '.join(cmd)))
>>>         
>>>        def preexec_fn(): os.environ['NOTMUCH_CONFIG'] = str(cfg_fname) 
>>>
>>>        proc = subprocess.run(cmd, timeout=5, preexec_fn=preexec_fn)
>>>        ...
>>>
>>> The unix fork ... (here preexec_fn called in child) ... exec model is
>>> superior to any other alternative ! =D
>>
>> I don't consider myself a good judge of python style, so I'll defer to
>> Floris on that one.
>
> I'd have gone with what David wrote to be fair, mostly because it's
> pretty simple to see what's going on where for the preexec_fn I'd have
> to look at the docs.  Not sure if this has much to do with Python style
> though, more with how much you like Unix tricks.

Yes, the simple approach taken (by env copy) is fine. It is also good to
know these powerful alternatives that may be useful elsewhere =D

>
>
> Cheers,
> Floris

Tomi

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-14 20:20       ` Tomi Ollila
@ 2019-11-14 22:24         ` Floris Bruynooghe
  2019-11-14 23:09           ` Floris Bruynooghe
  2019-11-16 15:43         ` David Bremner
  1 sibling, 1 reply; 32+ messages in thread
From: Floris Bruynooghe @ 2019-11-14 22:24 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

On Thu 14 Nov 2019 at 22:20 +0200, Tomi Ollila wrote:
> In git://notmuchmail.org/git/notmuch David has ref origin/wip/cffi
> which contains related changes -- You can fetch the code while waiting
> for more collaboration instructions from David.

Aha, thanks!

Floris

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-14 22:24         ` Floris Bruynooghe
@ 2019-11-14 23:09           ` Floris Bruynooghe
  2019-11-16 15:51             ` David Bremner
  0 siblings, 1 reply; 32+ messages in thread
From: Floris Bruynooghe @ 2019-11-14 23:09 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

On Thu 14 Nov 2019 at 23:24 +0100, Floris Bruynooghe wrote:

> On Thu 14 Nov 2019 at 22:20 +0200, Tomi Ollila wrote:
>> In git://notmuchmail.org/git/notmuch David has ref origin/wip/cffi
>> which contains related changes -- You can fetch the code while waiting
>> for more collaboration instructions from David.
>
> Aha, thanks!

So I can run the tests using `make` and `cd tests;
./T-391-pytest-cffi.sh`.  I can also run the tests still individually
using something like:

make
set PATH (pwd) $PATH  # I use fish
pushd bindings/python-cffi
pew workon notmuch  # how I manage python virtualenvs
pip install -e .
pip install pytest pytest-cov
pytest test/test_base.py

And all works.  I added a little function to the conftest.py to show
which notmuch it's testing with:

def pytest_report_header():
    vers = subprocess.run(['notmuch', '--version'],
                          capture_output=True, text=True)
    which = subprocess.run(['which', 'notmuch'],
                           capture_output=True, text=True)
    return ['{} ({})'.format(vers.stdout.strip(), which.stdout.strip())]

Maybe you find that useful too.

Anyway, this looks good.  Would you like some changes, e.g. the rename
to notmuch2 or so as patches?  What's the next step.



Kind of unrelated, but in my attempt to run the full notmuch test suite
(cd tests; make test) I somehow ended up with lots of weird tags in my
notmuch database and made the emacs UI pretty horrible - though it seems
I haven't lost any email.  I'm sure that was my fault somehow even
though I was just trying to follow the readme.  But if anyone has any
hints how to recover from this that could save me some time :)

Cheers,
Floris

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-14 20:20       ` Tomi Ollila
  2019-11-14 22:24         ` Floris Bruynooghe
@ 2019-11-16 15:43         ` David Bremner
  1 sibling, 0 replies; 32+ messages in thread
From: David Bremner @ 2019-11-16 15:43 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Thu, Nov 14 2019, Floris Bruynooghe wrote:
>
>> Hi,
>>
>> Thanks for carrying on with this!
>>
>> I'm a little confused with how to follow this, is the current state of
>> the code somewhere in a repo/branch where I can try things out and make
>> changes from?
>
> In git://notmuchmail.org/git/notmuch David has ref origin/wip/cffi
> which contains related changes -- You can fetch the code while waiting
> for more collaboration instructions from David.
>

That should be just a normal branch. I see

     % git ls-remote origin | grep cffi
     2a326a866ccd917f1d62bc7f2b475a0e2d26f6c8	refs/heads/wip/cffi

Anyway, for now I'm leary of reposting the whole series since we're
mainly discussing my mucking about, not Florian's largish patch.

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-14 23:09           ` Floris Bruynooghe
@ 2019-11-16 15:51             ` David Bremner
  2019-11-17 14:40               ` Floris Bruynooghe
  0 siblings, 1 reply; 32+ messages in thread
From: David Bremner @ 2019-11-16 15:51 UTC (permalink / raw)
  To: Floris Bruynooghe, Tomi Ollila, notmuch

Floris Bruynooghe <flub@devork.be> writes:
> Anyway, this looks good.  Would you like some changes, e.g. the rename
> to notmuch2 or so as patches?  What's the next step.

If you could look at the rename that would be great. Patches on top of
wip/cffi, or a ref for me to pull both sound fine.

> Kind of unrelated, but in my attempt to run the full notmuch test suite
> (cd tests; make test) I somehow ended up with lots of weird tags in my
> notmuch database and made the emacs UI pretty horrible - though it seems
> I haven't lost any email.  I'm sure that was my fault somehow even
> though I was just trying to follow the readme.  But if anyone has any
> hints how to recover from this that could save me some time :)

Hmm. I've never encountered that, but maybe it has to do with killing
the environment? The test harness uses the environment variable
NOTMUCH_CONFIG to point to a test database, and if that was deleted, it
would look in ~/.notmuch-config.

Still that would really only be tags from your pytest tests, and you'd
probably recognize those?

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-16 15:51             ` David Bremner
@ 2019-11-17 14:40               ` Floris Bruynooghe
  2019-11-17 16:09                 ` David Bremner
  0 siblings, 1 reply; 32+ messages in thread
From: Floris Bruynooghe @ 2019-11-17 14:40 UTC (permalink / raw)
  To: David Bremner, Tomi Ollila, notmuch

On Sat 16 Nov 2019 at 10:51 -0500, David Bremner wrote:

> Floris Bruynooghe <flub@devork.be> writes:
>> Anyway, this looks good.  Would you like some changes, e.g. the rename
>> to notmuch2 or so as patches?  What's the next step.
>
> If you could look at the rename that would be great. Patches on top of
> wip/cffi, or a ref for me to pull both sound fine.
>
>> Kind of unrelated, but in my attempt to run the full notmuch test suite
>> (cd tests; make test) I somehow ended up with lots of weird tags in my
>> notmuch database and made the emacs UI pretty horrible - though it seems
>> I haven't lost any email.  I'm sure that was my fault somehow even
>> though I was just trying to follow the readme.  But if anyone has any
>> hints how to recover from this that could save me some time :)
>
> Hmm. I've never encountered that, but maybe it has to do with killing
> the environment? The test harness uses the environment variable
> NOTMUCH_CONFIG to point to a test database, and if that was deleted, it
> would look in ~/.notmuch-config.

Yeah, I probably managed to make it use ~/.notmuch-config somehow.

> Still that would really only be tags from your pytest tests, and you'd
> probably recognize those?

The were a bunch of weird characters that looked like unicode
test-cases, the pytest tests don't try to test notmuch only the bindings
so doesn't really make up weird strings IIRC.  I recovered with
something like "notmuch dump | cleanup_tags.py | notmuch restore", oh
well.

I still have a bunch of non-existing messages with weird unicode message
IDs in them, but no associated filename just an empty string.  AFAIK
notmuch_database_remove_message() needs a filename though, I've tried
removing the empty file which does not fail but doesn't clear those
database entries.  The messages are not ghosts either.  Any idea how I
might be able to clear those bad messages from the database?

Cheers,
Floris

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: python CFFI bindings integration into notmuch build/test
  2019-11-17 14:40               ` Floris Bruynooghe
@ 2019-11-17 16:09                 ` David Bremner
  0 siblings, 0 replies; 32+ messages in thread
From: David Bremner @ 2019-11-17 16:09 UTC (permalink / raw)
  To: Floris Bruynooghe, Tomi Ollila, notmuch

Floris Bruynooghe <flub@devork.be> writes:

>
> I still have a bunch of non-existing messages with weird unicode message
> IDs in them, but no associated filename just an empty string.  AFAIK
> notmuch_database_remove_message() needs a filename though, I've tried
> removing the empty file which does not fail but doesn't clear those
> database entries.  The messages are not ghosts either.  Any idea how I
> might be able to clear those bad messages from the database?

In principle running "notmuch new" should do it, if there are no files
on disk corresponding to those messages. If that doesn't work, you might
have to use "xapian-delve" to figure out what terms are attached to
those message documents.

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-09 21:59           ` Tomi Ollila
@ 2019-11-22  3:03             ` David Bremner
  2019-11-26  0:52               ` David Bremner
  0 siblings, 1 reply; 32+ messages in thread
From: David Bremner @ 2019-11-22  3:03 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Fri, Nov 08 2019, David Bremner wrote:
>
>> Tomi Ollila <tomi.ollila@iki.fi> writes:
>>
>>>
>>> Right -- just that pytest-3 may not be available -- to iterate (tested)...
>>>
>>> if ${NOTMUCH_PYTHON} -m pytest -c $conf --version >/dev/null 2>&1; then
>>>
>>
>> The problem with this is that it might pass if pytest is only installed
>> for python2. 
>
> What is the minimum python3 version supported -- In the previous email thread all
> references show python 3.7....
>
> Anyway, in that wip/cffi "branch" commit 97caf16c15d (check for python cffi module)
>
> could also include check for minimum python version, e.g.
>
> +printf "Checking for python 3 cffi... "
> +if "${NOTMUCH_PYTHON}" -c 'import sys,cffi; assert sys.version_info >= (3,7)' 2>/dev/null; then
> +    printf "Yes.\n"
> +    have_python3_cffi=1
> +else
> +    printf "No.\n"
> +    have_python3_cffi=0
> +fi

I (force) pushed an updated version of wip/cffi (8d1f30cb) with several changes
along those lines to the configure / build / test scaffolding. In
particular running the tests from the notmuch suite doesn't require
pytest-cov anymore, and pytest and cffi are both checked for only if
python3 is found.

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-22  3:03             ` David Bremner
@ 2019-11-26  0:52               ` David Bremner
  2019-12-03 12:16                 ` David Bremner
  0 siblings, 1 reply; 32+ messages in thread
From: David Bremner @ 2019-11-26  0:52 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

David Bremner <david@tethera.net> writes:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>
> I (force) pushed an updated version of wip/cffi (8d1f30cb) with several changes
> along those lines to the configure / build / test scaffolding. In
> particular running the tests from the notmuch suite doesn't require
> pytest-cov anymore, and pytest and cffi are both checked for only if
> python3 is found.

Tomi rightly observed that I somehow dropped the shutil.which change. As
penance, I created a seprate commit and pushed that to wip/cffi.

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 4/5] tests: run python-cffi tests
  2019-11-26  0:52               ` David Bremner
@ 2019-12-03 12:16                 ` David Bremner
  0 siblings, 0 replies; 32+ messages in thread
From: David Bremner @ 2019-12-03 12:16 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

David Bremner <david@tethera.net> writes:


> Tomi rightly observed that I somehow dropped the shutil.which change. As
> penance, I created a seprate commit and pushed that to wip/cffi.
>
> d

I have pushed this series to master.

d

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH 3/5] build: optionally build python-cffi bindings
  2019-11-04 21:26   ` Tomi Ollila
  2019-11-05  0:32     ` David Bremner
@ 2020-05-22  1:06     ` Daniel Kahn Gillmor
  1 sibling, 0 replies; 32+ messages in thread
From: Daniel Kahn Gillmor @ 2020-05-22  1:06 UTC (permalink / raw)
  To: Tomi Ollila, David Bremner, notmuch


[-- Attachment #1.1: Type: text/plain, Size: 458 bytes --]

On Mon 2019-11-04 23:26:25 +0200, Tomi Ollila wrote:
> how bad does out-of-tree build break with this -- do we need to do
> the same as with ruby bindings (copy sources -- do we still do so)? or does
> python provide better alternative..?

I've just posted id:20200522010359.715688-1-dkg@fifthhorseman.net to
apply the same hack as the ruby bindings here.  I welcome any better
proposals, but think we should at least have the build complete!

        --dkg

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]



^ permalink raw reply	[flat|nested] 32+ messages in thread

end of thread, other threads:[~2020-05-22  1:06 UTC | newest]

Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-04  1:39 python CFFI bindings integration into notmuch build/test David Bremner
2019-11-04  1:39 ` [PATCH 1/5] bindings/python-cffi: preserve environment for tests David Bremner
2019-11-04  1:39 ` [PATCH 2/5] configure: check for python cffi module David Bremner
2019-11-04 21:24   ` Tomi Ollila
2019-11-04  1:39 ` [PATCH 3/5] build: optionally build python-cffi bindings David Bremner
2019-11-04 21:26   ` Tomi Ollila
2019-11-05  0:32     ` David Bremner
2020-05-22  1:06     ` Daniel Kahn Gillmor
2019-11-04  1:39 ` [PATCH 4/5] tests: run python-cffi tests David Bremner
2019-11-04 16:18   ` Tomi Ollila
2019-11-04 16:57     ` David Bremner
2019-11-04 21:45   ` Tomi Ollila
2019-11-04 22:04     ` Tomi Ollila
2019-11-06  2:19     ` David Bremner
2019-11-06 20:51       ` Tomi Ollila
2019-11-09  1:36         ` David Bremner
2019-11-09 21:59           ` Tomi Ollila
2019-11-22  3:03             ` David Bremner
2019-11-26  0:52               ` David Bremner
2019-12-03 12:16                 ` David Bremner
2019-11-04  1:39 ` [PATCH 5/5] travis: add python3-{cffi,pytest,setuptools} David Bremner
2019-11-04 11:10 ` python CFFI bindings integration into notmuch build/test David Bremner
2019-11-05 20:47 ` Tomi Ollila
2019-11-06  2:22   ` David Bremner
2019-11-14 18:52     ` Floris Bruynooghe
2019-11-14 20:20       ` Tomi Ollila
2019-11-14 22:24         ` Floris Bruynooghe
2019-11-14 23:09           ` Floris Bruynooghe
2019-11-16 15:51             ` David Bremner
2019-11-17 14:40               ` Floris Bruynooghe
2019-11-17 16:09                 ` David Bremner
2019-11-16 15:43         ` David Bremner

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).