python-cffi: read version number from notmuch

python-cffi: read version number from notmuch

[Floris Bruynooghe]

This reads the version from the toplevel notmuch version file.
The main assumption is obviously that setup.py is always in
bindings/python-cffi/setup.py together with the rest of the
notmuch git repo.

[PATCH] python-cffi: read version from notmuch version file

[Floris Bruynooghe]

This keeps it in sync with the main notmuch version which is less
confusing to users.
---
 bindings/python-cffi/setup.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/bindings/python-cffi/setup.py b/bindings/python-cffi/setup.py
index 37918e3d..1effcfc6 100644
--- a/bindings/python-cffi/setup.py
+++ b/bindings/python-cffi/setup.py
@@ -1,9 +1,17 @@
+import pathlib
+
 import setuptools
 
 
+THIS_FILE = pathlib.Path(__file__).absolute()
+PROJECT_ROOT = THIS_FILE.parent.parent.parent
+with open(PROJECT_ROOT.joinpath('version')) as fp:
+    VERSION = fp.read().strip()
+
+
 setuptools.setup(
     name='notmuch2',
-    version='0.1',
+    version=VERSION,
     description='Pythonic bindings for the notmuch mail database using CFFI',
     author='Floris Bruynooghe',
     author_email='flub@devork.be',
-- 
2.27.0

Re: [PATCH] python-cffi: read version from notmuch version file

[David Bremner]

Floris Bruynooghe <flub@devork.be> writes:

> This keeps it in sync with the main notmuch version which is less
> confusing to users.

ah, that's much nicer than what I did for the old bindings. merged.

BTW I noticed something (setuptools?) translates "0.30~rc2" to
"0.30-rc2". I assume that is as intended, and there are some stricter
rules for python module versions. It should only affect pre-release
versions in any case.

d

Re: [PATCH] python-cffi: read version from notmuch version file

[Floris Bruynooghe]

On Fri 19 Jun 2020 at 07:20 -0300, David Bremner wrote:

> Floris Bruynooghe <flub@devork.be> writes:
> BTW I noticed something (setuptools?) translates "0.30~rc2" to
> "0.30-rc2". I assume that is as intended, and there are some stricter
> rules for python module versions. It should only affect pre-release
> versions in any case.

Supposedly setuptools uses packaging.version:

   $ python3
   Python 3.8.3 (default, May 14 2020, 11:03:12)
   [GCC 9.3.0] on linux
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import packaging.version
   >>> rc1 = packaging.version.parse('0.30~rc1')
   >>> rc1
   <LegacyVersion('0.30~rc1')>
   >>> rc2 = packaging.version.parse('0.30~rc2')
   >>> maj = packaging.version.parse('0.30')
   >>> maj
   <Version('0.30')>
   >>> rc1 < rc2 < maj
   True

So I think this is ok?  Just for completeness:

   >>> rc1b = packaging.version.parse('0.30-rc1')
   >>> rc1b
   <Version('0.30rc1')>
   >>> rc1b < maj
   True

Apparently PEP440 has the full details of how Python wants to see
version numbers work, but it's too many years ago that I read that
thing ;).  This seems good enough to me.

Cheers,
Floris

Re: [PATCH] python-cffi: read version from notmuch version file

[Frank LENORMAND]

On Fri Jun 19 12:46:28 2020, Floris Bruynooghe wrote:
> This keeps it in sync with the main notmuch version which is less
> confusing to users.
> ---
>  bindings/python-cffi/setup.py | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/bindings/python-cffi/setup.py b/bindings/python-cffi/setup.py
> index 37918e3d..1effcfc6 100644
> --- a/bindings/python-cffi/setup.py
> +++ b/bindings/python-cffi/setup.py
> @@ -1,9 +1,17 @@
> +import pathlib
> +
>  import setuptools
>  
>  
> +THIS_FILE = pathlib.Path(__file__).absolute()
> +PROJECT_ROOT = THIS_FILE.parent.parent.parent
> +with open(PROJECT_ROOT.joinpath('version')) as fp:
> +    VERSION = fp.read().strip()
> +
> +
>  setuptools.setup(
>      name='notmuch2',
> -    version='0.1',
> +    version=VERSION,
>      description='Pythonic bindings for the notmuch mail database using CFFI',
>      author='Floris Bruynooghe',
>      author_email='flub@devork.be',
> -- 
> 2.27.0

It seems that this strategy doesn't work well when the user runs
`pip install .` in the `bindings/python-cffi` directory.

Apparently all the files are copied to a temporary directory first:

https://travis-ci.com/github/pazz/alot/jobs/351377760#L708-L710

It doesn't happen with the original bindings, probably because the version
number is stored in `bindings/python/notmuch/version.py`, which is also
copied when `pip` runs.

Regards,
-- 
Frank LENORMAND

Re: [PATCH] python-cffi: read version from notmuch version file

[Floris Bruynooghe]

On Fri 19 Jun 2020 at 15:26 +0300, Frank LENORMAND wrote:

> On Fri Jun 19 12:46:28 2020, Floris Bruynooghe wrote:
>> This keeps it in sync with the main notmuch version which is less
>> confusing to users.
>> ---
>>  bindings/python-cffi/setup.py | 10 +++++++++-
>>  1 file changed, 9 insertions(+), 1 deletion(-)
>> 
>> diff --git a/bindings/python-cffi/setup.py b/bindings/python-cffi/setup.py
>> index 37918e3d..1effcfc6 100644
>> --- a/bindings/python-cffi/setup.py
>> +++ b/bindings/python-cffi/setup.py
>> @@ -1,9 +1,17 @@
>> +import pathlib
>> +
>>  import setuptools
>>  
>>  
>> +THIS_FILE = pathlib.Path(__file__).absolute()
>> +PROJECT_ROOT = THIS_FILE.parent.parent.parent
>> +with open(PROJECT_ROOT.joinpath('version')) as fp:
>> +    VERSION = fp.read().strip()
>> +
>> +
>>  setuptools.setup(
>>      name='notmuch2',
>> -    version='0.1',
>> +    version=VERSION,
>>      description='Pythonic bindings for the notmuch mail database using CFFI',
>>      author='Floris Bruynooghe',
>>      author_email='flub@devork.be',
>> -- 
>> 2.27.0
>
> It seems that this strategy doesn't work well when the user runs
> `pip install .` in the `bindings/python-cffi` directory.
>
> Apparently all the files are copied to a temporary directory first:
>
> https://travis-ci.com/github/pazz/alot/jobs/351377760#L708-L710
>
> It doesn't happen with the original bindings, probably because the version
> number is stored in `bindings/python/notmuch/version.py`, which is also
> copied when `pip` runs.

Ouch, I only tested pip install -e, which does work.  But indeed a plain
pip install no longer works which is pretty bad.

I guess we could either revert this and do the same sed hackery as the
other bindings, or copy the version file into bindings/python-cffi and
have it loaded in the same way as now.  It would still have to be kept
in sync there sadly.

Re: [PATCH] python-cffi: read version from notmuch version file

[Frank LENORMAND]

On Tue Jun 23 00:45:01 2020, Floris Bruynooghe wrote:
> On Fri 19 Jun 2020 at 15:26 +0300, Frank LENORMAND wrote:
> > It seems that this strategy doesn't work well when the user runs
> > `pip install .` in the `bindings/python-cffi` directory.
> >
> > Apparently all the files are copied to a temporary directory first:
> >
> > https://travis-ci.com/github/pazz/alot/jobs/351377760#L708-L710
> >
> > It doesn't happen with the original bindings, probably because the version
> > number is stored in `bindings/python/notmuch/version.py`, which is also
> > copied when `pip` runs.
> 
> Ouch, I only tested pip install -e, which does work.  But indeed a plain
> pip install no longer works which is pretty bad.
> 
> I guess we could either revert this and do the same sed hackery as the
> other bindings, or copy the version file into bindings/python-cffi and
> have it loaded in the same way as now.  It would still have to be kept
> in sync there sadly.

Actually, instead of having the version of the bindings follow that of the
main repository, why not have a separate one, but based on the latter?

For example, 0.30.1, with the first two numbers coming from the main
repository, and the last one acting as major for the bindings.

0.29.3 → 0.29.1
0.30-rc2 → 0.30.1-rc2
etc.

That way, users can still detect changes to the bindings that don't require
them to re-install the entire repository. You can also differentiate two
iterations of the bindings that work with the same NotMuch install.

You don't have to do any `sed` magic, but you will have to manually maintain
the version number in the bindings' directories.

Regards,
-- 
Frank LENORMAND

Re: [PATCH] python-cffi: read version from notmuch version file

[David Bremner]

Frank LENORMAND <lenormf.ml@gmail.com> writes:
> For example, 0.30.1, with the first two numbers coming from the main
> repository, and the last one acting as major for the bindings.
>
> 0.29.3 → 0.29.1
> 0.30-rc2 → 0.30.1-rc2
> etc.
>

I'm mainly interested in supporting two use cases for notmuch: building
everything from source, and binary packages of released versions. We've
already gone to some trouble to tell Emacs users that try to mix and
match versions that they are on their own, and this seems to apply even
more strongly to bindings users.
   
With that said, if Floris thinks some hierarchical version is useful,
and is willing to maintain it, I can live with it. I would ask that:

1) You keep the whole "upstream" version number. So the first example
would be 0.29.3.1.  0.29.1 is a previous version of notmuch, and that
ambiguity can only cause trouble.

2) You don't insert things in the middle. So the second example would be
0.30-rc2.1

3) You have some way to distinguish between the notmuch version 0.30.1,
and the bindings version 0.30(.1) . I'd suggest using something
different than '.'  as a separator, but I don't know what the python
toolchain will tolerate.

Re: [PATCH] python-cffi: read version from notmuch version file

[Frank LENORMAND]

On Tue Jun 23 12:33:36 2020, David Bremner wrote:
> Frank LENORMAND <lenormf.ml@gmail.com> writes:
> > For example, 0.30.1, with the first two numbers coming from the main
> > repository, and the last one acting as major for the bindings.
> >
> > 0.29.3 → 0.29.1
> > 0.30-rc2 → 0.30.1-rc2
> > etc.
> >
> 
> I'm mainly interested in supporting two use cases for notmuch: building
> everything from source, and binary packages of released versions. We've
> already gone to some trouble to tell Emacs users that try to mix and
> match versions that they are on their own, and this seems to apply even
> more strongly to bindings users.
>    
> With that said, if Floris thinks some hierarchical version is useful,
> and is willing to maintain it, I can live with it. I would ask that:
> 
> 1) You keep the whole "upstream" version number. So the first example
> would be 0.29.3.1.  0.29.1 is a previous version of notmuch, and that
> ambiguity can only cause trouble.

The idea was that the bindings will work with the X.Y version they were
released for, since the last component in X.Y.Z is for minor changes that
shouldn't affect the API.

So we can keep X.Y from NotMuch itself, and append some information that
hint at the state of the bindings.

> 2) You don't insert things in the middle. So the second example would be
> 0.30-rc2.1

The -rc2 applies to the release of the whole project, so it applies to the
bindings as well. It can safely be placed at the back, because in the
current state of things, modifications to the bindings will cause the RC
number to increase as well.

> 3) You have some way to distinguish between the notmuch version 0.30.1,
> and the bindings version 0.30(.1) . I'd suggest using something
> different than '.'  as a separator, but I don't know what the python
> toolchain will tolerate.

That is confusing. But I don't think using a 4+ parts long version number
is relevant, because the only information we need from the base version
number are the X.Y components.

If cutting the base version number is not an acceptable solution, and using
one with 4+ components isn't either, the only other sane choice is a
completely different one, whose major component is incremented along with
the project's. Not too bad, it will just not be self-evident which bindings
were shipped with which release of NotMuch.

Or the exact same version number, but then what should happen to it when
the bindings are modified, but not NotMuch?

Regards,
-- 
Frank LENORMAND

Re: [PATCH] python-cffi: read version from notmuch version file

[Floris Bruynooghe]

On Tue 23 Jun 2020 at 13:43 +0300, Frank LENORMAND wrote:

> On Tue Jun 23 12:33:36 2020, David Bremner wrote:
>> Frank LENORMAND <lenormf.ml@gmail.com> writes:
>> > For example, 0.30.1, with the first two numbers coming from the main
>> > repository, and the last one acting as major for the bindings.
>> >
>> > 0.29.3 → 0.29.1
>> > 0.30-rc2 → 0.30.1-rc2
>> > etc.
>> >
>> 
>> I'm mainly interested in supporting two use cases for notmuch: building
>> everything from source, and binary packages of released versions. We've
>> already gone to some trouble to tell Emacs users that try to mix and
>> match versions that they are on their own, and this seems to apply even
>> more strongly to bindings users.
>>    
>> With that said, if Floris thinks some hierarchical version is useful,
>> and is willing to maintain it, I can live with it. I would ask that:
>> 
>> 1) You keep the whole "upstream" version number. So the first example
>> would be 0.29.3.1.  0.29.1 is a previous version of notmuch, and that
>> ambiguity can only cause trouble.
>
> The idea was that the bindings will work with the X.Y version they were
> released for, since the last component in X.Y.Z is for minor changes that
> shouldn't affect the API.

Minor nitpicking, but API is not strong enough here, you'd need to
ensure ABI compatibility.

> So we can keep X.Y from NotMuch itself, and append some information that
> hint at the state of the bindings.
[...]
> Or the exact same version number, but then what should happen to it when
> the bindings are modified, but not NotMuch?

If it was bad enough to need a new release then I guess everyone gets
the same version bump as the entire project gets a bugfix release?

I honestly like the simplicity of just having the same version number
and not having to think about maintaining it separately.  It also means
we mostly don't have to worry about how setuptools/pip is going to view
the version number.

The only way I think this could break is if we want to break backwards
compatibility in the bindings, but we're not supposed to do that
(realistically an impossible task in Python if you ask me, but we can
aim for at least avoiding doing this knowingly).

The most likely version number sin is that the python bindings get a new
feature while libnotmuch only gets bugfix.  I also don't think this is
terrible, but that's perhaps unusual and frowned upon.  Maybe this
warrants a README in the bindings to warn the version number just tracks
libnotmuch and as far as python goes can only be used to order the
releases.

Re: [PATCH] python-cffi: read version from notmuch version file

[Frank LENORMAND]

On Wed Jun 24 00:16:35 2020, Floris Bruynooghe wrote:
> On Tue 23 Jun 2020 at 13:43 +0300, Frank LENORMAND wrote:
> > On Tue Jun 23 12:33:36 2020, David Bremner wrote:
> >> Frank LENORMAND <lenormf.ml@gmail.com> writes:
> >> > For example, 0.30.1, with the first two numbers coming from the main
> >> > repository, and the last one acting as major for the bindings.
> >> >
> >> > 0.29.3 → 0.29.1
> >> > 0.30-rc2 → 0.30.1-rc2
> >> > etc.
> >> >
> >> 
> >> I'm mainly interested in supporting two use cases for notmuch: building
> >> everything from source, and binary packages of released versions. We've
> >> already gone to some trouble to tell Emacs users that try to mix and
> >> match versions that they are on their own, and this seems to apply even
> >> more strongly to bindings users.
> >>    
> >> With that said, if Floris thinks some hierarchical version is useful,
> >> and is willing to maintain it, I can live with it. I would ask that:
> >> 
> >> 1) You keep the whole "upstream" version number. So the first example
> >> would be 0.29.3.1.  0.29.1 is a previous version of notmuch, and that
> >> ambiguity can only cause trouble.
> >
> > The idea was that the bindings will work with the X.Y version they were
> > released for, since the last component in X.Y.Z is for minor changes that
> > shouldn't affect the API.
> 
> Minor nitpicking, but API is not strong enough here, you'd need to
> ensure ABI compatibility.
> 
> > So we can keep X.Y from NotMuch itself, and append some information that
> > hint at the state of the bindings.
> [...]
> > Or the exact same version number, but then what should happen to it when
> > the bindings are modified, but not NotMuch?
> 
> If it was bad enough to need a new release then I guess everyone gets
> the same version bump as the entire project gets a bugfix release?
> 
> I honestly like the simplicity of just having the same version number
> and not having to think about maintaining it separately.  It also means
> we mostly don't have to worry about how setuptools/pip is going to view
> the version number.
> 
> The only way I think this could break is if we want to break backwards
> compatibility in the bindings, but we're not supposed to do that
> (realistically an impossible task in Python if you ask me, but we can
> aim for at least avoiding doing this knowingly).
> 
> The most likely version number sin is that the python bindings get a new
> feature while libnotmuch only gets bugfix.  I also don't think this is
> terrible, but that's perhaps unusual and frowned upon.  Maybe this
> warrants a README in the bindings to warn the version number just tracks
> libnotmuch and as far as python goes can only be used to order the
> releases.

Fair enough. I've never been in that situation before, so brainstorming
about it was fun :)

I'll keep an eye out for a patch.

Regards,
-- 
Frank LENORMAND