[PATCH] debian: Add packaging for python3-notmuch2

[PATCH] debian: Add packaging for python3-notmuch2

[Daniel Kahn Gillmor]

Ship a new debian package for the notmuch2 CFFI-based Python interface
to notmuch.

Unlike the notmuch python module, the new notmuch2 module is no longer
arch-independent, because it builds and ships a shared object in
addition to the python code.

This patch encourages new downstream development to rely on notmuch2
instead of on notmuch, to get the benefits of the new module.

I welcome any suggested improvements to this packaging, but it appears
to me to be sufficient to get "import notmuch2" to work and do some
basic tests.
---
 debian/control | 24 +++++++++++++++++++++++-
 debian/rules   | 11 ++++++-----
 2 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/debian/control b/debian/control
index fb2b31c1..a1371fc8 100644
--- a/debian/control
+++ b/debian/control
@@ -98,6 +98,25 @@ Depends:
  libnotmuch5 (>= ${source:Version}),
  ${misc:Depends},
  ${python3:Depends},
+Description: Python 3 legacy interface to the notmuch mail search and index library
+ Notmuch is a system for indexing, searching, reading, and tagging
+ large collections of email messages in maildir or mh format. It uses
+ the Xapian library to provide fast, full-text search with a very
+ convenient search syntax.
+ .
+ This package provides a legacy Python 3 interface to the notmuch
+ functionality, directly interfacing with a shared notmuch library.
+ .
+ New projects are encouraged to use python3-notmuch2 instead.
+
+Package: python3-notmuch2
+Architecture: any
+Section: python
+Depends:
+ libnotmuch5 (>= ${source:Version}),
+ ${misc:Depends},
+ ${python3:Depends},
+ ${shlibs:Depends},
 Description: Python 3 interface to the notmuch mail search and index library
  Notmuch is a system for indexing, searching, reading, and tagging
  large collections of email messages in maildir or mh format. It uses
@@ -105,7 +124,10 @@ Description: Python 3 interface to the notmuch mail search and index library
  convenient search syntax.
  .
  This package provides a Python 3 interface to the notmuch
- functionality, directly interfacing with a shared notmuch library.
+ functionality using CFFI bindings, which interface with a shared
+ notmuch library.
+ .
+ This is the preferred way to use notmuch via Python.
 
 Package: ruby-notmuch
 Architecture: any
diff --git a/debian/rules b/debian/rules
index bf9d0bbd..8de49d0f 100755
--- a/debian/rules
+++ b/debian/rules
@@ -1,7 +1,5 @@
 #!/usr/bin/make -f
 
-export PYBUILD_NAME=notmuch
-
 export DEB_BUILD_MAINT_OPTIONS = hardening=+all
 
 %:
@@ -19,17 +17,20 @@ override_dh_auto_configure:
 
 override_dh_auto_build:
 	dh_auto_build -- V=1
-	dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python
+	PYBUILD_NAME=notmuch dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python
+	PYBUILD_NAME=notmuch2 dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python-cffi
 	$(MAKE) -C contrib/notmuch-mutt
 
 override_dh_auto_clean:
 	dh_auto_clean
-	dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python
+	PYBUILD_NAME=notmuch dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python
+	PYBUILD_NAME=notmuch2 dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python-cffi
 	dh_auto_clean --sourcedirectory bindings/ruby
 	$(MAKE) -C contrib/notmuch-mutt clean
 
 override_dh_auto_install:
 	dh_auto_install
-	dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python
+	PYBUILD_NAME=notmuch dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python
+	PYBUILD_NAME=notmuch2 dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python-cffi
 	$(MAKE) -C contrib/notmuch-mutt DESTDIR=$(CURDIR)/debian/tmp install
 	dh_auto_install --sourcedirectory bindings/ruby
-- 
2.24.0

Re: [PATCH] debian: Add packaging for python3-notmuch2

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 4680 bytes --]

On Mon 2019-12-23 17:17:22 -0500, Daniel Kahn Gillmor wrote:
> Ship a new debian package for the notmuch2 CFFI-based Python interface
> to notmuch.
>
> Unlike the notmuch python module, the new notmuch2 module is no longer
> arch-independent, because it builds and ships a shared object in
> addition to the python code.
>
> This patch encourages new downstream development to rely on notmuch2
> instead of on notmuch, to get the benefits of the new module.
>
> I welcome any suggested improvements to this packaging, but it appears
> to me to be sufficient to get "import notmuch2" to work and do some
> basic tests.
> ---
>  debian/control | 24 +++++++++++++++++++++++-
>  debian/rules   | 11 ++++++-----
>  2 files changed, 29 insertions(+), 6 deletions(-)
>
> diff --git a/debian/control b/debian/control
> index fb2b31c1..a1371fc8 100644
> --- a/debian/control
> +++ b/debian/control
> @@ -98,6 +98,25 @@ Depends:
>   libnotmuch5 (>= ${source:Version}),
>   ${misc:Depends},
>   ${python3:Depends},
> +Description: Python 3 legacy interface to the notmuch mail search and index library
> + Notmuch is a system for indexing, searching, reading, and tagging
> + large collections of email messages in maildir or mh format. It uses
> + the Xapian library to provide fast, full-text search with a very
> + convenient search syntax.
> + .
> + This package provides a legacy Python 3 interface to the notmuch
> + functionality, directly interfacing with a shared notmuch library.
> + .
> + New projects are encouraged to use python3-notmuch2 instead.
> +
> +Package: python3-notmuch2
> +Architecture: any
> +Section: python
> +Depends:
> + libnotmuch5 (>= ${source:Version}),
> + ${misc:Depends},
> + ${python3:Depends},
> + ${shlibs:Depends},
>  Description: Python 3 interface to the notmuch mail search and index library
>   Notmuch is a system for indexing, searching, reading, and tagging
>   large collections of email messages in maildir or mh format. It uses
> @@ -105,7 +124,10 @@ Description: Python 3 interface to the notmuch mail search and index library
>   convenient search syntax.
>   .
>   This package provides a Python 3 interface to the notmuch
> - functionality, directly interfacing with a shared notmuch library.
> + functionality using CFFI bindings, which interface with a shared
> + notmuch library.
> + .
> + This is the preferred way to use notmuch via Python.
>  
>  Package: ruby-notmuch
>  Architecture: any
> diff --git a/debian/rules b/debian/rules
> index bf9d0bbd..8de49d0f 100755
> --- a/debian/rules
> +++ b/debian/rules
> @@ -1,7 +1,5 @@
>  #!/usr/bin/make -f
>  
> -export PYBUILD_NAME=notmuch
> -
>  export DEB_BUILD_MAINT_OPTIONS = hardening=+all
>  
>  %:
> @@ -19,17 +17,20 @@ override_dh_auto_configure:
>  
>  override_dh_auto_build:
>  	dh_auto_build -- V=1
> -	dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python
> +	PYBUILD_NAME=notmuch dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python
> +	PYBUILD_NAME=notmuch2 dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python-cffi
>  	$(MAKE) -C contrib/notmuch-mutt
>  
>  override_dh_auto_clean:
>  	dh_auto_clean
> -	dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python
> +	PYBUILD_NAME=notmuch dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python
> +	PYBUILD_NAME=notmuch2 dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python-cffi
>  	dh_auto_clean --sourcedirectory bindings/ruby
>  	$(MAKE) -C contrib/notmuch-mutt clean
>  
>  override_dh_auto_install:
>  	dh_auto_install
> -	dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python
> +	PYBUILD_NAME=notmuch dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python
> +	PYBUILD_NAME=notmuch2 dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python-cffi
>  	$(MAKE) -C contrib/notmuch-mutt DESTDIR=$(CURDIR)/debian/tmp install
>  	dh_auto_install --sourcedirectory bindings/ruby
> -- 

Pinging on this patch as well.  If we want the notmuch2 python module to
be used, we need to get it in wider distribution.  having it installable
via "apt install python3-notmuch2" would be a good start.

Note that making this change means that the notmuch package will pass
through the NEW queue, which can take longer than we might like,
depending on the Debian FTP team's available time.  I see this as an
argument for making this change earlier, rather than later, though, so
that the NEW queue doesn't trip us up if we have a more urgent change we
want to see made later.

    --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH] debian: Add packaging for python3-notmuch2

[Antoine Beaupré]

On 2020-01-08 16:58:04, Daniel Kahn Gillmor wrote:
> On Mon 2019-12-23 17:17:22 -0500, Daniel Kahn Gillmor wrote:
>> Ship a new debian package for the notmuch2 CFFI-based Python interface
>> to notmuch.
>>
>> Unlike the notmuch python module, the new notmuch2 module is no longer
>> arch-independent, because it builds and ships a shared object in
>> addition to the python code.
>>
>> This patch encourages new downstream development to rely on notmuch2
>> instead of on notmuch, to get the benefits of the new module.
>>
>> I welcome any suggested improvements to this packaging, but it appears
>> to me to be sufficient to get "import notmuch2" to work and do some
>> basic tests.
>> ---
>>  debian/control | 24 +++++++++++++++++++++++-
>>  debian/rules   | 11 ++++++-----
>>  2 files changed, 29 insertions(+), 6 deletions(-)
>>
>> diff --git a/debian/control b/debian/control
>> index fb2b31c1..a1371fc8 100644
>> --- a/debian/control
>> +++ b/debian/control
>> @@ -98,6 +98,25 @@ Depends:
>>   libnotmuch5 (>= ${source:Version}),
>>   ${misc:Depends},
>>   ${python3:Depends},
>> +Description: Python 3 legacy interface to the notmuch mail search and index library
>> + Notmuch is a system for indexing, searching, reading, and tagging
>> + large collections of email messages in maildir or mh format. It uses
>> + the Xapian library to provide fast, full-text search with a very
>> + convenient search syntax.
>> + .
>> + This package provides a legacy Python 3 interface to the notmuch
>> + functionality, directly interfacing with a shared notmuch library.
>> + .
>> + New projects are encouraged to use python3-notmuch2 instead.
>> +
>> +Package: python3-notmuch2
>> +Architecture: any
>> +Section: python
>> +Depends:
>> + libnotmuch5 (>= ${source:Version}),
>> + ${misc:Depends},
>> + ${python3:Depends},
>> + ${shlibs:Depends},
>>  Description: Python 3 interface to the notmuch mail search and index library
>>   Notmuch is a system for indexing, searching, reading, and tagging
>>   large collections of email messages in maildir or mh format. It uses
>> @@ -105,7 +124,10 @@ Description: Python 3 interface to the notmuch mail search and index library
>>   convenient search syntax.
>>   .
>>   This package provides a Python 3 interface to the notmuch
>> - functionality, directly interfacing with a shared notmuch library.
>> + functionality using CFFI bindings, which interface with a shared
>> + notmuch library.
>> + .
>> + This is the preferred way to use notmuch via Python.
>>  
>>  Package: ruby-notmuch
>>  Architecture: any
>> diff --git a/debian/rules b/debian/rules
>> index bf9d0bbd..8de49d0f 100755
>> --- a/debian/rules
>> +++ b/debian/rules
>> @@ -1,7 +1,5 @@
>>  #!/usr/bin/make -f
>>  
>> -export PYBUILD_NAME=notmuch
>> -
>>  export DEB_BUILD_MAINT_OPTIONS = hardening=+all
>>  
>>  %:
>> @@ -19,17 +17,20 @@ override_dh_auto_configure:
>>  
>>  override_dh_auto_build:
>>  	dh_auto_build -- V=1
>> -	dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python
>> +	PYBUILD_NAME=notmuch dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python
>> +	PYBUILD_NAME=notmuch2 dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python-cffi
>>  	$(MAKE) -C contrib/notmuch-mutt
>>  
>>  override_dh_auto_clean:
>>  	dh_auto_clean
>> -	dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python
>> +	PYBUILD_NAME=notmuch dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python
>> +	PYBUILD_NAME=notmuch2 dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python-cffi
>>  	dh_auto_clean --sourcedirectory bindings/ruby
>>  	$(MAKE) -C contrib/notmuch-mutt clean
>>  
>>  override_dh_auto_install:
>>  	dh_auto_install
>> -	dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python
>> +	PYBUILD_NAME=notmuch dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python
>> +	PYBUILD_NAME=notmuch2 dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python-cffi
>>  	$(MAKE) -C contrib/notmuch-mutt DESTDIR=$(CURDIR)/debian/tmp install
>>  	dh_auto_install --sourcedirectory bindings/ruby
>> -- 
>
> Pinging on this patch as well.  If we want the notmuch2 python module to
> be used, we need to get it in wider distribution.  having it installable
> via "apt install python3-notmuch2" would be a good start.
>
> Note that making this change means that the notmuch package will pass
> through the NEW queue, which can take longer than we might like,
> depending on the Debian FTP team's available time.  I see this as an
> argument for making this change earlier, rather than later, though, so
> that the NEW queue doesn't trip us up if we have a more urgent change we
> want to see made later.

I don't of the merits of the two python libraries or how they compare. I
think i used the former (non-CFFI) version, and it worked fine. Assuming
we do want to migrate to the new one, it does seem important that we do
this change. I also agree on the NEW transition proposal.

Is the new library radically different from the older one?

What's the plan for the old one? Deprecation in bullseye and removal in
+1 kind of thing?

a.
-- 
Every one of us is, in the cosmic perspective, precious. If a human
disagrees with you, let him live. In a hundred billion galaxies, you
will not find another.   - Carl Sagan

Re: [PATCH] debian: Add packaging for python3-notmuch2

[David Bremner]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> On Mon 2019-12-23 17:17:22 -0500, Daniel Kahn Gillmor wrote:
>> Ship a new debian package for the notmuch2 CFFI-based Python interface
>> to notmuch.
>>
>> Unlike the notmuch python module, the new notmuch2 module is no longer
>> arch-independent, because it builds and ships a shared object in
>> addition to the python code.
>>

Do you understand if installed the _capi.abi3.so into /usr/lib/python3.8/...
is correct? It does seem a bit suspicious to tie it to the specific
version of python3, and lintian complains.
>
> Pinging on this patch as well.  If we want the notmuch2 python module to
> be used, we need to get it in wider distribution.  having it installable
> via "apt install python3-notmuch2" would be a good start.

We'll need an upstream notmuch release including the new bindings for
this to make a difference. It might make sense to do that soon-ish.

Re: [PATCH] debian: Add packaging for python3-notmuch2

[David Bremner]

Antoine Beaupré <anarcat@debian.org> writes:

> Is the new library radically different from the older one?

The old library memory management is broken with python after 3.5. 

> What's the plan for the old one? Deprecation in bullseye and removal in
> +1 kind of thing?
>

I'm leaning towards not packaging the old bindings in bullseye. It will
depend if clients like alot can get ported to the new bindings without
too much pain.

d

Re: [PATCH] debian: Add packaging for python3-notmuch2

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 3313 bytes --]

On Thu 2020-01-09 22:38:02 -0400, David Bremner wrote:
> Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:
>
>> On Mon 2019-12-23 17:17:22 -0500, Daniel Kahn Gillmor wrote:
>>> Ship a new debian package for the notmuch2 CFFI-based Python interface
>>> to notmuch.
>>>
>>> Unlike the notmuch python module, the new notmuch2 module is no longer
>>> arch-independent, because it builds and ships a shared object in
>>> addition to the python code.
>
> Do you understand if installed the _capi.abi3.so into /usr/lib/python3.8/...
> is correct? It does seem a bit suspicious to tie it to the specific
> version of python3, and lintian complains.

What lintian complaint do you see?  When i built it originally back in
december, there was no lintian complaint.

When i build it today, I see:

W: python3-notmuch2: python-module-in-wrong-location usr/lib/python3.8/dist-packages/notmuch2/ usr/lib/python3/dist-packages/notmuch2/
W: python3-notmuch2: python-module-in-wrong-location usr/lib/python3.8/dist-packages/notmuch2/_capi.abi3.so usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so

in the older package, i see only one copy of the .so, but in the version
built today, i see two copies (i think identical):

0 dkg@alice:~/src/notmuch/notmuch$ dpkg --contents ../python3-notmuch2_0.29.3+109~gef0d513+1_amd64.deb | grep so$
-rw-r--r-- root/root     73616 2019-12-23 17:23 ./usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so
0 dkg@alice:~/src/notmuch/notmuch$ dpkg --contents ../python3-notmuch2_0.29.3+111~g4c95a79+1_amd64.deb | grep so$
-rw-r--r-- root/root     73784 2020-01-10 15:10 ./usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so
-rw-r--r-- root/root     73784 2020-01-10 15:10 ./usr/lib/python3.8/dist-packages/notmuch2/_capi.abi3.so
0 dkg@alice:~/src/notmuch/notmuch$ 

during the more recent build, i also see this warning from dh_python3:


W: dh_python3 fs:112: Paths differ: debian/python3-notmuch2/usr/lib/python3.8/dist-packages/notmuch2/_capi.abi3.so and debian/python3-notmuch2/usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so


I confess i don't really understand what happened to change this, since
the package should be almost exactly the same source.

comparing the .buildinfo files, i see a few minor version changes that
might be relevant:

- python3 (= 3.7.5-1),
+ python3 (= 3.7.5-3),

- python3-all (= 3.7.5-1),
+ python3-all (= 3.7.5-3),

- python3.7 (= 3.7.5-2),
- python3.7-minimal (= 3.7.5-2),
- python3.8 (= 3.8.0-5),
- python3.8-minimal (= 3.8.0-5),
+ python3.7 (= 3.7.6-1),
+ python3.7-minimal (= 3.7.6-1),
+ python3.8 (= 3.8.1-2),
+ python3.8-minimal (= 3.8.1-2),

- debhelper (= 12.7.2),
+ debhelper (= 12.7.3),

they were both built with dh-python 4.20191017)

but none of these changes ought to warrant shipping an extra copy of the
.so file.

So i'm a little bit confused about what happened here. :/

>> Pinging on this patch as well.  If we want the notmuch2 python module to
>> be used, we need to get it in wider distribution.  having it installable
>> via "apt install python3-notmuch2" would be a good start.
>
> We'll need an upstream notmuch release including the new bindings for
> this to make a difference. It might make sense to do that soon-ish.

agreed.

        --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH] debian: Add packaging for python3-notmuch2

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 3214 bytes --]

On Fri 2020-01-10 16:01:35 -0500, Daniel Kahn Gillmor wrote:
> during the more recent build, i also see this warning from dh_python3:
>
> W: dh_python3 fs:112: Paths differ: debian/python3-notmuch2/usr/lib/python3.8/dist-packages/notmuch2/_capi.abi3.so and debian/python3-notmuch2/usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so

over on #debian-python, olasd helped me understand that this warning
means that dh-python noticed that the files do actually differ (that
they can't be de-duped the way that dh-python would typically de-dup
them to ensure that they live only in the generic
/usr/lib/python3/dist-packages/ location)

i inspected the files further -- they're the same size, but they appear
to differ only in NT_GNU_BUILD_ID:

0 dkg@alice:~/src/notmuch/notmuch$ sha256sum debian/python3-notmuch2/usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so debian/python3-notmuch2/usr/lib/python3.8/dist-packages/notmuch2/_capi.abi3.so
feee7627824594620b1e8095c1eabbfcc680a927e62413d4ffd0b2eb9695749c  debian/python3-notmuch2/usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so
ba05f2b5e8a376a63b9538ba72a084b11b72fc6adc0a4f7c362f3dc31ea1508e  debian/python3-notmuch2/usr/lib/python3.8/dist-packages/notmuch2/_capi.abi3.so
0 dkg@alice:~/src/notmuch/notmuch$ diffoscope debian/python3-notmuch2/usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so debian/python3-notmuch2/usr/lib/python3.8/dist-packages/notmuch2/_capi.abi3.so
--- debian/python3-notmuch2/usr/lib/python3/dist-packages/notmuch2/_capi.abi3.so
+++ debian/python3-notmuch2/usr/lib/python3.8/dist-packages/notmuch2/_capi.abi3.so
├── readelf --wide --notes {}
│ @@ -1,4 +1,4 @@
│  
│  Displaying notes found in: .note.gnu.build-id
│    Owner                Data size 	Description
│ -  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: 4de286a577ebb11464a28179ec6dc4637fce7a0b
│ +  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: 037ddaf33920a423c143d4812f3ce3fecd450953
├── readelf --wide --decompress --hex-dump=.gnu_debuglink {}
│ @@ -1,7 +1,7 @@
│  
│  Hex dump of section '.gnu_debuglink':
│ -  0x00000000 65323836 61353737 65626231 31343634 e286a577ebb11464
│ -  0x00000010 61323831 37396563 36646334 36333766 a28179ec6dc4637f
│ -  0x00000020 63653761 30622e64 65627567 00000000 ce7a0b.debug....
│ -  0x00000030 fe625d33                            .b]3
│ +  0x00000000 37646461 66333339 32306134 32336331 7ddaf33920a423c1
│ +  0x00000010 34336434 38313266 33636533 66656364 43d4812f3ce3fecd
│ +  0x00000020 34353039 35332e64 65627567 00000000 450953.debug....
│ +  0x00000030 fbdc3213                            ..2.
1 dkg@alice:~/src/notmuch/notmuch$ 


using diffoscope to compare these objects with the one i built back in
December shows that the December object has a .gnu_debugaltlink section,
but these new objects do not.  (so there appears to be a bit more
leftover debugging info in the december objects)

So i'm still stuck, but in a different place here.  i don't understand
why the build id would differ without anything else differing.

    --dkg


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH] debian: Add packaging for python3-notmuch2

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2073 bytes --]

On Fri 2020-01-10 17:01:46 -0500, Daniel Kahn Gillmor wrote:
> using diffoscope to compare these objects with the one i built back in
> December shows that the December object has a .gnu_debugaltlink section,
> but these new objects do not.  (so there appears to be a bit more
> leftover debugging info in the december objects)

Sorry, i got this precisely backward.

The difference is that the newer shared objects *have* a
.gnu_debugaltlink section, but the one built in december does not.

I took one more step at debugging the newly-built modules to try to
understand why the non-stripped versions might differ, and noticed that
the debugging info in each module itself is different.  in particular,
in the 3.7 module, the debugging info contains different paths:

 -./bindings/python-cffi/build/temp.linux-amd64-3.7/notmuch2._capi.c:1272
 +./bindings/python-cffi/build/temp.linux-amd64-3.8/notmuch2._capi.c:1272

(notmuch2._capi.c is a generated C file here, iiuc)

so maybe the build-id is being generated based on the contents of the
debug info, in addition to the contents of the stripped-down data?

If that's the case, then what's surprising to me is that i *didn't* see
this happening when i built it in december.  Very confusing, and i'd
appreciate another set of eyeballs on it.

all that debugging said and done, i don't think that this weirdness
should block us from including the package in debian.

If anything, getting it into the debian build infrastructure will expose
this weirdness to other people more directly, and might help to iron out
some more fundamental issue with either cffi itself, dh-python3's
deduplication strategies for shared objects, and/or debian's policies
about stripped data and binaries.

Shipping the extra .so in a python3-notmuch2 package doesn't appear to
be a problem for notmuch itself, because the difference between the
.so's shouldn't be relevant for any functional concerns.

If we do merge this and get it into debian, then i'll be able to open a
more specific bug report with the aforementioned packages.

     --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH] debian: Add packaging for python3-notmuch2

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 1314 bytes --]

On Fri 2020-01-10 18:16:37 -0500, Daniel Kahn Gillmor wrote:
> I took one more step at debugging the newly-built modules to try to
> understand why the non-stripped versions might differ, and noticed that
> the debugging info in each module itself is different.  in particular,
> in the 3.7 module, the debugging info contains different paths:
>
>  -./bindings/python-cffi/build/temp.linux-amd64-3.7/notmuch2._capi.c:1272
>  +./bindings/python-cffi/build/temp.linux-amd64-3.8/notmuch2._capi.c:1272
>
> (notmuch2._capi.c is a generated C file here, iiuc)
>
> so maybe the build-id is being generated based on the contents of the
> debug info, in addition to the contents of the stripped-down data?

FWIW, I think the above is indeed what is happening -- we're getting
different build-ids because the paths to the source files (to be stored
in the debug info) are different.

I talked with folks on #debian-python, and they said, basically
(paraphrasing):

  Don't worry about the fact that we're shipping two distinct .so's,
  because debian will only release with one version of python anyway.
  It's an artifact that is only relevant for testing, during the
  transition.

So i hope my public diagnosis of this concern didn't scare people off of
this patch.  We should apply it and ship it in debian.

     --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH] debian: Add packaging for python3-notmuch2

[David Bremner]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> Ship a new debian package for the notmuch2 CFFI-based Python interface
> to notmuch.

pushed.

d