macOS globals.py issue

macOS globals.py issue

[Dominyk Tiller]

[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]

Hi there,

I believe there's potentially a bug in the handling of signposting the
notmuch dynamic library when installed in non-standard prefixes. For
example, Homebrew moved their standard prefix on macOS from /usr/local
to /opt/homebrew for machines running with Apple's ARM-based chip(s),
which has caused another piece of software
(https://github.com/pazz/alot) to fail to find `notmuch.dylib` because
of the way `globals.py` handles it.

With notmuch installed in /opt/homebrew and notmuch's library available
in /opt/homebrew/lib, alot builds successfully but then aborts at
runtime because it cannot find the shared library:

```
~> alot
Traceback (most recent call last):
  File
"/opt/homebrew/opt/notmuch/lib/python3.9/site-packages/notmuch/globals.py",
line 28, in <module>
    nmlib = CDLL("libnotmuch.{0:s}.dylib".format(SOVERSION))
  File
"/opt/homebrew/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py",
line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(libnotmuch.5.dylib, 6): image not found
```

I've attached a patch that resolves the issue for me locally using
Homebrew installed in /opt/homebrew, and I expect as long as a user's
custom python was configured to search the path it was installed inside
(by modifying `DEFAULT_LIBRARY_FALLBACK` in `macholib/dyld.py`) it
should work for any custom prefix on macOS. You may well be able to
achieve the same result a different way, my python knowledge is fairly
rusty.

Hope this helps,

Dominyk
===
Sent from macOS

[-- Attachment #2: macos_find_lib.patch --]
[-- Type: text/plain, Size: 2527 bytes --]

From 013bdf454db0067005bd481ec0fa8e9ba7d546c4 Mon Sep 17 00:00:00 2001
From: Dominyk Tiller <dominyktiller@gmail.com>
Date: Mon, 19 Apr 2021 01:58:08 +0100
Subject: [PATCH] globals.py: use find_library to help locate macos dylib

This (I think!) fixes a bug in the handling of signposting the
notmuch dynamic library when installed in non-standard prefixes.
For example, Homebrew moved their standard prefix on macOS from
/usr/local to /opt/homebrew with the release of Apple's ARM-based chip,
which has caused another piece of software (https://github.com/pazz/alot)
to fail to find `notmuch.dylib` because of the way `globals.py` handles
it.

With both alot and notmuch installed in /opt/homebrew and notmuch's
library available in /opt/homebrew/lib, alot aborts at runtime because
it cannot find the shared library:

```
~> alot
Traceback (most recent call last):
  File "/opt/homebrew/opt/notmuch/lib/python3.9/site-packages/notmuch/globals.py", line 28, in <module>
    nmlib = CDLL("libnotmuch.{0:s}.dylib".format(SOVERSION))
  File "/opt/homebrew/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(libnotmuch.5.dylib, 6): image not found
```

Assuming the user/package manager/etc has configured their Python
in a sane way `find_library` should ensure non-system Pythons can find
libraries in non-standard locations, and I can at least confirm that it
fixes the problem in Homebrew's case.

Signed-off-by: Dominyk Tiller <dominyktiller@gmail.com>
---
 bindings/python/notmuch/globals.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/bindings/python/notmuch/globals.py b/bindings/python/notmuch/globals.py
index 11e328b7..64f75444 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -18,6 +18,7 @@ Copyright 2010 Sebastian Spaeth <Sebastian@SSpaeth.de>
 """
 
 from ctypes import CDLL, Structure, POINTER
+from ctypes.util import find_library
 from notmuch.version import SOVERSION
 
 #-----------------------------------------------------------------------------
@@ -25,7 +26,7 @@ from notmuch.version import SOVERSION
 try:
     from os import uname
     if uname()[0] == 'Darwin':
-        nmlib = CDLL("libnotmuch.{0:s}.dylib".format(SOVERSION))
+        nmlib = CDLL(find_library("libnotmuch.{0:s}.dylib".format(SOVERSION)))
     else:
         nmlib = CDLL("libnotmuch.so.{0:s}".format(SOVERSION))
 except:
-- 
2.31.1


[-- Attachment #3: Type: text/plain, Size: 0 bytes --]

macOS globals.py issue

[Dominyk Tiller]

[-- Attachment #1: Type: text/plain, Size: 1614 bytes --]

Hi there,

I believe there's potentially a bug in the handling of signposting the
notmuch dynamic library when installed in non-standard prefixes. For
example, Homebrew moved their standard prefix on macOS from /usr/local
to /opt/homebrew for machines running with Apple's ARM-based chip(s),
which has caused another piece of software (https://github.com/pazz/alot)
to fail to find `notmuch.dylib` because of the way `globals.py` handles it.

With notmuch installed in /opt/homebrew and notmuch's library available
in /opt/homebrew/lib, alot builds successfully but then aborts at
runtime because it cannot find the shared library:

```
~> alot
Traceback (most recent call last):
  File
"/opt/homebrew/opt/notmuch/lib/python3.9/site-packages/notmuch/globals.py",
line 28, in <module>
    nmlib = CDLL("libnotmuch.{0:s}.dylib".format(SOVERSION))
  File
"/opt/homebrew/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py",
line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(libnotmuch.5.dylib, 6): image not found
```

I've attached a patch that resolves the issue for me locally using
Homebrew installed in /opt/homebrew, and I expect as long as a user's
custom python was configured to search the path it was installed inside
(by modifying `DEFAULT_LIBRARY_FALLBACK` in `macholib/dyld.py`) it
should work for any custom prefix on macOS. You may well be able to
achieve the same result a different way, my python knowledge is fairly
rusty.

Hope this helps,

Dominyk
===
Sent from macOS


[-- Attachment #2: macos_find_lib.patch --]
[-- Type: text/plain, Size: 2527 bytes --]

From 013bdf454db0067005bd481ec0fa8e9ba7d546c4 Mon Sep 17 00:00:00 2001
From: Dominyk Tiller <dominyktiller@gmail.com>
Date: Mon, 19 Apr 2021 01:58:08 +0100
Subject: [PATCH] globals.py: use find_library to help locate macos dylib

This (I think!) fixes a bug in the handling of signposting the
notmuch dynamic library when installed in non-standard prefixes.
For example, Homebrew moved their standard prefix on macOS from
/usr/local to /opt/homebrew with the release of Apple's ARM-based chip,
which has caused another piece of software (https://github.com/pazz/alot)
to fail to find `notmuch.dylib` because of the way `globals.py` handles
it.

With both alot and notmuch installed in /opt/homebrew and notmuch's
library available in /opt/homebrew/lib, alot aborts at runtime because
it cannot find the shared library:

```
~> alot
Traceback (most recent call last):
  File "/opt/homebrew/opt/notmuch/lib/python3.9/site-packages/notmuch/globals.py", line 28, in <module>
    nmlib = CDLL("libnotmuch.{0:s}.dylib".format(SOVERSION))
  File "/opt/homebrew/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(libnotmuch.5.dylib, 6): image not found
```

Assuming the user/package manager/etc has configured their Python
in a sane way `find_library` should ensure non-system Pythons can find
libraries in non-standard locations, and I can at least confirm that it
fixes the problem in Homebrew's case.

Signed-off-by: Dominyk Tiller <dominyktiller@gmail.com>
---
 bindings/python/notmuch/globals.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/bindings/python/notmuch/globals.py b/bindings/python/notmuch/globals.py
index 11e328b7..64f75444 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -18,6 +18,7 @@ Copyright 2010 Sebastian Spaeth <Sebastian@SSpaeth.de>
 """
 
 from ctypes import CDLL, Structure, POINTER
+from ctypes.util import find_library
 from notmuch.version import SOVERSION
 
 #-----------------------------------------------------------------------------
@@ -25,7 +26,7 @@ from notmuch.version import SOVERSION
 try:
     from os import uname
     if uname()[0] == 'Darwin':
-        nmlib = CDLL("libnotmuch.{0:s}.dylib".format(SOVERSION))
+        nmlib = CDLL(find_library("libnotmuch.{0:s}.dylib".format(SOVERSION)))
     else:
         nmlib = CDLL("libnotmuch.so.{0:s}".format(SOVERSION))
 except:
-- 
2.31.1


[-- Attachment #3: Type: text/plain, Size: 0 bytes --]

Re: macOS globals.py issue

[David Bremner]

Dominyk Tiller <dominyktiller@gmail.com> writes:

> Hi there,
>
> I believe there's potentially a bug in the handling of signposting the
> notmuch dynamic library when installed in non-standard prefixes. For
> example, Homebrew moved their standard prefix on macOS from /usr/local
> to /opt/homebrew for machines running with Apple's ARM-based chip(s),
> which has caused another piece of software (https://github.com/pazz/alot)
> to fail to find `notmuch.dylib` because of the way `globals.py` handles it.
>

Hi Dominyk;

Thanks for the message. I'll have to wait for feedback from other macOS
users before knowing how to proceed.

d

Re: macOS globals.py issue

[David Bremner]

Dominyk Tiller <dominyktiller@gmail.com> writes:

> Hi there,
>
> I believe there's potentially a bug in the handling of signposting the
> notmuch dynamic library when installed in non-standard prefixes. For
> example, Homebrew moved their standard prefix on macOS from /usr/local
> to /opt/homebrew for machines running with Apple's ARM-based chip(s),
> which has caused another piece of software (https://github.com/pazz/alot)
> to fail to find `notmuch.dylib` because of the way `globals.py` handles it.
>
> With notmuch installed in /opt/homebrew and notmuch's library available
> in /opt/homebrew/lib, alot builds successfully but then aborts at
> runtime because it cannot find the shared library:
>

Since alot has now been ported to the new python bindings, I guess this
change is less urgent than before. Nonetheless if some knowlegable Mac
user wants to help review this, I'm not opposed.

d