From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 2MDlEoXJ/mCS6QAAgWs5BA (envelope-from ) for ; Mon, 26 Jul 2021 16:41:09 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id +AltDoXJ/mCOfwAAbx9fmQ (envelope-from ) for ; Mon, 26 Jul 2021 14:41:09 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 43FC3206A4 for ; Mon, 26 Jul 2021 16:41:08 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id AF220290DA; Mon, 26 Jul 2021 10:41:03 -0400 (EDT) X-Greylist: delayed 1196 seconds by postgrey-1.36 at nmbug; Mon, 26 Jul 2021 10:40:59 EDT Received: from venus.etour.fr (venus.etour.fr [188.165.44.197]) by mail.notmuchmail.org (Postfix) with ESMTPS id 85902290A4 for ; Mon, 26 Jul 2021 10:40:59 -0400 (EDT) Received: from snek.local (astrasbourg-651-1-68-19.w90-40.abo.wanadoo.fr [90.40.155.19]) (authenticated bits=0) by venus.etour.fr (8.15.2/8.15.2) with ESMTPSA id 16QEKv7R023985 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Mon, 26 Jul 2021 16:20:58 +0200 DKIM-Filter: OpenDKIM Filter v2.11.0 venus.etour.fr 16QEKv7R023985 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lange.nom.fr; s=201709; t=1627309258; bh=cpWa1vvFK53odjYtBkBzWgkqeXJ5WxMcxtsNA1PkZQY=; h=From:To:Subject:Date:From; b=ekkZ48LvD4oG6yict+eiXyvlM0XoKfkqECjo3V8jXXD5rKqJRVNqZC6nZ476z47zp aT57TNTvjK62py21ZpnXlBiX3fPb2gq1xReEJ0FM+OEhEFjJJaeuH5CYINNrhbNrv8 Crq4X57VU66/IW0TEU8E5MtZ/tD+UR7pq20J/ksU= From: Ludovic LANGE To: notmuch@notmuchmail.org Subject: [PATCH 1/1] python/notmuch2: provide binding for Database.directory() Date: Mon, 26 Jul 2021 16:20:56 +0200 Message-Id: <20210726142056.25243-1-ll-notmuchmail@lange.nom.fr> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (venus.etour.fr [188.165.44.197]); Mon, 26 Jul 2021 16:20:58 +0200 (CEST) X-lange-nom-fr-MailScanner-Information: Please contact the ISP for more information X-lange-nom-fr-MailScanner-ID: 16QEKv7R023985 X-lange-nom-fr-MailScanner: Not scanned: please contact your Internet E-Mail Service Provider for details X-lange-nom-fr-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3.099, required 6, autolearn=not spam, ALL_TRUSTED -1.00, BAYES_00 -1.90, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, DKIM_VALID_EF -0.10, URIBL_BLOCKED 0.00) X-lange-nom-fr-MailScanner-From: ll-notmuchmail@lange.nom.fr X-lange-nom-fr-MailScanner-Watermark: 1627914058.72146@K/ZELOBfuGwItvDJnp8XbA X-Spam-Status: No Message-ID-Hash: N46FC3NNT7WNI6V6QKOU3KCUJZEYRDUD X-Message-ID-Hash: N46FC3NNT7WNI6V6QKOU3KCUJZEYRDUD X-MailFrom: ll-notmuchmail@lange.nom.fr X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1627310468; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=t+In9QCpHUSLE7IfcVlCRuyBhAOMJuwgwEUtl7Z/cSk=; b=R8uBau8SvDNkcW3SRSG04p/KAld9dWtn3LiEY4cL9AshX5lKTzMvC0ZU2uuNtQaV/1s0Y7 AgS1KU99Vc+zDN7l+LKCWAcT5xHZ9WHau3gtz87c3se/DenZUV95HOWMQwG8q0wmS2XCkd jNNtndvm+5UXjmpJqRQ1Cv0CrtOpE9siE0tOfB2DygmtkoeFb69z9LqnO6iqUJ4EP2OlqE jD5CWBcpCtgPR1mblizw80IQTr3dqimn9L3TuJrE0wizPRJghhgQdwH+LV89V296BMCWpf 9FzmCw/w5SvjT227NXrUlV8YNrNX0OwWXq5ctCgVXw+Ym106o3Fwy45Dyl1WBw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1627310468; a=rsa-sha256; cv=none; b=VbTxKUjjhwMUsDiNZlwqHVrM1wxZhmmNjvOn85zc+YTgixfQ0rrpZYkjNesd1Uzrf+9du6 UXmDmZZDj9c4JlVTXFMB2cdVCQG3ylrtJm8bTTN1nMdm/oT1tULqweTv7jLP5P43Mh3KCZ ZHcBXJ4i65jTQSe9mALwdJiO9OXCW+b/E8+NesutPu41ZxIXFgQ4yIeuLa6k29WsJqJZtq /wJ7xDxIqQFuWAjtmjUwbwNoagdHAHRkLOXxfkLxbz0sfLF048lBtmRx4getzo/T2penjQ MbiE4nX1VC3LVzb4bVAJ71ZJQMPkesP0FniIX6vQ0x2McCzEhGP5GcmGK61X5A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=lange.nom.fr header.s=201709 header.b=ekkZ48Lv; dmarc=fail reason="SPF not aligned (strict)" header.from=lange.nom.fr (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 0.90 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=lange.nom.fr header.s=201709 header.b=ekkZ48Lv; dmarc=fail reason="SPF not aligned (strict)" header.from=lange.nom.fr (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 43FC3206A4 X-Spam-Score: 0.90 X-Migadu-Scanner: scn0.migadu.com X-TUID: ooJA8oIDsDZy In the legacy python bindings, database_get_directory() is a method on the database object corresponding to notmuch_database_get_directory() C API. In the (new) cffi bindings, the corresponding call was not yet implemented. We provide a naive implementation (renamed to Database.directory()), and the implementation of the needed (internal) Directory object. --- Hello, This my second try at updating the python-cffi bindings. Thanks to the feedback of @Floris and @Tomi, this new tentative includes most of the suggestions made: * Database.get_directory(..) renamed to Database.directory() * Database.directory() is properly documented * Database.directory() accepts, for its path argument, str, bytes or os.PathLike * Iterators returning directories or files are now returning Path (not PurePath) * Directory.__init__() accepts, for its path argument, str, bytes or os.PathLike * Directory class is properly documented * Directory.set_mtime() accepts int or float (uses int internally) * Directory.get_child_files() renamed to Directory.files() * Directory.files() can, optionally, return absolute paths (instead of, by default, relative paths) (*) * Directory.get_child_directories() renamed to Directory.directories() * Directory.directories() can, optionally, return absolute paths (instead of, by default, relative paths) (*) * Directory.__repr__ simplified Not (yet ?) implemented: * Suggestion to remove the code `if not hasattr(os, 'PathLike') and isinstance( path, pathlib.Path`, I'd like to understand if we should do it for all the 6 other occurences in Database or if they are special cases. May be in another patch ? * Choosing betwen getters/setters OR property for Directory.set/get/mtime - some more discussion could be helping ? (*) about relative vs absolute and default value: My use-case for this whole patch is related to [Alot MUA](https://github.com/pazz/alot) and more specifically trying to revive [an old patch](https://github.com/pazz/alot/pull/1170), which tries to display the folders in a TreeView. The previous python bindings, as well as the C-API, all return relative paths ( relative to the current Directory). Having list of a relative paths is a plus for the abovementionned patch, as we won't have to iterate on the result and basename() those paths. It'll save some (precious) time on a lot of folders. That being said, I can see an interest for having a full, workable path (also without having to convert them). So I introduced an optional parameter (defaulting to 'relative' mode) to allow the end-user to choose absolute paths. Thank you in advance for your time reviewing this patchset ! Regards, Ludovic. PS: I very slightly changed my email address since last patch, this one is the good one. PPS: Should we provide an update of the NEWS file ? bindings/python-cffi/notmuch2/__init__.py | 1 + bindings/python-cffi/notmuch2/_build.py | 18 ++ bindings/python-cffi/notmuch2/_database.py | 39 ++++- bindings/python-cffi/notmuch2/_directory.py | 174 ++++++++++++++++++++ 4 files changed, 230 insertions(+), 2 deletions(-) create mode 100644 bindings/python-cffi/notmuch2/_directory.py diff --git a/bindings/python-cffi/notmuch2/__init__.py b/bindings/python-cffi/notmuch2/__init__.py index f281edc1..32067ddc 100644 --- a/bindings/python-cffi/notmuch2/__init__.py +++ b/bindings/python-cffi/notmuch2/__init__.py @@ -45,6 +45,7 @@ usually expect from Python containers. from notmuch2 import _capi from notmuch2._base import * from notmuch2._database import * +from notmuch2._directory import * from notmuch2._errors import * from notmuch2._message import * from notmuch2._tags import * diff --git a/bindings/python-cffi/notmuch2/_build.py b/bindings/python-cffi/notmuch2/_build.py index f712b6c5..0f0a0a46 100644 --- a/bindings/python-cffi/notmuch2/_build.py +++ b/bindings/python-cffi/notmuch2/_build.py @@ -134,6 +134,10 @@ ffibuilder.cdef( notmuch_database_get_revision (notmuch_database_t *notmuch, const char **uuid); notmuch_status_t + notmuch_database_get_directory (notmuch_database_t *database, + const char *path, + notmuch_directory_t **directory); + notmuch_status_t notmuch_database_index_file (notmuch_database_t *database, const char *filename, notmuch_indexopts_t *indexopts, @@ -303,6 +307,20 @@ ffibuilder.cdef( void notmuch_tags_destroy (notmuch_tags_t *tags); + notmuch_status_t + notmuch_directory_set_mtime (notmuch_directory_t *directory, + time_t mtime); + time_t + notmuch_directory_get_mtime (notmuch_directory_t *directory); + notmuch_filenames_t * + notmuch_directory_get_child_files (notmuch_directory_t *directory); + notmuch_filenames_t * + notmuch_directory_get_child_directories (notmuch_directory_t *directory); + notmuch_status_t + notmuch_directory_delete (notmuch_directory_t *directory); + void + notmuch_directory_destroy (notmuch_directory_t *directory); + notmuch_bool_t notmuch_filenames_valid (notmuch_filenames_t *filenames); const char * diff --git a/bindings/python-cffi/notmuch2/_database.py b/bindings/python-cffi/notmuch2/_database.py index 868f4408..a42d5402 100644 --- a/bindings/python-cffi/notmuch2/_database.py +++ b/bindings/python-cffi/notmuch2/_database.py @@ -9,6 +9,7 @@ import weakref import notmuch2._base as base import notmuch2._config as config import notmuch2._capi as capi +import notmuch2._directory as directory import notmuch2._errors as errors import notmuch2._message as message import notmuch2._query as querymod @@ -337,8 +338,42 @@ class Database(base.NotmuchObject): rev = capi.lib.notmuch_database_get_revision(self._db_p, raw_uuid) return DbRevision(rev, capi.ffi.string(raw_uuid[0])) - def get_directory(self, path): - raise NotImplementedError + def directory(self, path): + """Returns a :class:`Directory` from the database given a pathname. + + If a directory with the given pathname exists in the database + return the :class:`Directory` instance for it. + Otherwise raise a :exc:`LookupError` exception. + + :param path: A path relative to the path of database (see :attr:`path`), + or else an absolute path with initial components that match the + path of database. + :type path: str, bytes, os.PathLike or pathlib.Path + :returns: :class:`Directory` or raises an exception. + :raises LookupError: The directory object referred to by ``path`` + does not exist in the database. + :raises XapianError: A Xapian exception occurred. + :raises UpgradeRequiredError: The database must be upgraded + first. + :raises OutOfMemoryError: When there is no memory to allocate + the message instance. + """ + if not hasattr(os, 'PathLike') and isinstance(path, pathlib.Path): + path = bytes(path) + directory_pp = capi.ffi.new('notmuch_directory_t **') + ret = capi.lib.notmuch_database_get_directory( + self._db_p, os.fsencode(path), directory_pp) + + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + directory_p = directory_pp[0] + if directory_p == capi.ffi.NULL: + raise LookupError + + path = pathlib.Path(path) + ret_dir = directory.Directory(path.resolve(), directory_p, self) + return ret_dir def default_indexopts(self): """Returns default index options for the database. diff --git a/bindings/python-cffi/notmuch2/_directory.py b/bindings/python-cffi/notmuch2/_directory.py new file mode 100644 index 00000000..fea11e22 --- /dev/null +++ b/bindings/python-cffi/notmuch2/_directory.py @@ -0,0 +1,174 @@ +import os +import pathlib + +import notmuch2._base as base +import notmuch2._capi as capi +import notmuch2._errors as errors +from ._message import FilenamesIter + +__all__ = ["Directory"] + + +class PathIter(FilenamesIter): + """Iterator for pathlib.Path objects.""" + + def __init__(self, parent, iter_p, basepath=""): + self._basepath = basepath + super().__init__(parent, iter_p) + + def __next__(self): + fname = super().__next__() + return pathlib.Path(self._basepath, os.fsdecode(fname)) + + +class Directory(base.NotmuchObject): + """Represents a directory entry in the notmuch database. + + This should not be directly created, instead it will be returned + by calling :meth:`Database.directory`. A directory keeps a + reference to the database object since the database object can not + be released while the directory is in use. + + Modifying attributes of this object will modify the + database, not the real directory attributes. + + :param path: The absolute path of the directory object. + :type path: str, bytes, os.PathLike or pathlib.Path + :param dir_p: The pointer to an internal notmuch_directory_t object. + :type dir_p: C-api notmuch_directory_t + :param parent: The object this Directory is derived from + (usually a :class:`Database`). We do not directly use + this, but store a reference to it as long as + this Directory object lives. This keeps the + parent object alive. + :type parent: Database + """ + + _msg_p = base.MemoryPointer() + + def __init__(self, path, dir_p, parent): + """ + """ + self._path = pathlib.Path(path) + self._dir_p = dir_p + self._parent = parent + + @property + def alive(self): + if not self._parent.alive: + return False + try: + self._dir_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def __del__(self): + """Close and free the Directory""" + self._destroy() + + def _destroy(self): + if self.alive: + capi.lib.notmuch_directory_destroy(self._dir_p) + self._dir_p = None + + def set_mtime(self, mtime): + """Sets the mtime value of this directory in the database + + The intention is for the caller to use the mtime to allow efficient + identification of new messages to be added to the database. The + recommended usage is as follows: + + * Read the mtime of a directory from the filesystem + + * Call :meth:`Database.index_file` for all mail files in + the directory + + * Call notmuch_directory_set_mtime with the mtime read from the + filesystem. Then, when wanting to check for updates to the + directory in the future, the client can call :meth:`get_mtime` + and know that it only needs to add files if the mtime of the + directory and files are newer than the stored timestamp. + + .. note:: + + :meth:`get_mtime` function does not allow the caller to + distinguish a timestamp of 0 from a non-existent timestamp. So + don't store a timestamp of 0 unless you are comfortable with + that. + + :param mtime: A POSIX timestamp + :type mtime: int or float + :raises: :exc:`XapianError` a Xapian exception occurred, mtime + not stored + :raises: :exc:`ReadOnlyDatabaseError` the database was opened + in read-only mode so directory mtime cannot be modified + """ + ret = capi.lib.notmuch_directory_set_mtime(self._dir_p, int(mtime)) + + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def get_mtime(self): + """Gets the mtime value of this directory in the database + + Retrieves a previously stored mtime for this directory. + + :returns: A POSIX timestamp + :rtype: int + """ + return capi.lib.notmuch_directory_get_mtime(self._dir_p) + + # Make mtime attribute a property of Directory() + mtime = property( + get_mtime, + set_mtime, + doc="""Property that allows getting + and setting of the Directory *mtime* (read-write) + + See :meth:`get_mtime` and :meth:`set_mtime` for usage and + possible exceptions.""", + ) + + def files(self, absolute=False): + """Gets a :class:`PathIter` iterator listing all the filenames of + messages in the database within the given directory. + + :param absolute: `True` to return complete paths, and `False` + to return basename-entries only (not complete paths), + defaults to `False` + :type absolute: boolean, optional + + :returns: An iterator yielding :class:`pathlib.Path` instances. + :rtype: PathIter + """ + fnames_p = capi.lib.notmuch_directory_get_child_files(self._dir_p) + return PathIter(self, fnames_p, self.path if absolute else "") + + def directories(self, absolute=False): + """Gets a :class:`PathIter` iterator listing all the filenames of + sub-directories in the database within the given directory. + + :param absolute: `True` to return complete paths, and `False` + to return basename-entries only (not complete paths), + defaults to `False` + :type absolute: boolean, optional + + :returns: An iterator yielding :class:`pathlib.Path` instances. + :rtype: PathIter + """ + fnames_p = capi.lib.notmuch_directory_get_child_directories(self._dir_p) + return PathIter(self, fnames_p, self.path if absolute else "") + + @property + def path(self): + """the absolute path of this Directory as a :class:`pathlib.Path` instance. + + :rtype: pathlib.Path + """ + return self._path + + def __repr__(self): + """Object representation""" + return ''.format(self=self) -- 2.32.0