unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* [WIP PATCH 0/1] autodoc failed to import notmuch2
@ 2021-10-21  9:51 Michael J Gruber
  2021-10-21  9:51 ` [WIP PATCH 1/1] autodoc: fix search path for notmuch2 module Michael J Gruber
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Michael J Gruber @ 2021-10-21  9:51 UTC (permalink / raw)
  To: notmuch

Hi there

During Fedora package builds I noticed that autodoc fails to import the
notmuch2 module:

```
WARNING: autodoc: failed to import module 'notmuch2'; the following
exception was raised:
No module named 'notmuch2'
```

You never notice this on your own box where notmuch2 is installed,
because then the build finds the previously installed module. But this
is wrong, of course, because autodoc is supposed to document tthe module
being build (in-source module).

There are actually several problems leading to this:

1. search path
The module residing in `bindings/python-cffi/notmuch2/__init__.py` needs
a search path `bindings/python-cffi/`, not `bindings/python-cffi/notmuch2`.

If we fix just that we get:
```
WARNING: autodoc: failed to import module 'notmuch2'; the following
exception was raised:
cannot import name '_capi' from partially initialized module 'notmuch2'
(most likely due to a circular import)
(/builddir/build/BUILD/notmuch-0.34/doc/../bindings/python-cffi/notmuch2/__init__.py)
```

2. search path, again
The module is built in `bindings/python-cffi/build/stage`, that is where
`_capi` is to be found, and PATCH 1/1 fixes the path in `doc/conf.py`.

3. build order
The patch from 2. actually helps only if we build in the following
order (make targets):
notmuch notmuch-shared ruby-bindings python-cffi-bindings
all

In particular, the bindings need to be built before the docs

I can do this using two make calls, but a proper solution would specify
the dependencies in the makefile.

4. library search path

When you do 1. through 3. then autodoc almost manages to import
notmuch2: it finds the proper module, _capi is found, but then
libnotmuch.so is not found because LD_LIBRARY_PATH would need to be
amended to find the freshly build library.

Now, maybe I should simply ignore the autodoc warning. Or I should
change my packaging to build twice (make && make install && make clean
&& make && make install). At least I wanted to raise attention to this.

This problem (if it is one) is not specific to 0.34, it was the same
at least with 0.33 but - being just a warning - was unnoticed.

Cheers
Michael

Michael J Gruber (1):
  autodoc: fix search path for notmuch2 module

 doc/conf.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.33.0.1146.g7ac282475c

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [WIP PATCH 1/1] autodoc: fix search path for notmuch2 module
  2021-10-21  9:51 [WIP PATCH 0/1] autodoc failed to import notmuch2 Michael J Gruber
@ 2021-10-21  9:51 ` Michael J Gruber
  2021-10-21 12:20 ` [WIP PATCH 0/1] autodoc failed to import notmuch2 David Bremner
  2021-10-29 14:54 ` [PATCH] doc/python-cffi: import from built bindings, not installed module David Bremner
  2 siblings, 0 replies; 6+ messages in thread
From: Michael J Gruber @ 2021-10-21  9:51 UTC (permalink / raw)
  To: notmuch

a05da455 ("doc: add new python bindings to main documentation tree.", 2020-07-11)
added the path "bindings/python-cffi/notmuch2" to the search path for
autodoc. But the notmuch2 module cannot be imported from there - the dir
"is" the module, and it resides in "bindings/python-cffi" and in
"bindings/python-cffi/build/stage", with only the latter location having
the compiled capi part.

This goes unnoticed as long as you have notmuch2 in your standard search
path, but it means autodocs finds the wrong module, not the one one we
are building the doc against.

Fix the path so that the build tree module comes first.

Signed-off-by: Michael J Gruber <git@grubix.eu>
---
 doc/conf.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/conf.py b/doc/conf.py
index 1fbd102b..c7fd8f5a 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -25,7 +25,7 @@ for pathdir in ['.', '..']:
             version=infile.read().replace('\n','')
 
 # for autodoc
-sys.path.insert(0, os.path.join(location, '..', 'bindings', 'python-cffi', 'notmuch2'))
+sys.path.insert(0, os.path.join(location, '..', 'bindings', 'python-cffi', 'build', 'stage'))
 
 # read generated config
 for pathdir in ['.', '..']:
-- 
2.33.0.1146.g7ac282475c

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [WIP PATCH 0/1] autodoc failed to import notmuch2
  2021-10-21  9:51 [WIP PATCH 0/1] autodoc failed to import notmuch2 Michael J Gruber
  2021-10-21  9:51 ` [WIP PATCH 1/1] autodoc: fix search path for notmuch2 module Michael J Gruber
@ 2021-10-21 12:20 ` David Bremner
  2021-10-21 12:36   ` Michael J Gruber
  2021-10-29 14:54 ` [PATCH] doc/python-cffi: import from built bindings, not installed module David Bremner
  2 siblings, 1 reply; 6+ messages in thread
From: David Bremner @ 2021-10-21 12:20 UTC (permalink / raw)
  To: Michael J Gruber, notmuch

Michael J Gruber <git@grubix.eu> writes:

> Hi there
>
> During Fedora package builds I noticed that autodoc fails to import the
> notmuch2 module:
>
> ```
> WARNING: autodoc: failed to import module 'notmuch2'; the following
> exception was raised:
> No module named 'notmuch2'
> ```
>
> You never notice this on your own box where notmuch2 is installed,
> because then the build finds the previously installed module. But this
> is wrong, of course, because autodoc is supposed to document tthe module
> being build (in-source module).
>

I also see this error when doing builds in a clean environment for
Debian. I guess I'm currently not using the docs from those builds, so I
don't know how bad the damage is.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [WIP PATCH 0/1] autodoc failed to import notmuch2
  2021-10-21 12:20 ` [WIP PATCH 0/1] autodoc failed to import notmuch2 David Bremner
@ 2021-10-21 12:36   ` Michael J Gruber
  2021-10-21 13:02     ` David Bremner
  0 siblings, 1 reply; 6+ messages in thread
From: Michael J Gruber @ 2021-10-21 12:36 UTC (permalink / raw)
  To: David Bremner, notmuch

David Bremner venit, vidit, dixit 2021-10-21 14:20:11:
> Michael J Gruber <git@grubix.eu> writes:
> 
> > Hi there
> >
> > During Fedora package builds I noticed that autodoc fails to import the
> > notmuch2 module:
> >
> > ```
> > WARNING: autodoc: failed to import module 'notmuch2'; the following
> > exception was raised:
> > No module named 'notmuch2'
> > ```
> >
> > You never notice this on your own box where notmuch2 is installed,
> > because then the build finds the previously installed module. But this
> > is wrong, of course, because autodoc is supposed to document tthe module
> > being build (in-source module).
> >
> 
> I also see this error when doing builds in a clean environment for
> Debian. I guess I'm currently not using the docs from those builds, so I
> don't know how bad the damage is.

In that case: Which docs do you use, and which docs do you want us
(packagers) to use? The usual fedora policy is to (re)build docs if
the doc source is shipped.

Disclaimer: I have no clue about autodoc ;)

Michael

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [WIP PATCH 0/1] autodoc failed to import notmuch2
  2021-10-21 12:36   ` Michael J Gruber
@ 2021-10-21 13:02     ` David Bremner
  0 siblings, 0 replies; 6+ messages in thread
From: David Bremner @ 2021-10-21 13:02 UTC (permalink / raw)
  To: Michael J Gruber, notmuch

Michael J Gruber <git@grubix.eu> writes:

>
> In that case: Which docs do you use, and which docs do you want us
> (packagers) to use? The usual fedora policy is to (re)build docs if
> the doc source is shipped.
>
> Disclaimer: I have no clue about autodoc ;)

I agree with this in general. Currently Debian does not ship the docs. I
also don't know anything about autodoc, so curently I'm just deploying
locally built docs on notmuchmail.org.

d

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH] doc/python-cffi: import from built bindings, not installed module
  2021-10-21  9:51 [WIP PATCH 0/1] autodoc failed to import notmuch2 Michael J Gruber
  2021-10-21  9:51 ` [WIP PATCH 1/1] autodoc: fix search path for notmuch2 module Michael J Gruber
  2021-10-21 12:20 ` [WIP PATCH 0/1] autodoc failed to import notmuch2 David Bremner
@ 2021-10-29 14:54 ` David Bremner
  2 siblings, 0 replies; 6+ messages in thread
From: David Bremner @ 2021-10-29 14:54 UTC (permalink / raw)
  To: Michael J Gruber, notmuch

Previously the python-cffi bindings either failed to build, or built
for the wrong module by using the installed module.

The fix requires correction the module path, building the bindings
before docs, and helping python find the built libnotmuch.

Based on patch / discussion from Micheal Gruber [1]

[1]: id:cover.1634808719.git.git@grubix.eu
---
 doc/Makefile.local | 12 ++++++++----
 doc/conf.py        |  2 +-
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index f476d1da..730ad4fb 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -4,7 +4,7 @@ dir := doc
 
 # You can set these variables from the command line.
 SPHINXOPTS    := -q
-SPHINXBUILD   = sphinx-build
+SPHINXBUILD   = env LD_LIBRARY_PATH=${NOTMUCH_BUILDDIR}/lib sphinx-build
 DOCBUILDDIR      := $(dir)/_build
 
 # Internal variables.
@@ -35,7 +35,7 @@ endif
 
 INFO_INFO_FILES := $(INFO_TEXI_FILES:.texi=.info)
 
-.PHONY: sphinx-html sphinx-texinfo sphinx-info
+.PHONY: sphinx-html sphinx-texinfo sphinx-info doc-prereqs
 
 .PHONY: install-man build-man apidocs install-apidocs
 
@@ -46,15 +46,19 @@ ifeq ($(WITH_EMACS),1)
 $(DOCBUILDDIR)/.roff.stamp $(DOCBUILDDIR)/.html.stamp $(DOCBUILDDIR)/.texi.stamp : docstring.stamp
 endif
 
+ifeq ($(HAVE_PYTHON3_CFFI),1)
+doc-prereqs: python-cffi-bindings
+endif
+
 sphinx-html: $(DOCBUILDDIR)/.html.stamp
 
-$(DOCBUILDDIR)/.html.stamp: $(ALL_RST_FILES)
+$(DOCBUILDDIR)/.html.stamp: $(ALL_RST_FILES) doc-prereqs
 	$(SPHINXBUILD) -b html -d $(DOCBUILDDIR)/html_doctrees $(ALLSPHINXOPTS) $(DOCBUILDDIR)/html
 	touch $@
 
 sphinx-texinfo: $(DOCBUILDDIR)/.texi.stamp
 
-$(DOCBUILDDIR)/.texi.stamp: $(ALL_RST_FILES)
+$(DOCBUILDDIR)/.texi.stamp: $(ALL_RST_FILES) doc-prereqs
 	$(SPHINXBUILD) -b texinfo -d $(DOCBUILDDIR)/texinfo_doctrees $(ALLSPHINXOPTS) $(DOCBUILDDIR)/texinfo
 	touch $@
 
diff --git a/doc/conf.py b/doc/conf.py
index 1fbd102b..c7fd8f5a 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -25,7 +25,7 @@ for pathdir in ['.', '..']:
             version=infile.read().replace('\n','')
 
 # for autodoc
-sys.path.insert(0, os.path.join(location, '..', 'bindings', 'python-cffi', 'notmuch2'))
+sys.path.insert(0, os.path.join(location, '..', 'bindings', 'python-cffi', 'build', 'stage'))
 
 # read generated config
 for pathdir in ['.', '..']:
-- 
2.33.0

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-10-29 14:55 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-21  9:51 [WIP PATCH 0/1] autodoc failed to import notmuch2 Michael J Gruber
2021-10-21  9:51 ` [WIP PATCH 1/1] autodoc: fix search path for notmuch2 module Michael J Gruber
2021-10-21 12:20 ` [WIP PATCH 0/1] autodoc failed to import notmuch2 David Bremner
2021-10-21 12:36   ` Michael J Gruber
2021-10-21 13:02     ` David Bremner
2021-10-29 14:54 ` [PATCH] doc/python-cffi: import from built bindings, not installed module David Bremner

Code repositories for project(s) associated with this inbox:

	notmuch.git.git (no URL configured)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).