unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* doc build warnings when building out-of-tree
@ 2020-05-26 17:03 Daniel Kahn Gillmor
  2020-05-30 16:28 ` David Bremner
  0 siblings, 1 reply; 6+ messages in thread
From: Daniel Kahn Gillmor @ 2020-05-26 17:03 UTC (permalink / raw)
  To: Notmuch Mail


[-- Attachment #1.1: Type: text/plain, Size: 7367 bytes --]

When building out-of-tree, the documentation (both manpages and info
files) are incomplete, but they do not explicitly fail.  build logs
follow from doing "mkdir build && ../configure && make".

If there's a way to make these warnings into hard failures, i think that
would be good -- we don't want to accidentally ship the output they
create.  And of course actually fixing it would be even better.

I'm afraid i'm not proficient enough in sphinx to know how to do either,
so i'm just reporting a bug for now.

    --dkg

WITH_EMACS=1 sphinx-build -b man -d doc/_build/man_doctrees -q ../doc doc/_build/man
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:383: WARNING: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:385: WARNING: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-lib.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:387: WARNING: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-show.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:389: WARNING: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-tag.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:220: WARNING: Undefined substitution referenced: "docstring::notmuch-message-headers".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:223: WARNING: Undefined substitution referenced: "docstring::notmuch-message-headers-visible".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:236: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-filename".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:239: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-git-send-email".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:242: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-message-id-stripped".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:245: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-mlarchive-link-and-go".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:248: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-tags".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:251: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-cc".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:254: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-date".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:257: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-from".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:260: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-message-id".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:263: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-mlarchive-link".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:266: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-subject".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:269: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-to".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:347: WARNING: Undefined substitution referenced: "docstring::notmuch-tagging-keys".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:353: WARNING: Undefined substitution referenced: "docstring::notmuch-cycle-notmuch-buffers".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:364: WARNING: Undefined substitution referenced: "docstring::notmuch-poll".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:367: WARNING: Undefined substitution referenced: "docstring::notmuch-poll-script".
[…]


WITH_EMACS=1 sphinx-build -b texinfo -d doc/_build/texinfo_doctrees -q ../doc doc/_build/texinfo
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:383: WARNING: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:385: WARNING: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-lib.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:387: WARNING: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-show.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:389: WARNING: Problems with "include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-tag.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:220: WARNING: Undefined substitution referenced: "docstring::notmuch-message-headers".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:223: WARNING: Undefined substitution referenced: "docstring::notmuch-message-headers-visible".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:236: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-filename".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:239: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-git-send-email".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:242: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-message-id-stripped".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:245: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-mlarchive-link-and-go".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:248: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-tags".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:251: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-cc".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:254: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-date".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:257: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-from".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:260: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-message-id".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:263: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-mlarchive-link".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:266: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-subject".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:269: WARNING: Undefined substitution referenced: "docstring::notmuch-show-stash-to".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:347: WARNING: Undefined substitution referenced: "docstring::notmuch-tagging-keys".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:353: WARNING: Undefined substitution referenced: "docstring::notmuch-cycle-notmuch-buffers".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:364: WARNING: Undefined substitution referenced: "docstring::notmuch-poll".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:367: WARNING: Undefined substitution referenced: "docstring::notmuch-poll-script".

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: doc build warnings when building out-of-tree
  2020-05-26 17:03 doc build warnings when building out-of-tree Daniel Kahn Gillmor
@ 2020-05-30 16:28 ` David Bremner
  2020-05-31 12:26   ` David Bremner
  0 siblings, 1 reply; 6+ messages in thread
From: David Bremner @ 2020-05-30 16:28 UTC (permalink / raw)
  To: Daniel Kahn Gillmor, Notmuch Mail

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> When building out-of-tree, the documentation (both manpages and info
> files) are incomplete, but they do not explicitly fail.  build logs
> follow from doing "mkdir build && ../configure && make".
>
> If there's a way to make these warnings into hard failures, i think that
> would be good -- we don't want to accidentally ship the output they
> create.  And of course actually fixing it would be even better.
>
> I'm afraid i'm not proficient enough in sphinx to know how to do either,
> so i'm just reporting a bug for now.
>

I can't duplicate these errors with sphinx-doc 2.4.3-3 on Debian.

Are the rsti files actually being built before calling sphinx-build?

d

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: doc build warnings when building out-of-tree
  2020-05-30 16:28 ` David Bremner
@ 2020-05-31 12:26   ` David Bremner
  2020-05-31 16:15     ` [PATCH] doc: fix for out-of-tree builds of notmuch-emacs docs David Bremner
  0 siblings, 1 reply; 6+ messages in thread
From: David Bremner @ 2020-05-31 12:26 UTC (permalink / raw)
  To: Daniel Kahn Gillmor, Notmuch Mail

David Bremner <david@tethera.net> writes:

> Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:
>
> I can't duplicate these errors with sphinx-doc 2.4.3-3 on Debian.
>
> Are the rsti files actually being built before calling sphinx-build?
>
> d

OK, I understand now. The case I thought was working was actually using
the rsti files from a previous in-tree build.

I think the quick and dirty fix is to add '-W' to the sphinx-build
line. I'll see if I can find a better fix. It looks like the issue is
using relative paths in the rst. In the out of tree build those are
still resolving relative to the source file (reasonably enough). Maybe
we can use the environment variable NOTMUCH_BUILDDIR in the config
file.

d

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH] doc: fix for out-of-tree builds of notmuch-emacs docs
  2020-05-31 12:26   ` David Bremner
@ 2020-05-31 16:15     ` David Bremner
  2020-05-31 20:29       ` Tomi Ollila
  0 siblings, 1 reply; 6+ messages in thread
From: David Bremner @ 2020-05-31 16:15 UTC (permalink / raw)
  To: David Bremner, Daniel Kahn Gillmor, Notmuch Mail

The sphinx-doc include directive does not have the ability to include
files from the build tree, so we replace the include with reading the
files in conf.py. The non-trivial downside of this is that the emacs
docstrings are now defined for every rst source file. They are
namespaced with docstring::, so hopefully there will not be any
surprises. One thing that is noticable is a small (absolute) time
penalty in running sphinx-doc.
---
 doc/Makefile.local    |  2 +-
 doc/conf.py           | 19 +++++++++++++++----
 doc/notmuch-emacs.rst | 10 ----------
 3 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index b4e0c955..769438ed 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -4,7 +4,7 @@ dir := doc
 
 # You can set these variables from the command line.
 SPHINXOPTS    := -q
-SPHINXBUILD   = WITH_EMACS=${WITH_EMACS} sphinx-build
+SPHINXBUILD   = WITH_EMACS=${WITH_EMACS} RSTI_DIR=$(realpath emacs) sphinx-build
 DOCBUILDDIR      := $(dir)/_build
 
 # Internal variables.
diff --git a/doc/conf.py b/doc/conf.py
index fc9738ff..d8841d99 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -29,10 +29,21 @@ release = version
 # directories to ignore when looking for source files.
 exclude_patterns = ['_build']
 
-# If we don't have emacs (or the user configured --without-emacs),
-# don't build the notmuch-emacs docs, as they need emacs to generate
-# the docstring include files
-if os.environ.get('WITH_EMACS') != '1':
+if os.environ.get('WITH_EMACS') == '1':
+    # Hacky reimplementation of include to workaround limitations of
+    # sphinx-doc
+    accumulator='.. include:: /../emacs/rstdoc.rsti\n\n' # in the source tree
+    rsti_dir=os.environ.get('RSTI_DIR')
+    # the other files are from the build tree
+    for file in ['notmuch.rsti', 'notmuch-lib.rsti', 'notmuch-show.rsti', 'notmuch-tag.rsti']:
+        with open(rsti_dir+'/'+file) as f:
+            for line in f:
+                accumulator += line
+    rst_epilog=accumulator
+else:
+    # If we don't have emacs (or the user configured --without-emacs),
+    # don't build the notmuch-emacs docs, as they need emacs to generate
+    # the docstring include files
     exclude_patterns.append('notmuch-emacs.rst')
 
 # The name of the Pygments (syntax highlighting) style to use.
diff --git a/doc/notmuch-emacs.rst b/doc/notmuch-emacs.rst
index 1655e2f0..de47b726 100644
--- a/doc/notmuch-emacs.rst
+++ b/doc/notmuch-emacs.rst
@@ -377,13 +377,3 @@ suffix exist it will be read instead (just one of these, chosen in this
 order). Most often users create ``~/.emacs.d/notmuch-config.el`` and just
 work with it. If Emacs was invoked with the ``-q`` or ``--no-init-file``
 options, ``notmuch-init-file`` is not read.
-
-.. include:: ../emacs/rstdoc.rsti
-
-.. include:: ../emacs/notmuch.rsti
-
-.. include:: ../emacs/notmuch-lib.rsti
-
-.. include:: ../emacs/notmuch-show.rsti
-
-.. include:: ../emacs/notmuch-tag.rsti
-- 
2.26.2

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] doc: fix for out-of-tree builds of notmuch-emacs docs
  2020-05-31 16:15     ` [PATCH] doc: fix for out-of-tree builds of notmuch-emacs docs David Bremner
@ 2020-05-31 20:29       ` Tomi Ollila
  2020-06-01 12:12         ` David Bremner
  0 siblings, 1 reply; 6+ messages in thread
From: Tomi Ollila @ 2020-05-31 20:29 UTC (permalink / raw)
  To: David Bremner, Daniel Kahn Gillmor, Notmuch Mail

On Sun, May 31 2020, David Bremner wrote:

> The sphinx-doc include directive does not have the ability to include
> files from the build tree, so we replace the include with reading the
> files in conf.py. The non-trivial downside of this is that the emacs
> docstrings are now defined for every rst source file. They are
> namespaced with docstring::, so hopefully there will not be any
> surprises. One thing that is noticable is a small (absolute) time
> penalty in running sphinx-doc.

IMO good enough tolerable hack to get this done for no (read: I don't
know alternatives).

Some comments below...

> ---
>  doc/Makefile.local    |  2 +-
>  doc/conf.py           | 19 +++++++++++++++----
>  doc/notmuch-emacs.rst | 10 ----------
>  3 files changed, 16 insertions(+), 15 deletions(-)
>
> diff --git a/doc/Makefile.local b/doc/Makefile.local
> index b4e0c955..769438ed 100644
> --- a/doc/Makefile.local
> +++ b/doc/Makefile.local
> @@ -4,7 +4,7 @@ dir := doc
>  
>  # You can set these variables from the command line.
>  SPHINXOPTS    := -q
> -SPHINXBUILD   = WITH_EMACS=${WITH_EMACS} sphinx-build
> +SPHINXBUILD   = WITH_EMACS=${WITH_EMACS} RSTI_DIR=$(realpath emacs) sphinx-build

I was about to comment the above, before realized $(realpath ...) executes
GNU Make function instead of system command :) ...

>  DOCBUILDDIR      := $(dir)/_build
>  
>  # Internal variables.
> diff --git a/doc/conf.py b/doc/conf.py
> index fc9738ff..d8841d99 100644
> --- a/doc/conf.py
> +++ b/doc/conf.py
> @@ -29,10 +29,21 @@ release = version
>  # directories to ignore when looking for source files.
>  exclude_patterns = ['_build']
>  
> -# If we don't have emacs (or the user configured --without-emacs),
> -# don't build the notmuch-emacs docs, as they need emacs to generate
> -# the docstring include files
> -if os.environ.get('WITH_EMACS') != '1':
> +if os.environ.get('WITH_EMACS') == '1':
> +    # Hacky reimplementation of include to workaround limitations of
> +    # sphinx-doc
> +    accumulator='.. include:: /../emacs/rstdoc.rsti\n\n' # in the source tree

accumulator=['.. include:: /../emacs/rstdoc.rsti\n\n' # in the source tree]

(in list, see below)

> +    rsti_dir=os.environ.get('RSTI_DIR')

spaces around '='

> +    # the other files are from the build tree
> +    for file in ['notmuch.rsti', 'notmuch-lib.rsti', 'notmuch-show.rsti', 'notmuch-tag.rsti']:

tuple ('notmuch.rsti', 'notmuch-lib.rsti', ...) could have been used

> +        with open(rsti_dir+'/'+file) as f:
> +            for line in f:
> +                accumulator += line
> +    rst_epilog=accumulator

           accumulator.append(open(rsti_dir + '/' + file).read())
       accumulator = ''.join(accumulator)
       rst_epilog = accumulator

alternative last 2 lines

       rst_epilog = ''.join(accumulator)
       del accumulator

(I personally would have left 'accumulator' away and used 'rst_epilog'
directly (and perhaps commented its use as temporary list type).)


> +else:
> +    # If we don't have emacs (or the user configured --without-emacs),
> +    # don't build the notmuch-emacs docs, as they need emacs to generate
> +    # the docstring include files
>      exclude_patterns.append('notmuch-emacs.rst')
>  
>  # The name of the Pygments (syntax highlighting) style to use.

Tomi

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] doc: fix for out-of-tree builds of notmuch-emacs docs
  2020-05-31 20:29       ` Tomi Ollila
@ 2020-06-01 12:12         ` David Bremner
  0 siblings, 0 replies; 6+ messages in thread
From: David Bremner @ 2020-06-01 12:12 UTC (permalink / raw)
  To: Tomi Ollila, Daniel Kahn Gillmor, Notmuch Mail

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Sun, May 31 2020, David Bremner wrote:
>
>> The sphinx-doc include directive does not have the ability to include
>> files from the build tree, so we replace the include with reading the
>> files in conf.py. The non-trivial downside of this is that the emacs
>> docstrings are now defined for every rst source file. They are
>> namespaced with docstring::, so hopefully there will not be any
>> surprises. One thing that is noticable is a small (absolute) time
>> penalty in running sphinx-doc.
>
> IMO good enough tolerable hack to get this done for no (read: I don't
> know alternatives).
>
> Some comments below...

Pushed a version with most of Tomi's comments applied (and
s/append/extend/)

d

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-06-01 12:12 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-26 17:03 doc build warnings when building out-of-tree Daniel Kahn Gillmor
2020-05-30 16:28 ` David Bremner
2020-05-31 12:26   ` David Bremner
2020-05-31 16:15     ` [PATCH] doc: fix for out-of-tree builds of notmuch-emacs docs David Bremner
2020-05-31 20:29       ` Tomi Ollila
2020-06-01 12:12         ` David Bremner

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).