[PATCH] doc: sequentialize calls to sphinx-build

[PATCH] doc: sequentialize calls to sphinx-build

[David Bremner]

In certain conditions the parallel calls to sphinx-build could
collide, yielding a crash like

Exception occurred:
  File "/usr/lib/python3/dist-packages/sphinx/environment.py", line 1261, in get_doctree
    doctree = pickle.load(f)
EOFError: Ran out of input
---

I first observed this on Debian/ppc64le. There it occured about 1 in
50 builds. Thanks to Olly Betts for pointing me in the right
direction.

 doc/Makefile.local | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index 16459e35..eed243a0 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -37,6 +37,13 @@ INFO_INFO_FILES := $(INFO_TEXI_FILES:.texi=.info)
 %.gz: %
 	rm -f $@ && gzip --stdout $^ > $@
 
+# Sequentialize the calls to sphinx-build to avoid races with
+# reading/writing state.
+
+sphinx-html: | $(DOCBUILDDIR)/.roff.stamp
+sphinx-texinfo: | sphinx-html
+sphinx-info: | sphinx-texinfo
+
 sphinx-html:
 	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(DOCBUILDDIR)/html
 
-- 
2.20.1

Re: [PATCH] doc: sequentialize calls to sphinx-build

[Matt Armstrong]

I had never seen this make syntax before so think it might be a bit
mysterious for the next person coming along.  I dug up the reference.
The GNU make info pages call them "order-only" prerequisites.  I might
comment this as:

# Prevent sphinx-build from running in parallel by using "order-only"
# prerequisites.  This avoids races when sphinx-build reads and writes
# cached state.


David Bremner <david@tethera.net> writes:

> In certain conditions the parallel calls to sphinx-build could
> collide, yielding a crash like
>
> Exception occurred:
>   File "/usr/lib/python3/dist-packages/sphinx/environment.py", line 1261, in get_doctree
>     doctree = pickle.load(f)
> EOFError: Ran out of input
> ---
>
> I first observed this on Debian/ppc64le. There it occured about 1 in
> 50 builds. Thanks to Olly Betts for pointing me in the right
> direction.
>
>  doc/Makefile.local | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/doc/Makefile.local b/doc/Makefile.local
> index 16459e35..eed243a0 100644
> --- a/doc/Makefile.local
> +++ b/doc/Makefile.local
> @@ -37,6 +37,13 @@ INFO_INFO_FILES := $(INFO_TEXI_FILES:.texi=.info)
>  %.gz: %
>  	rm -f $@ && gzip --stdout $^ > $@
>  
> +# Sequentialize the calls to sphinx-build to avoid races with
> +# reading/writing state.
> +
> +sphinx-html: | $(DOCBUILDDIR)/.roff.stamp
> +sphinx-texinfo: | sphinx-html
> +sphinx-info: | sphinx-texinfo
> +
>  sphinx-html:
>  	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(DOCBUILDDIR)/html
>  
> -- 
> 2.20.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch