[PATCH] doc: Fix parallel build of roff files

[PATCH] doc: Fix parallel build of roff files

[Austin Clements]

The roff build rule builds all of the roff files in a single command.
Previously, this was expressed as a multi-target rule, but since this
is equivalent to specifying a copy of the rule for each target, make
-jN could start up to N parallel instances of this command.  Fix this
by bottlenecking this rule through a single stamp file.

This also removes the unused man.stamp from CLEAN.
---
 doc/Makefile.local | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index 0980c71..5cf140d 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -35,7 +35,13 @@ MAN_GZIP_FILES := $(addsuffix .gz,${MAN_ROFF_FILES})
 
 # Use the man page converter that is available. We should never depend
 # on MAN_ROFF_FILES if a converter is not available.
-${MAN_ROFF_FILES}: ${MAN_RST_FILES}
+${MAN_ROFF_FILES}: $(DOCBUILDDIR)/.roff.stamp
+
+# By using $(DOCBUILDDIR)/roff.stamp instead of ${MAN_ROFF_FILES}, we
+# convey to make that a single invocation of this receipe builds all
+# of the roff files.  This prevents parallel make from starting an
+# instance of this recipe for each roff file.
+$(DOCBUILDDIR)/.roff.stamp: ${MAN_RST_FILES}
 ifeq ($(HAVE_SPHINX),1)
 	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(DOCBUILDDIR)/man
 	for section in 1 5 7; do \
@@ -48,6 +54,7 @@ else
 	@echo "Fatal: build dependency fail."
 	@false
 endif
+	touch ${MAN_ROFF_FILES} $@
 
 # Do not try to build or install man pages if a man page converter is
 # not available.
@@ -70,5 +77,5 @@ endif
 $(dir)/docdeps.mk: $(dir)/conf.py $(dir)/mkdocdeps.py
 	$(mkdocdeps) $(srcdir)/doc $(DOCBUILDDIR) $@
 
-CLEAN := $(CLEAN) $(DOCBUILDDIR) $(dir)/docdeps.mk $(dir)/man.stamp
+CLEAN := $(CLEAN) $(DOCBUILDDIR) $(dir)/docdeps.mk $(DOCBUILDDIR)/.roff.stamp
 CLEAN := $(CLEAN) $(MAN_GZIP_FILES) $(MAN_ROFF_FILES)
-- 
1.9.1

Re: [PATCH] doc: Fix parallel build of roff files

[Tomi Ollila]

On Thu, Apr 17 2014, Austin Clements <amdragon@MIT.EDU> wrote:

> The roff build rule builds all of the roff files in a single command.
> Previously, this was expressed as a multi-target rule, but since this
> is equivalent to specifying a copy of the rule for each target, make
> -jN could start up to N parallel instances of this command.  Fix this
> by bottlenecking this rule through a single stamp file.
>
> This also removes the unused man.stamp from CLEAN.
> ---
>  doc/Makefile.local | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/doc/Makefile.local b/doc/Makefile.local
> index 0980c71..5cf140d 100644
> --- a/doc/Makefile.local
> +++ b/doc/Makefile.local
> @@ -35,7 +35,13 @@ MAN_GZIP_FILES := $(addsuffix .gz,${MAN_ROFF_FILES})
>  
>  # Use the man page converter that is available. We should never depend
>  # on MAN_ROFF_FILES if a converter is not available.
> -${MAN_ROFF_FILES}: ${MAN_RST_FILES}
> +${MAN_ROFF_FILES}: $(DOCBUILDDIR)/.roff.stamp
> +
> +# By using $(DOCBUILDDIR)/roff.stamp instead of ${MAN_ROFF_FILES}, we
> +# convey to make that a single invocation of this receipe builds all

LGTM. This 'receipe' could be amended to recipe(?:)

> +# of the roff files.  This prevents parallel make from starting an
> +# instance of this recipe for each roff file.
> +$(DOCBUILDDIR)/.roff.stamp: ${MAN_RST_FILES}
>  ifeq ($(HAVE_SPHINX),1)
>  	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(DOCBUILDDIR)/man
>  	for section in 1 5 7; do \
> @@ -48,6 +54,7 @@ else
>  	@echo "Fatal: build dependency fail."
>  	@false
>  endif
> +	touch ${MAN_ROFF_FILES} $@
>  
>  # Do not try to build or install man pages if a man page converter is
>  # not available.
> @@ -70,5 +77,5 @@ endif
>  $(dir)/docdeps.mk: $(dir)/conf.py $(dir)/mkdocdeps.py
>  	$(mkdocdeps) $(srcdir)/doc $(DOCBUILDDIR) $@
>  
> -CLEAN := $(CLEAN) $(DOCBUILDDIR) $(dir)/docdeps.mk $(dir)/man.stamp
> +CLEAN := $(CLEAN) $(DOCBUILDDIR) $(dir)/docdeps.mk $(DOCBUILDDIR)/.roff.stamp
>  CLEAN := $(CLEAN) $(MAN_GZIP_FILES) $(MAN_ROFF_FILES)
> -- 
> 1.9.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] doc: Fix parallel build of roff files

[David Bremner]

Austin Clements <amdragon@MIT.EDU> writes:

> The roff build rule builds all of the roff files in a single command.
> Previously, this was expressed as a multi-target rule, but since this
> is equivalent to specifying a copy of the rule for each target, make
> -jN could start up to N parallel instances of this command.  Fix this
> by bottlenecking this rule through a single stamp file.

pushed, with comment typofixed.

d

Re: [PATCH] doc: Fix parallel build of roff files

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

On Thu, Apr 17, 2014 at 04:34:57PM -0400, Austin Clements wrote:
> +$(DOCBUILDDIR)/.roff.stamp: ${MAN_RST_FILES}
>  ifeq ($(HAVE_SPHINX),1)
>  	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(DOCBUILDDIR)/man
>  	for section in 1 5 7; do \
> @@ -48,6 +54,7 @@ else
>  	@echo "Fatal: build dependency fail."
>  	@false
>  endif
> +	touch ${MAN_ROFF_FILES} $@

I'm poking around in the Makefile, and don't understand why we're
touching ${MAN_ROFF_FILES} here.  Either they were just created by
Sphinx/rst2man, or we died with the @false.  I doubt Sphinx or
prerst2man.py are intelligent enough to not clobber the roff files if
their content hasn't changed, but I don't see a point in explicitly
bumping timestamps either.

I think I'll drop the ${MAN_ROFF_FILES} as part of the patch I'm
working on, but let me know if I'm just missing something.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] doc: Fix parallel build of roff files

[Austin Clements]

On Sat, 10 May 2014, "W. Trevor King" <wking@tremily.us> wrote:
> On Thu, Apr 17, 2014 at 04:34:57PM -0400, Austin Clements wrote:
>> +$(DOCBUILDDIR)/.roff.stamp: ${MAN_RST_FILES}
>>  ifeq ($(HAVE_SPHINX),1)
>>  	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(DOCBUILDDIR)/man
>>  	for section in 1 5 7; do \
>> @@ -48,6 +54,7 @@ else
>>  	@echo "Fatal: build dependency fail."
>>  	@false
>>  endif
>> +	touch ${MAN_ROFF_FILES} $@
>
> I'm poking around in the Makefile, and don't understand why we're
> touching ${MAN_ROFF_FILES} here.  Either they were just created by
> Sphinx/rst2man, or we died with the @false.  I doubt Sphinx or
> prerst2man.py are intelligent enough to not clobber the roff files if
> their content hasn't changed, but I don't see a point in explicitly
> bumping timestamps either.
>
> I think I'll drop the ${MAN_ROFF_FILES} as part of the patch I'm
> working on, but let me know if I'm just missing something.

I think you're right.  I can't reconstruct why I touched
${MAN_ROFF_FILES} here.  Touching $@ obviously matters, but I think it
should be safe to drop the ${MAN_ROFF_FILES}.

> Cheers,
> Trevor