[PATCH] Record dependencies during build instead of before

[PATCH] Record dependencies during build instead of before

[Austin Clements]

Previously, the makefile created dependency files in a separate, first
pass.  In particular, include-ing the dependency files would cause
make to attempt to rebuild those files using the dependency-generation
rules in the makefile.  Unfortunately, this approach required obtuse
rules and silently delayed the start of the build process (by quite a
bit on a clean tree without any dependency files).  Worse, this
required the dependency files to themselves depend on all of the
headers the source file depended on, which meant that, if a header
file was removed, the depedency file could not be updated because of a
missing dependency (!), which would cause make to silently fail.

This patch eliminates the dependency generation rules and instead
generates dependency files as a side-effect of the regular build rule.
On the first build, we don't need to know the dependencies beforehand;
the object file doesn't exist, so it will be built anyway.  On
subsequent builds, if a header file is updated, the dependency rules
generated by the previous build will force a rebuild.  If a source
file is updated, the dependency rules may be stale, but it doesn't
matter because the updated source file will force a rebuild.

In the final case above, the stale dependency rules may refer to a
header file that no longer exists but is also no longer needed.  In
order to prevent this from breaking the build, we also pass gcc the
-MP option, which generates phony targets for every depended-on header
file, so make won't complain if it can't find them during a later
build.
---
The description is way more complicated than the patch.

 Makefile.local |   18 ++++--------------
 1 files changed, 4 insertions(+), 14 deletions(-)

diff --git a/Makefile.local b/Makefile.local
index 1131dea..525eda0 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -256,23 +256,12 @@ endif
 quiet ?= $($(shell echo $1 | sed -e s'/ .*//'))
 
 %.o: %.cc $(global_deps)
+	@mkdir -p .deps/$(@D)
+	$(call quiet,CXX $(CXXFLAGS)) -c $(FINAL_CXXFLAGS) $< -o $@ -MD -MP -MF .deps/$*.d
-	$(call quiet,CXX $(CXXFLAGS)) -c $(FINAL_CXXFLAGS) $< -o $@
 
 %.o: %.c $(global_deps)
+	@mkdir -p .deps/$(@D)
+	$(call quiet,CC $(CFLAGS)) -c $(FINAL_CFLAGS) $< -o $@ -MD -MP -MF .deps/$*.d
-	$(call quiet,CC $(CFLAGS)) -c $(FINAL_CFLAGS) $< -o $@
-
-.deps/%.d: %.c $(global_deps)
-	@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
-	$(CC) -M $(CPPFLAGS) $(FINAL_CFLAGS) $< > $@.$$$$ 2>/dev/null ; \
-	sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@.$$$$ > $@; \
-	rm -f $@.$$$$
-
-.deps/%.d: %.cc $(global_deps)
-	@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
-	$(CXX) -M $(CPPFLAGS) $(FINAL_CXXFLAGS) $< > $@.$$$$ 2>/dev/null ; \
-	sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@.$$$$ > $@; \
-	rm -f $@.$$$$
 
 .PHONY : clean
 clean:
-- 
1.7.9.1

[PATCH v2] Record dependencies during build instead of before

[Austin Clements]

Previously, the makefile created dependency files in a separate, first
pass.  In particular, include-ing the dependency files would cause
make to attempt to rebuild those files using the dependency-generation
rules in the makefile.  Unfortunately, this approach required obtuse
rules and silently delayed the start of the build process (by quite a
bit on a clean tree without any dependency files).  Worse, this
required the dependency files to themselves depend on all of the
headers the source file depended on, which meant that, if a header
file was removed, the depedency file could not be updated because of a
missing dependency (!), which would cause make to silently fail.

This patch eliminates the dependency generation rules and instead
generates dependency files as a side-effect of the regular build rule.
On the first build, we don't need to know the dependencies beforehand;
the object file doesn't exist, so it will be built anyway.  On
subsequent builds, if a header file is updated, the dependency rules
generated by the previous build will force a rebuild.  If a source
file is updated, the dependency rules may be stale, but it doesn't
matter because the updated source file will force a rebuild.

In the final case above, the stale dependency rules may refer to a
header file that no longer exists but is also no longer needed.  In
order to prevent this from breaking the build, we also pass gcc the
-MP option, which generates phony targets for every depended-on header
file, so make won't complain if it can't find them during a later
build.
---
Sorry, the previous version of this patch was corrupted.  Hopefully
this one will be correct.

 Makefile.local |   18 ++++--------------
 1 files changed, 4 insertions(+), 14 deletions(-)

diff --git a/Makefile.local b/Makefile.local
index 1131dea..525eda0 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -256,22 +256,12 @@ endif
 quiet ?= $($(shell echo $1 | sed -e s'/ .*//'))
 
 %.o: %.cc $(global_deps)
+	@mkdir -p .deps/$(@D)
+	$(call quiet,CXX $(CXXFLAGS)) -c $(FINAL_CXXFLAGS) $< -o $@ -MD -MP -MF .deps/$*.d
-	$(call quiet,CXX $(CXXFLAGS)) -c $(FINAL_CXXFLAGS) $< -o $@
 
 %.o: %.c $(global_deps)
+	@mkdir -p .deps/$(@D)
+	$(call quiet,CC $(CFLAGS)) -c $(FINAL_CFLAGS) $< -o $@ -MD -MP -MF .deps/$*.d
-	$(call quiet,CC $(CFLAGS)) -c $(FINAL_CFLAGS) $< -o $@
-
-.deps/%.d: %.c $(global_deps)
-	@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
-	$(CC) -M $(CPPFLAGS) $(FINAL_CFLAGS) $< > $@.$$$$ 2>/dev/null ; \
-	sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@.$$$$ > $@; \
-	rm -f $@.$$$$
-
-.deps/%.d: %.cc $(global_deps)
-	@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
-	$(CXX) -M $(CPPFLAGS) $(FINAL_CXXFLAGS) $< > $@.$$$$ 2>/dev/null ; \
-	sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@.$$$$ > $@; \
-	rm -f $@.$$$$
 
 .PHONY : clean
 clean:
-- 
1.7.9.1

Re: [PATCH v2] Record dependencies during build instead of before

[Tomi Ollila]

On Wed, Apr 11 2012, Austin Clements <amdragon@MIT.EDU> wrote:

> Previously, the makefile created dependency files in a separate, first
> pass.  In particular, include-ing the dependency files would cause
> make to attempt to rebuild those files using the dependency-generation
> rules in the makefile.  Unfortunately, this approach required obtuse
> rules and silently delayed the start of the build process (by quite a
> bit on a clean tree without any dependency files).  Worse, this
> required the dependency files to themselves depend on all of the
> headers the source file depended on, which meant that, if a header
> file was removed, the depedency file could not be updated because of a
> missing dependency (!), which would cause make to silently fail.
>
> This patch eliminates the dependency generation rules and instead
> generates dependency files as a side-effect of the regular build rule.
> On the first build, we don't need to know the dependencies beforehand;
> the object file doesn't exist, so it will be built anyway.  On
> subsequent builds, if a header file is updated, the dependency rules
> generated by the previous build will force a rebuild.  If a source
> file is updated, the dependency rules may be stale, but it doesn't
> matter because the updated source file will force a rebuild.
>
> In the final case above, the stale dependency rules may refer to a
> header file that no longer exists but is also no longer needed.  In
> order to prevent this from breaking the build, we also pass gcc the
> -MP option, which generates phony targets for every depended-on header
> file, so make won't complain if it can't find them during a later
> build.
> ---

Looks good, patch applies and works (at least the cases I tested and
examined). And is definitely better than before.

+1

Tomi 

> Sorry, the previous version of this patch was corrupted.  Hopefully
> this one will be correct.
>
>  Makefile.local |   18 ++++--------------
>  1 files changed, 4 insertions(+), 14 deletions(-)
>

Re: [PATCH v2] Record dependencies during build instead of before

[David Bremner]

Austin Clements <amdragon@MIT.EDU> writes:

> This patch eliminates the dependency generation rules and instead
> generates dependency files as a side-effect of the regular build rule.

pushed,

d