bug#62762: Incremental builds and Lisp files dependencies pulling a new version of the code

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sun, 7 May 2023 09:45:20 +0700
> Cc: monnier@iro.umontreal.ca, yantar92@posteo.net, bzg@gnu.org,
>  dmitry@gutov.dev, 62762@debbugs.gnu.org, acm@muc.de
> From: Max Nikulin <manikulin@gmail.com>
> 
> On 06/05/2023 23:26, Eli Zaretskii wrote:
> >> Date: Sat, 6 May 2023 22:50:22 +0700 From: Max Nikulin
> >>
> > No, I mean that the order in which Make builds the targets might not
> > be what you think, if all you tell it that all org/*.elc files depend
> > on org-macs.elc.
> 
> Sorry, but it is still not clear to we what "doesn't really work" and I 
> feel that it may be important for further discussion.
> 
> If a Makefile contains a dependency like
> 
> org/ol.elc: org/org-macs.elc
> 
> and org/org-macs.elc needs update then the rule for org/ol.elc is 
> executed after completion of the rule for org/org-macs.elc. The purpose 
> of dependencies is to define order of execution of build rules.

My point was that with the above rule, if org-macs.elc doesn't exist,
Make will want to produce it, even though for your purposes there's no
reason to do that.

> However I suggested dependencies on org-macs.el, not on org-macs.elc:
> 
> Sat, 6 May 2023 13:00:38 +0700
> >> they 
> >> should be at least that all lisp/org/*.elc files depend on 
> >> lisp/org/org-version.el and lisp/org/org-macs.el.

I don't really understand in enough detail what is it that you are
suggesting, because you show your suggestion piecemeal.  Can you
describe the suggestion in its entirety, top to bottom, including all
the details?  In particular, since the proposed automatic dependency
generation will be side effect of byte-compilation (right?), how do
you make sure the compilation is in the correct order when
dependencies change due to changes in the source *.el files?

Please also consider parallel builds, something that almost all of the
Emacs developers use all the time.  Thus, if N files depend on foo.el,
when foo.el changes Make will want to run N parallel jobs compiling
each one of the N dependencies.

In addition, some of the *.el files are generated, so they change as
result of changes in other *.el files.  AFAIK, automatic dependency
generation doesn't usually deal with such cases, it only deals with
dependencies on files that are not regenerated as part of the build.
How do you suggest to handle this?

And finally, some of the Lisp files are preloaded, so their changes,
and the changes in their *.elc files as result of changes in their
prerequisites, affect the Emacs binary that is used to byte-compile
the Lisp files.  If the preloaded files are outdated, many
compilations might fail just because of this aspect.

If you have an idea for solving all of these problems (which are
really just the initial tips of the iceberg), please describe your
idea with all of its details.  Otherwise, this discussion is not
really useful.