bug#62762: circular dependencies in elisp files and make

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sat, 13 May 2023 17:25:06 +0700
> Cc: monnier@iro.umontreal.ca, yantar92@posteo.net, bzg@gnu.org,
>  dmitry@gutov.dev, 62762@debbugs.gnu.org, acm@muc.de
> From: Max Nikulin <manikulin@gmail.com>
> 
> On 13/05/2023 15:46, Eli Zaretskii wrote:
> >> Date: Sat, 13 May 2023 14:34:06 +0700 From: Max Nikulin
> >>
> >> 1. A script reads dependency files (if they exist) created during
> >> previous build and removes stale .elc files.
> > 
> > This will break if the updated files have different dependencies.  You
> > need to recreate the dependencies each build.
> 
> Let's consider a.el that contains (require 'd) or it autoloads some 
> macro from d.el. If a.el changed then a.elc is removed on this stage, so 
> updated a.elc will be created on the next stage. Dependency on d.elc is 
> known from previous build, so if d.el is changed then both a.elc and 
> d.elc are removed. If new (require 'd-new) is added to a.el then it is 
> not a problem as well. a.elc is removed due to changed a.el. d-new.elc 
> is either up to date or it is removed due to changes in its dependencies 
> or in d-new.el.

I was talking about dependencies, not about stale files.  They will
need to be refreshed every build, because, unlike C sources, the *.el
files change in that aspect much more frequently.

> So no stale files should survive stage 1. The only issue with obsolete 
> dependencies is that compilation order may be not optimal.

Stale files are already not an issue, because the build runs with
load-prefer-newer set non-nil.

> > Also, it is not clear what is the plan for the macros.  If one of the
> > macros used during byte compilation changes in incompatible ways,
> > trying to byte-compile using Emacs which preloaded the previous
> > (outdated) definition of the macro will fail.
> 
> I hope, it is addressed above.

It isn't.

> >  So some changes need
> > also to generate a new Emacs binary, not just byte-compile in the
> > right order.
> 
> I expect that dependencies of the script removing stale files are 
> minimized. If changes in the breaks this script then it is time for 
> "make clean". If emacs binary depends on .elc files then some file may 
> be touched to let "make" know that the executable needs rebuild.

I don't understand how this will work, but don't let me discourage you
from trying to make it work and showing how.

> > What is the plan for the various autoloads files?  They depend on all
> > the files, and many (all?) files depend on them.
> 
> My plan is that autoloads/loaddefs are order-only dependencies for .elc 
> files. However autoloads/loaddefs files depends on all files necessary 
> to generate them.

Same here.

Thanks for working on this, anyway.  If it can work, it could be an
important change.