bug#62762: 'make' often errors with "Org version mismatch" after pulling a new version of the code

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> The fact the Org error is triggered means that Emacs does not re-compile
>> .el files that refer to macros that have been changed.
>
> You know this is not really possible in Emacs.

I don't. Say, why not to store hash of the macro sources in the byte
code and verifying the hash when re-compilation is requested? Though I
am not that much familiar with Emacs source compilation.

>> Macros not being updated, as one may expect, may break the code using
>> old macro expansions. So, I cannot say that Org is wrong complaining
>> about mismatch here. In fact, we already made the check less strict to
>> work around the same issue on ELPA. See
>> https://list.orgmode.org/orgmode/87r10ey8ov.fsf@localhost/ and
>> https://list.orgmode.org/orgmode/87pmfijrvw.fsf@localhost/
>
> How is this different from any other Lisp file in Emacs?  We have
> quite a few that define macros used elsewhere, and we rely on
> byte-compilation during the build to tell us when the old definitions
> get in the way.  We don't forcefully abort the build, just because
> problems _might_ exist.  And people who want to make sure their builds
> are 110% perfect always bootstrap to begin with.  Building a tarball
> also requires a bootstrap, so the chance of these issues slipping
> through cracks are minuscule to say the least.

Sure, in the context of Emacs compilation. The code in question is
mostly aiming at newer Org installed on top of built-in Org. We are
consistently getting issues related to incorrect macro expansion and
mixing different Org versions.

> Anyway, any other words of wisdom?  Like which org/*.el files actually
> need to be recompiled in this case?  Or are you saying all of them
> need to be recompiled?

All of them, because org-assert-version expands into
 (unless (equal "version string" (org-release)) ...)
at compile time. "Version string" contains the return value of
(org-release) for the time the .elc file is compiled.

Note that the reported issue will only happen when Org version string
changes between the builds.

What we might do to work around the problem is detecting Emacs compilation
and disable the check. Is there a way to detect that Emacs source is
being compiled from Elisp?

> I also am not sure I understand the logic of this test: AFAIU it only
> tests the version string, not the actual macros that might have been
> changed.  Are you saying that Org bumps its version string each time
> _any_ Org macro is modified?

No. We originally tested by commit hash. Version string check is a
trade-off between the problem with ELPA builds (see the links in my
previous message) and the need to detect mixed installation problems.

> Also, how about the alternative of including the version string in
> every Org Lisp file that needs to be recompiled when the version
> changes (or the macros change)?  Then recompilation will happen
> automatically, and this problem will go away.  Does this make sense?

No. We will not be able to maintain this manually.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>