Re: [Emacs-diffs] master d826037 3/3: Remove the need for temacs.in

["Daniel Colascione" <dancol@dancol.org>]

> Daniel Colascione wrote:
>
>>> what exactly is the failure scenario here? I'm not seeing any
>>> failure scenarios for the current approach that can't also be failure
>>> scenarios for the previous approach
>>
>> It's likely that the linker to lay out objects in a section identically
>> in
>> two different builds when the only difference between the builds is the
>> content of an array.
>
> Sure, but it's even more likely for linkers to lay out objects in a
> section
> identically in two different builds when there is no difference whatsoever
> between the inputs to the builds. So I'm still not seeing the failure
> scenario
> for the current approach that wouldn't also be a failure scenario for the
> previous (temacs.in) approach.

You're not capturing all the inputs into the build. What about the linker
itself? What about arguments? What about the environment? Consider a
difference in section alignment between two versions of a linker. Consider
LTO, in which the differences may be even more drastic, since in this
case, object files contain an IR and not even machine code. What are you
going to do, fold into the hash the linker itself and all the code and
data on which it depends?

>> If we're worried about the array being folded into
>> the code, we can make it volatile.
>
> That wouldn't be enough; we'd need the volatile accesses to memory under
> the
> program's control being tricky enough (they aren't now) so that the
> compiler
> couldn't optimize them away or reorder the array or whatever. Admittedly
> this is
> getting a little theoretical (but then this particular point is pretty
> theoretical anyway :-).

It is theoretical: I agree. Whether or not the array is folded into
program code doesn't matter.

>> If someone changes a linker flag that
>> changes object ordering, temacs.in will change.
>
> Right, but that can also affect temacs in the previous approach; that is,
> temacs.in might be linked with different flags than temacs is. Or the
> linker
> might be upgraded between the time that temacs.in is built, and the time
> that
> temacs is built. So these failure scenarios apply to the previous approach
> too.

No, the temacs.in approach is *not* broken. temacs.in doesn't have to be
identical to temacs in order for the scheme to work. The only requirement
is that if something in the build environment or in Emacs itself changes
incompatibly, then temacs.in changes. If the temacs.in scheme *were* to
break, it would have to be in such a way that 1) some change in Emacs or
the environment resulted in temacs.in_1 (before the change) and
temacs.in_2 (after the change) having the same hash, but temacs_1 (before
the change) and temacs_2 (after the change) being different in a way that
breaks pdumper. Can you think of such a scenario? I prefer the temacs.in
scheme because it's  agnostic to whatever it is that the linker might be
doing. It's future-proof.

> If we rely on a fingerprint we have to give up on the idea of an ironclad
> guarantee that if the fingerprint matches, Emacs is compatible. We have to
> settle for just a high-enough probability in practice.
>
> We could document ways in which the low-probability events can occur (hash
> collision, linker change that breaks reproducible builds, etc.). Or we
> could
> change the pdumper so that it doesn't rely on a fingerprint: instead,
> Emacs
> could record a complete description of what it's assuming in the dump
> file, and
> check this description when it reads the dump back in. However, that'd be
> some
> work and is almost surely overkill.


If you want to address the build time issue, just rewrite the fingerprint
in place. I very strongly suspect that a simple volatile declaration will
be sufficient to ensure that the fingerprint array is contiguous in the
binary. We could even locate the array in temacs.in via brute-force
search, substituting a well-known highly-unlikely byte sequence as the
dummy fingerprint.