Racket 8 and store references (was [security] Chunked store references in .zo files in Racket 8 #47614)

Racket 8 and store references (was [security] Chunked store references in .zo files in Racket 8 #47614)

[Jack Hill]

On Tue, 6 Apr 2021, Mark H Weaver wrote:

> Anyway, I doubt that imposing such a limitation would adequately solve
> the problem here of chunked references in Racket 8, because I suspect
> that Racket 8 could split store references at arbitrary points in the
> string.  I doubt that we can safely assume that the hash component of
> store references will be stored contiguously in *.zo files.

Mark and everyone,

I wanted to spin off a subthread on guix-devel, to make you aware of 
another problem that we've run into with reference in .zo getting mangled: 
https://issues.guix.gnu.org/47180

Best,
Jack

Re: Racket 8 and store references (was [security] Chunked store references in .zo files in Racket 8 #47614)

[Philip McGrath]

Indeed, I expect this is a more precise diagnosis of the same problem. 
My patch in https://issues.guix.gnu.org/47180 solves it by putting the 
store references (search paths for foreign libraries) in a configuration 
data file that isn't compiled, so they don't end up in .zo files in the 
first place.

The .zo format is intentionally undocumented and subject to breaking 
change, including from different compilation options. At a minimum, a 
change to the Racket version number signals a breaking change to 
compiled code (e.g. Git is now at 8.0.0.13, so 13 breaking changes since 
the release). Internally, I don't know all the details, but the normal 
8.0 .zo format has a Racket layer around the Chez Scheme object format, 
which seems to be very complex: it looks like it supports 
user-configurable compression at the granularity of the individual 
object within an object file. So it seems much better to avoid rewriting 
.zo files altogether.

-Philip

On 4/6/21 9:20 PM, Jack Hill wrote:
> On Tue, 6 Apr 2021, Mark H Weaver wrote:
> 
>> Anyway, I doubt that imposing such a limitation would adequately solve
>> the problem here of chunked references in Racket 8, because I suspect
>> that Racket 8 could split store references at arbitrary points in the
>> string.  I doubt that we can safely assume that the hash component of
>> store references will be stored contiguously in *.zo files.
> 
> Mark and everyone,
> 
> I wanted to spin off a subthread on guix-devel, to make you aware of 
> another problem that we've run into with reference in .zo getting 
> mangled: https://issues.guix.gnu.org/47180
> 
> Best,
> Jack
>