all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / Atom feed
* Racket 8 and store references (was [security] Chunked store references in .zo files in Racket 8 #47614)
@ 2021-04-07  1:20 Jack Hill
  2021-04-07  1:38 ` Philip McGrath
  0 siblings, 1 reply; 6+ messages in thread
From: Jack Hill @ 2021-04-07  1:20 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: guix-devel

On Tue, 6 Apr 2021, Mark H Weaver wrote:

> Anyway, I doubt that imposing such a limitation would adequately solve
> the problem here of chunked references in Racket 8, because I suspect
> that Racket 8 could split store references at arbitrary points in the
> string.  I doubt that we can safely assume that the hash component of
> store references will be stored contiguously in *.zo files.

Mark and everyone,

I wanted to spin off a subthread on guix-devel, to make you aware of 
another problem that we've run into with reference in .zo getting mangled: 
https://issues.guix.gnu.org/47180

Best,
Jack


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Racket 8 and store references (was [security] Chunked store references in .zo files in Racket 8 #47614)
  2021-04-07  1:20 Racket 8 and store references (was [security] Chunked store references in .zo files in Racket 8 #47614) Jack Hill
@ 2021-04-07  1:38 ` Philip McGrath
  2021-04-07  1:48   ` bug#47614: [security] Chunked store references in .zo files in Racket 8 #47614 Philip McGrath
  0 siblings, 1 reply; 6+ messages in thread
From: Philip McGrath @ 2021-04-07  1:38 UTC (permalink / raw)
  To: Jack Hill, Mark H Weaver; +Cc: guix-devel

Indeed, I expect this is a more precise diagnosis of the same problem. 
My patch in https://issues.guix.gnu.org/47180 solves it by putting the 
store references (search paths for foreign libraries) in a configuration 
data file that isn't compiled, so they don't end up in .zo files in the 
first place.

The .zo format is intentionally undocumented and subject to breaking 
change, including from different compilation options. At a minimum, a 
change to the Racket version number signals a breaking change to 
compiled code (e.g. Git is now at 8.0.0.13, so 13 breaking changes since 
the release). Internally, I don't know all the details, but the normal 
8.0 .zo format has a Racket layer around the Chez Scheme object format, 
which seems to be very complex: it looks like it supports 
user-configurable compression at the granularity of the individual 
object within an object file. So it seems much better to avoid rewriting 
.zo files altogether.

-Philip

On 4/6/21 9:20 PM, Jack Hill wrote:
> On Tue, 6 Apr 2021, Mark H Weaver wrote:
> 
>> Anyway, I doubt that imposing such a limitation would adequately solve
>> the problem here of chunked references in Racket 8, because I suspect
>> that Racket 8 could split store references at arbitrary points in the
>> string.  I doubt that we can safely assume that the hash component of
>> store references will be stored contiguously in *.zo files.
> 
> Mark and everyone,
> 
> I wanted to spin off a subthread on guix-devel, to make you aware of 
> another problem that we've run into with reference in .zo getting 
> mangled: https://issues.guix.gnu.org/47180
> 
> Best,
> Jack
> 


^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#47614: [security] Chunked store references in .zo files in Racket 8 #47614
  2021-04-07  1:38 ` Philip McGrath
@ 2021-04-07  1:48   ` Philip McGrath
  2021-04-16 15:46     ` bug#47614: [security] Chunked store references in .zo files in Racket 8 Ludovic Courtès
  0 siblings, 1 reply; 6+ messages in thread
From: Philip McGrath @ 2021-04-07  1:48 UTC (permalink / raw)
  To: 47614

Ah, I see the thread for https://issues.guix.gnu.org/47614 wasn't cc'ed 
here:


-------- Forwarded Message --------
Subject: Re: Racket 8 and store references (was [security] Chunked store 
references in .zo files in Racket 8 #47614)
Date: Tue, 6 Apr 2021 21:38:57 -0400
From: Philip McGrath <philip@philipmcgrath.com>
To: Jack Hill <jackhill@jackhill.us>, Mark H Weaver <mhw@netris.org>
CC: guix-devel@gnu.org

Indeed, I expect this is a more precise diagnosis of the same problem. 
My patch in https://issues.guix.gnu.org/47180 solves it by putting the 
store references (search paths for foreign libraries) in a configuration 
data file that isn't compiled, so they don't end up in .zo files in the 
first place.

The .zo format is intentionally undocumented and subject to breaking 
change, including from different compilation options. At a minimum, a 
change to the Racket version number signals a breaking change to 
compiled code (e.g. Git is now at 8.0.0.13, so 13 breaking changes since 
the release). Internally, I don't know all the details, but the normal 
8.0 .zo format has a Racket layer around the Chez Scheme object format, 
which seems to be very complex: it looks like it supports 
user-configurable compression at the granularity of the individual 
object within an object file. So it seems much better to avoid rewriting 
.zo files altogether.

-Philip

On 4/6/21 9:20 PM, Jack Hill wrote:
> On Tue, 6 Apr 2021, Mark H Weaver wrote:
> 
>> Anyway, I doubt that imposing such a limitation would adequately solve
>> the problem here of chunked references in Racket 8, because I suspect
>> that Racket 8 could split store references at arbitrary points in the
>> string.  I doubt that we can safely assume that the hash component of
>> store references will be stored contiguously in *.zo files.
> 
> Mark and everyone,
> 
> I wanted to spin off a subthread on guix-devel, to make you aware of 
> another problem that we've run into with reference in .zo getting 
> mangled: https://issues.guix.gnu.org/47180
> 
> Best,
> Jack
> 




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#47614: [security] Chunked store references in .zo files in Racket 8
  2021-04-07  1:48   ` bug#47614: [security] Chunked store references in .zo files in Racket 8 #47614 Philip McGrath
@ 2021-04-16 15:46     ` Ludovic Courtès
  2021-04-16 19:46       ` Philip McGrath
  2021-04-17  9:25       ` Mark H Weaver
  0 siblings, 2 replies; 6+ messages in thread
From: Ludovic Courtès @ 2021-04-16 15:46 UTC (permalink / raw)
  To: Philip McGrath; +Cc: 47614

Hi Philip and all,

Philip McGrath <philip@philipmcgrath.com> skribis:

> Indeed, I expect this is a more precise diagnosis of the same
> problem. My patch in https://issues.guix.gnu.org/47180 solves it by
> putting the store references (search paths for foreign libraries) in a
> configuration data file that isn't compiled, so they don't end up in
> .zo files in the first place.

IIUC, now that <https://issues.guix.gnu.org/47180> has been closed, this
bug is fixed.  Am I right?

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#47614: [security] Chunked store references in .zo files in Racket 8
  2021-04-16 15:46     ` bug#47614: [security] Chunked store references in .zo files in Racket 8 Ludovic Courtès
@ 2021-04-16 19:46       ` Philip McGrath
  2021-04-17  9:25       ` Mark H Weaver
  1 sibling, 0 replies; 6+ messages in thread
From: Philip McGrath @ 2021-04-16 19:46 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 47614

Hi Ludo’,

On 4/16/21 11:46 AM, Ludovic Courtès wrote:
> IIUC, now that <https://issues.guix.gnu.org/47180> has been closed, this
> bug is fixed.  Am I right?

Yes, it seems to be fixed with respect to Racket, though Mark mentioned 
here in <https://issues.guix.gnu.org/47064#9> (also now fixed) some 
broader implications for the grafting code.

-Philip




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#47614: [security] Chunked store references in .zo files in Racket 8
  2021-04-16 15:46     ` bug#47614: [security] Chunked store references in .zo files in Racket 8 Ludovic Courtès
  2021-04-16 19:46       ` Philip McGrath
@ 2021-04-17  9:25       ` Mark H Weaver
  1 sibling, 0 replies; 6+ messages in thread
From: Mark H Weaver @ 2021-04-17  9:25 UTC (permalink / raw)
  To: Ludovic Courtès, Philip McGrath; +Cc: 47614-done

Ludovic Courtès <ludo@gnu.org> writes:
> IIUC, now that <https://issues.guix.gnu.org/47180> has been closed,
> this bug is fixed.  Am I right?

Yes, I believe so.  All store items referenced by Racket now seem to be
properly grafted, so I'm closing this bug now.

The more general issue with the grafting code--namely that since commit
57bdd79e48, it no longer has the desirable property of checking every
byte against an expected value before rewriting it, which can lead to
silent corruption of files such as Racket .zo files if any store items
references sneak in--can be addressed in another bug report.

     Thanks,
       Mark




^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-04-17  9:28 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-07  1:20 Racket 8 and store references (was [security] Chunked store references in .zo files in Racket 8 #47614) Jack Hill
2021-04-07  1:38 ` Philip McGrath
2021-04-07  1:48   ` bug#47614: [security] Chunked store references in .zo files in Racket 8 #47614 Philip McGrath
2021-04-16 15:46     ` bug#47614: [security] Chunked store references in .zo files in Racket 8 Ludovic Courtès
2021-04-16 19:46       ` Philip McGrath
2021-04-17  9:25       ` Mark H Weaver

all messages for Guix-related lists mirrored at yhetil.org

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://yhetil.org/guix

Example config snippet for mirrors.


AGPL code for this site: git clone http://ou63pmih66umazou.onion/public-inbox.git