Re: pdumper's performance

[Stefan Monnier <monnier@IRO.UMontreal.CA>]

> Dumping objects individually instead of in blocks both simplifies the
> implementation and allows for a more compact dump, as you point
> out below.

I understand the compactness argument, but I'm surprised it makes the
implementation simpler.  I'll trust you on that, tho.

>> I naively thought your code would take
>> cons_blocks, symbol_blocks, ... and write those blocks as-is so objects
>> keep the same layout, and things like mark_maybe_object don't need to be
>> changed at all.  I understand this would end up writing larger dumps
>> (since they would include some free objects), but I'd have expected it
>> would lead to simpler code and a smaller patch.
>
> If we keep the mark bits out of the objects, we can avoid modifying the
> object pages just for GC. In the non-PIC case, in which in principle we
> don't have to relocate the dump, that means that the pages in the dump
> stay clean and file-backed, not dirty, COWed, and pagefile-backed as they
> would if we banged on them just for the GC. That's an efficiency win.

I kind of see the benefit, but:
- unlike the purespace, the dumped&restored heap is not read-only,
  so even with the markbits elsewhere the pages will generally not all
  stay clean.
- the benefit you mention only applies to the case where
  no-relocation was needed.  And I suspect that in practice keeping those
  pages "clean and file-backed" will very rarely make a noticeable
  difference anyway.
- I like the idea of keeping markbits separately (XEmacs does the same,
  IIUC, for the needs of their incremental GC where they want to mark
  pages as read-only to catch writes from the mutator, while still
  letting the collector twiddle the mark bits).  But I think this should
  be a separate patch and should apply to all objects rather than only
  the pdump'd ones.

> For a future more-efficient GC, contiguous object storage with external
> mark bits is probably the way to go for the entire heap.

Agreed, obviously [ not only because of the text above, but also because
I implemented the separate markbits for floats and cons cells ;-)  ]


        Stefan