> I was thinking of doing it via unload-feature

Isn't this out of scope for unload-feature? After all,

package-initialize does not load features.

> if it doesn't work quite-right, it's a bug in the package.

I think if we take an approach like this, we'll end up uncovering

quite a few bugs, indeed. And I worry that this will result in people

thinking that package.el is buggy, since the bugs will only show up

when people use package.el, as opposed to any other package manager or

as opposed to loading the packages manually.