bug#29220: 26.0.90; eieio-persistent-read fail to restore saved object.

[Eric Ludlam <ericludlam@gmail.com>]

On 12/20/2017 09:21 PM, Eric Abrahamsen wrote:
> On 12/20/17 15:54 PM, Stefan Monnier wrote:
>>> As for the quoting thing, my current idea is to not add quotes to lists
>>> when writing, then strip quotes on restore, if the car of the list is a
>>> valid class symbol then try to restore an object, but wrap in
>>> `condition-case' and return the plain list if an error is raised.
>> I recommend you ask the opinion of Eric M. Ludlam <zappo@gnu.org>
>> (the original author of that code).
> Okay!
>
> Eric, I'm dragging you into this because we're having a little trouble
> with the eieio persistence functionality. Sorry...
>
> In Emacs 26 objects are implemented using records, which (long story
> short) means that their representations from `prin1' are no longer
> reconstructable using `read', which is causing problems for
> eieio-persistent.
Hey, thanks for checking in.

I know a bunch of eieio was re-written to be faster and better, but I 
haven't been keeping up with the latest version.  For mysterious 
reasons, Ubuntu is still on Emacs 24 so I'm still on the old stuff.
> The current persistence read/write process basically only handles
> top-level slot values. It looks at the value, does one of the following:
>
> 1. Writes an object using `object-write'
> 2. Writes a list of objects using a "list" plus `object-write'
> 3. Quotes a list and prints it
> 4. Prints value directly
>
> If objects are inside lists, hash tables, or vectors (or anything else),
> they are not written using `object-write'. This used to not be a problem
> because their `prin1' representation was still readable.
>
> This is no longer the case, which leads to failure for libraries that
> put objects inside other data structures.

The original purpose of object-write on eieio objects was to prevent
circular references which kept popping up in EDE and SemanticDB. The first
version just dumped the old raw vector.  That's one of the reasons why it
doesn't try to be pedantic about carefully going down through every list.
It only had to deal with EDE and Semantic classes.

> My feeling is that both the write and restore process should walk the
> entire tree of whatever object is being written. The write process
> should turn objects into list representations, nothing more. The restore
> process should restore objects, and strip strings of properties (I've
> got the outline of this in bug#29541), and nothing more. If the restore
> process was destructive, it would save some consing.
I think that makes sense.  I was definitely taking short-cuts in letting
lists write themselves b/c I was lazy.  Based on what you describe, it
probably makes sense to walk entire structures and write each step.

It might be even better if that was a core Emacs feature, and not something
specific to a simple eieio utility base class.  :)

> Three items for consideration:
>
> 1. What do you think about the above proposal? Have you had some ideas in
>     this direction already?
> 2. Stefan pointed out that, if we don't quote lists, there's a potential
>     ambiguity during restore time between lists that are lists, and lists
>     that represent an object. Do you have any thoughts about this?
Objects, as a general rule, need to execute code of some sort to 
instantiate,
link themselves into a system (like semanticdb), or generate transient data
for slots that are derived (and not saved).

Thus, each layer in the save file needs some way to mark themselves as 
to what
they are so the restore process can either instantiate or just use the 
raw data.

 From the quote / not quote point of view, you could use any kind of 
keys you
want to indicate what is at each layer.

For example, you might say that any list that starts with a class symbol 
is an
object and everything else is data.  Of course, then you can't have a 
list of
class symbols, and definitely need to avoid having classes with simple 
names.

I used lists that could be read with 'read', but you could use xml or 
all sorts of
other save formats to ensure you know exactly what's going on.  I have a 
vague
recollection of writing an xml export for eieio, but I don't think I 
used it for
anything.
> 3. Emacs 26 is looming, and our current solutions either break pcache,
>     or break the Gnus registry. We might want to do a stop-gap for Emacs
>     26, and a more complete fix for master/27.

Makes sense to me.

Don't forget that objects can execute code in their 'object-write' to 
convert
an inconvenient data structure into something simple, and in the constructor
convert it back again.

For example, have a slot with the hash table be non-savable (no :initarg).
In the object-write, convert the hash table into a simple flat list on
a different slot with an :initarg.

On load, copy the content of the :initarg slot back into your hash table.

In this way, you may be able to have the 2 examples avoid having any 
problems
with the current system in E26.

Not that I don't know what the real issues are.  I'm just tossing some 
options out
there.

Good Luck
Eric