bug#29220: 26.0.90; eieio-persistent-read fail to restore saved object.

[Eric Abrahamsen <eric@ericabrahamsen.net>]

On 12/23/17 21:18 PM, Eric Ludlam wrote:
> On 12/20/2017 09:21 PM, Eric Abrahamsen wrote:
>> On 12/20/17 15:54 PM, Stefan Monnier wrote:
>>>> As for the quoting thing, my current idea is to not add quotes to lists
>>>> when writing, then strip quotes on restore, if the car of the list is a
>>>> valid class symbol then try to restore an object, but wrap in
>>>> `condition-case' and return the plain list if an error is raised.
>>> I recommend you ask the opinion of Eric M. Ludlam <zappo@gnu.org>
>>> (the original author of that code).
>> Okay!
>>
>> Eric, I'm dragging you into this because we're having a little trouble
>> with the eieio persistence functionality. Sorry...
>>
>> In Emacs 26 objects are implemented using records, which (long story
>> short) means that their representations from `prin1' are no longer
>> reconstructable using `read', which is causing problems for
>> eieio-persistent.
> Hey, thanks for checking in.
>
> I know a bunch of eieio was re-written to be faster and better, but I
> haven't been keeping up with the latest version.  For mysterious
> reasons, Ubuntu is still on Emacs 24 so I'm still on the old stuff.
>> The current persistence read/write process basically only handles
>> top-level slot values. It looks at the value, does one of the following:
>>
>> 1. Writes an object using `object-write'
>> 2. Writes a list of objects using a "list" plus `object-write'
>> 3. Quotes a list and prints it
>> 4. Prints value directly
>>
>> If objects are inside lists, hash tables, or vectors (or anything else),
>> they are not written using `object-write'. This used to not be a problem
>> because their `prin1' representation was still readable.
>>
>> This is no longer the case, which leads to failure for libraries that
>> put objects inside other data structures.
>
> The original purpose of object-write on eieio objects was to prevent
> circular references which kept popping up in EDE and SemanticDB. The first
> version just dumped the old raw vector.  That's one of the reasons why it
> doesn't try to be pedantic about carefully going down through every list.
> It only had to deal with EDE and Semantic classes.

Thanks for the response! It's good to have a bit of background.

Would you mind pointing me in the direction of some tests or other code
that I can use to make sure the temporary fix works for CEDET and
family? It seems likely that what will happen for Emacs 26 is that I'll
do a limited set of fixes to ensure that no major packages break, and
then try to come up with a more comprehensive solution for Emacs 27.
Right now I've got tests aimed at pcache and the Gnus registry, but I'd
like to make sure EDE and Semantic work correctly, as well.

>> My feeling is that both the write and restore process should walk the
>> entire tree of whatever object is being written. The write process
>> should turn objects into list representations, nothing more. The restore
>> process should restore objects, and strip strings of properties (I've
>> got the outline of this in bug#29541), and nothing more. If the restore
>> process was destructive, it would save some consing.
> I think that makes sense.  I was definitely taking short-cuts in letting
> lists write themselves b/c I was lazy.  Based on what you describe, it
> probably makes sense to walk entire structures and write each step.
>
> It might be even better if that was a core Emacs feature, and not something
> specific to a simple eieio utility base class.  :)

Coincidentally, John Wiegley posted a possible generic structure-walking
function (called "traverse") to this list not too long ago.

[...]

> For example, you might say that any list that starts with a class
> symbol is an
> object and everything else is data.  Of course, then you can't have a
> list of
> class symbols, and definitely need to avoid having classes with simple
> names.

[...]

> Don't forget that objects can execute code in their 'object-write' to
> convert
> an inconvenient data structure into something simple, and in the constructor
> convert it back again.
>
> For example, have a slot with the hash table be non-savable (no :initarg).
> In the object-write, convert the hash table into a simple flat list on
> a different slot with an :initarg.
>
> On load, copy the content of the :initarg slot back into your hash table.
>
> In this way, you may be able to have the 2 examples avoid having any
> problems
> with the current system in E26.
>
> Not that I don't know what the real issues are.  I'm just tossing some
> options out
> there.

Okay, thanks very much for these responses. I've got some ideas about
the full fix, and will work on those in a separate bug report.

Thanks again,
Eric