From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eric Ludlam Newsgroups: gmane.emacs.bugs Subject: bug#29220: 26.0.90; eieio-persistent-read fail to restore saved object. Date: Sat, 23 Dec 2017 21:18:44 -0500 Message-ID: References: <87y3nga0lv.fsf@killashandra.ballybran.fr> <87lgimpdhp.fsf@ericabrahamsen.net> <87h8t8is1u.fsf@ericabrahamsen.net> <87zi6zmzh7.fsf@killashandra.ballybran.fr> <87h8t69d7b.fsf@ericabrahamsen.net> <87609lcbip.fsf@ericabrahamsen.net> <87bmjcc2rh.fsf@ericabrahamsen.net> <874lp4aije.fsf@ericabrahamsen.net> <83po7pwlgs.fsf@gnu.org> <87lgib4z2o.fsf@ericabrahamsen.net> <87wp1rcz21.fsf@users.sourceforge.net> <87mv2jkaap.fsf@killashandra.ballybran.fr> <87wp1nsk52.fsf@killashandra.ballybran.fr> <87k1xjke54.fsf@ericabrahamsen.net> <87bmit45je.fsf@ericabrahamsen.net> <87tvwk253j.fsf@ericabrahamsen.net> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1514081838 3253 195.159.176.226 (24 Dec 2017 02:17:18 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 24 Dec 2017 02:17:18 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 Cc: 29220@debbugs.gnu.org, "Eric M. Ludlam" , Pierre =?UTF-8?Q?T=C3=A9choueyres?= , Noam Postavsky To: Eric Abrahamsen , Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Dec 24 03:17:13 2017 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eSvqh-0000JJ-Mk for geb-bug-gnu-emacs@m.gmane.org; Sun, 24 Dec 2017 03:17:11 +0100 Original-Received: from localhost ([::1]:46437 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eSvsg-0006en-5U for geb-bug-gnu-emacs@m.gmane.org; Sat, 23 Dec 2017 21:19:14 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:50550) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eSvsZ-0006eR-32 for bug-gnu-emacs@gnu.org; Sat, 23 Dec 2017 21:19:08 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eSvsU-0003Zb-1R for bug-gnu-emacs@gnu.org; Sat, 23 Dec 2017 21:19:07 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:41079) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eSvsT-0003ZO-Tx for bug-gnu-emacs@gnu.org; Sat, 23 Dec 2017 21:19:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1eSvsT-0004E6-L7 for bug-gnu-emacs@gnu.org; Sat, 23 Dec 2017 21:19:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eric Ludlam Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 24 Dec 2017 02:19:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 29220 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 29220-submit@debbugs.gnu.org id=B29220.151408193616232 (code B ref 29220); Sun, 24 Dec 2017 02:19:01 +0000 Original-Received: (at 29220) by debbugs.gnu.org; 24 Dec 2017 02:18:56 +0000 Original-Received: from localhost ([127.0.0.1]:49760 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eSvsN-0004Dk-Pl for submit@debbugs.gnu.org; Sat, 23 Dec 2017 21:18:56 -0500 Original-Received: from mail-qk0-f169.google.com ([209.85.220.169]:42310) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eSvsM-0004DX-Bt for 29220@debbugs.gnu.org; Sat, 23 Dec 2017 21:18:54 -0500 Original-Received: by mail-qk0-f169.google.com with SMTP id d202so14300886qkc.9 for <29220@debbugs.gnu.org>; Sat, 23 Dec 2017 18:18:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=+TGvD4smHBhOd84qKAjl5Mwb3nEqJrFEFUZAUTGHbPE=; b=nWqaaJcgWVMAvP7ydGyWuQGO/G7CgzjkB7d0LrSnUzUHbbwFqVUotX/8+YjRh7Fgw4 3uh1dsK0De27maqS3NBEmtvjuaZP0QTx3+AM6qnuvna+8xOFu7Np8hbxwghEU1f5N725 Kc+QKoooDbY7w3QxLY04KQpLNi0zFtyeO9jljPg4GAoeA8ewXIUcPOxnW/hP+BJbHsgl 0zQpMuYhr+dHteR0l1b+rvZuBB4xPGS6yToNOLFeC3mn7IJjIV7SnfK42RLJT2lqT2Ur 9UbmuO3Bok2ipF8gv9hkLNd7hENfuGuPLnvLTizCoY14HnqqwFlzf8v0C5XaSOqrGccc 7Csw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=+TGvD4smHBhOd84qKAjl5Mwb3nEqJrFEFUZAUTGHbPE=; b=c9SWaFtPbSXv5YZxiXgpf2opJvSt9nEL5n5aIL9EsFpbfBZoehfqEAj1Gtx06AjGFx dmIXJdiWej17tUG9LVgI+TNmpKmDS2FpFvARxh3HhEE8Xng0NoX9RdUNR2Tvbekfr7YQ T9kxTlFQEwdQm9fsrFRrOyONaZhRiwZNHuJ7d+4gJGePgaFvN/9BukgoqZ/HzfNnLfnK fZ4lwm8m4T9CWtKhM0qCQXEKqE/B9Umymaokvlh/zg6rws+/hqwF4HP6AlpY3t4Snld4 PdzT1R0JWnFtzwtZe+9GpHn7mA6ADz8i9ORJrigfqc9Ybs2b4j1ozZw+DYmnjncgCCCN Jkiw== X-Gm-Message-State: AKGB3mIi/v7VtIQsENoCVvdTHZCfLcZsFzGJNdVIN+eEPeTS/IoGfCLD 7MZ57+X01CQFLTstu55FYec= X-Google-Smtp-Source: ACJfBotdTjtxoJWY8KZS4JdBlN31YqvxTrvUpHOaFL6CfCSvkSHsj9ghQH/N08xC5BBhxHjFu0mmUg== X-Received: by 10.55.73.13 with SMTP id w13mr24064643qka.201.1514081928437; Sat, 23 Dec 2017 18:18:48 -0800 (PST) Original-Received: from [192.168.1.202] (pool-74-104-122-152.bstnma.fios.verizon.net. [74.104.122.152]) by smtp.googlemail.com with ESMTPSA id f4sm11323974qtj.21.2017.12.23.18.18.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 23 Dec 2017 18:18:46 -0800 (PST) In-Reply-To: <87tvwk253j.fsf@ericabrahamsen.net> Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:141435 Archived-At: On 12/20/2017 09:21 PM, Eric Abrahamsen wrote: > On 12/20/17 15:54 PM, Stefan Monnier wrote: >>> As for the quoting thing, my current idea is to not add quotes to lists >>> when writing, then strip quotes on restore, if the car of the list is a >>> valid class symbol then try to restore an object, but wrap in >>> `condition-case' and return the plain list if an error is raised. >> I recommend you ask the opinion of Eric M. Ludlam >> (the original author of that code). > Okay! > > Eric, I'm dragging you into this because we're having a little trouble > with the eieio persistence functionality. Sorry... > > In Emacs 26 objects are implemented using records, which (long story > short) means that their representations from `prin1' are no longer > reconstructable using `read', which is causing problems for > eieio-persistent. Hey, thanks for checking in. I know a bunch of eieio was re-written to be faster and better, but I haven't been keeping up with the latest version.  For mysterious reasons, Ubuntu is still on Emacs 24 so I'm still on the old stuff. > The current persistence read/write process basically only handles > top-level slot values. It looks at the value, does one of the following: > > 1. Writes an object using `object-write' > 2. Writes a list of objects using a "list" plus `object-write' > 3. Quotes a list and prints it > 4. Prints value directly > > If objects are inside lists, hash tables, or vectors (or anything else), > they are not written using `object-write'. This used to not be a problem > because their `prin1' representation was still readable. > > This is no longer the case, which leads to failure for libraries that > put objects inside other data structures. The original purpose of object-write on eieio objects was to prevent circular references which kept popping up in EDE and SemanticDB. The first version just dumped the old raw vector.  That's one of the reasons why it doesn't try to be pedantic about carefully going down through every list. It only had to deal with EDE and Semantic classes. > My feeling is that both the write and restore process should walk the > entire tree of whatever object is being written. The write process > should turn objects into list representations, nothing more. The restore > process should restore objects, and strip strings of properties (I've > got the outline of this in bug#29541), and nothing more. If the restore > process was destructive, it would save some consing. I think that makes sense.  I was definitely taking short-cuts in letting lists write themselves b/c I was lazy.  Based on what you describe, it probably makes sense to walk entire structures and write each step. It might be even better if that was a core Emacs feature, and not something specific to a simple eieio utility base class.  :) > Three items for consideration: > > 1. What do you think about the above proposal? Have you had some ideas in > this direction already? > 2. Stefan pointed out that, if we don't quote lists, there's a potential > ambiguity during restore time between lists that are lists, and lists > that represent an object. Do you have any thoughts about this? Objects, as a general rule, need to execute code of some sort to instantiate, link themselves into a system (like semanticdb), or generate transient data for slots that are derived (and not saved). Thus, each layer in the save file needs some way to mark themselves as to what they are so the restore process can either instantiate or just use the raw data. From the quote / not quote point of view, you could use any kind of keys you want to indicate what is at each layer. For example, you might say that any list that starts with a class symbol is an object and everything else is data.  Of course, then you can't have a list of class symbols, and definitely need to avoid having classes with simple names. I used lists that could be read with 'read', but you could use xml or all sorts of other save formats to ensure you know exactly what's going on.  I have a vague recollection of writing an xml export for eieio, but I don't think I used it for anything. > 3. Emacs 26 is looming, and our current solutions either break pcache, > or break the Gnus registry. We might want to do a stop-gap for Emacs > 26, and a more complete fix for master/27. Makes sense to me. Don't forget that objects can execute code in their 'object-write' to convert an inconvenient data structure into something simple, and in the constructor convert it back again. For example, have a slot with the hash table be non-savable (no :initarg). In the object-write, convert the hash table into a simple flat list on a different slot with an :initarg. On load, copy the content of the :initarg slot back into your hash table. In this way, you may be able to have the 2 examples avoid having any problems with the current system in E26. Not that I don't know what the real issues are.  I'm just tossing some options out there. Good Luck Eric