Saving a elisp data structure into string and evaluating it back into objects

Saving a elisp data structure into string and evaluating it back into objects

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 2488 bytes --]

Hello list,

I'm trying to combine DeskTop, layout-restore.el and revive.el to create the
ultimate persistence experience :)

Well, what I'm trying to do is, every time I exit emacs, the whole memory
state (elisp object's tree) will be dumped to a file and reloaded when I
restart emacs again.

Why am I trying to do this? Well, there are many settings in emacs that are
volatile, many vars that are not saved. I know that emacs has the capability
to save its vars into .emacs, but this depends on the mode and the var, no?

Anyway, the point is, I love how layout-restore.el works, and this is the
main reason I'm trying to do this. I want to persist all the state created
by layout-restore, meaning all the window layouts saved per buffer while I
was working in an emacs session. When I exit and start emacs again, I want
the very same layouts associated with the very same buffers.

I'm not an emacs/elisp guru, but I could find out that layout-restore saves
its state into the layout-configuration-alist global var. This var is a list
of buffer objects, here's a stringfied version of its contents (I saved a
layout using layout-save-current and it filled the var with this
data-structure):

*"((#<buffer *scratch*> #<window-configuration> (#<buffer *scratch*> .
\"*scratch*\") (#<buffer *Help*> . \"*Help*\")))"*

So far so good. I can get the value of the var into a string, and I also
have found out how to tell revive.el to also save the contents of this var:
To add this var's symbol to the list in revive:save-variables-global, like
this:

        (setq revive:save-variables-global
              '(layout-configuration-alist))

When I do M-x save-current-configuration, it does get the value of the var.
The following is an excerpt from ~/.revive.el:

((layout-configuration-alist (#<buffer dot.emacs.el> #<window-configuration>
(#<buffer dot.emacs.el> . "dot.emacs.el") (#<buffer *Help*> . "*Help*"))))

But, when I try to restore all this state by calling resume (M-x resume), it
fails with the following message:

Invalid syntax "#".

Well, it is pretty obvious that the string is not representing the objects
in a manner that can be revaluated. These are simple to_s (as in Ruby)
representations and not full data structures that could be used to
reconstruct the object in memory again.

My question is: How could I dump the contents of layout-configuration-alist
it in a way that resume could eval and eval it back into first class elisp
objects again?

Thanks,

Marcelo.

[-- Attachment #2: Type: text/html, Size: 2763 bytes --]

Re: Saving a elisp data structure into string and evaluating it back into objects

[Pascal J. Bourguignon]

Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
> Well, what I'm trying to do is, every time I exit emacs, the whole
> memory state (elisp object's tree) will be dumped to a file and
> reloaded when I restart emacs again.

This is not feasible using print and read, since not all lisp objects
are printable readably.

What you could do is to save the emacs lisp image, and use it to boot
emacs the next time.  This is a usual operation in other lisps, but
with emacs it's more rarely used (only at built time usually).

See the dump-emacs function.  Unfortunately, it is usable only in
batch mode.  This restriction should be removed...


> Invalid syntax "#".

Yes, that's the symptom of trying to read unreadable objects.


> My question is: How could I dump the contents of
> layout-configuration-alist it in a way that resume could eval and
> eval it back into first class elisp objects again?

An alternative would be not to try to save EVERYTHING, but only what
really matters to you, taking care of saving only readably printable
objects, or for which you write a serialization/deserialization
function pair.


Notice that there are already emacs features to save and restore some
of the state, such as the open files, etc.  See for example the
functions desktop-save and desktop-read.


-- 
__Pascal Bourguignon__

Re: Saving a elisp data structure into string and evaluating it back into objects

[Kevin Rodgers]

Marcelo de Moraes Serpa wrote:
> Hello list,
> 
> I'm trying to combine DeskTop, layout-restore.el and revive.el to create 
> the ultimate persistence experience :)
> 
> Well, what I'm trying to do is, every time I exit emacs, the whole 
> memory state (elisp object's tree) will be dumped to a file and reloaded 
> when I restart emacs again.
> 
> Why am I trying to do this? Well, there are many settings in emacs that 
> are volatile, many vars that are not saved. I know that emacs has the 
> capability to save its vars into .emacs, but this depends on the mode 
> and the var, no?

C-h v desktop-globals-to-save
C-h v desktop-locals-to-save

-- 
Kevin Rodgers
Denver, Colorado, USA

Re: Saving a elisp data structure into string and evaluating it back into objects

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 1879 bytes --]

Thank you all for taking your time to reply!

@Kevin: I've added "layout-configuration-alist" to this var, and indeed, it
saved, but here's what it wrote:

(setq layout-configuration-alist '(("Unprintable entity" "Unprintable
> entity" ("Unprintable entity" . "revive.el")) ("Unprintable entity"
> "Unprintable entity" ("Unprintable entity" . "dot.emacs.el") ("Unprintable
> entity" . "dot.emacs.el"))))
>

"Unprintable entity". Any way I could fix it from the top of your head?

@Pascal: Thanks for the comprehensive answer!

An alternative would be not to try to save EVERYTHING, but only what
> really matters to you, taking care of saving only readably printable
> objects, or for which you write a serialization/deserialization
> function pair.
>

Well, you are absolutely right -- actually, the only thing that interests me
is saving and restoring the value of "layout-configuration-alist", so I can
have my buffer-indexed window layouts persisted.

How could I go and write a serialization/deserialization function pair? And
where would I integrate it?

Thanks,

Marcelo.

On Wed, Oct 7, 2009 at 7:57 AM, Kevin Rodgers <kevin.d.rodgers@gmail.com>wrote:

> Marcelo de Moraes Serpa wrote:
>
>> Hello list,
>>
>> I'm trying to combine DeskTop, layout-restore.el and revive.el to create
>> the ultimate persistence experience :)
>>
>> Well, what I'm trying to do is, every time I exit emacs, the whole memory
>> state (elisp object's tree) will be dumped to a file and reloaded when I
>> restart emacs again.
>>
>> Why am I trying to do this? Well, there are many settings in emacs that
>> are volatile, many vars that are not saved. I know that emacs has the
>> capability to save its vars into .emacs, but this depends on the mode and
>> the var, no?
>>
>
> C-h v desktop-globals-to-save
> C-h v desktop-locals-to-save
>
> --
> Kevin Rodgers
> Denver, Colorado, USA
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2886 bytes --]

Re: Saving a elisp data structure into string and evaluating it back into objects

[Lennart Borgman]

On Wed, Oct 7, 2009 at 5:09 PM, Marcelo de Moraes Serpa
<celoserpa@gmail.com> wrote:
>
> Well, you are absolutely right -- actually, the only thing that interests me
> is saving and restoring the value of "layout-configuration-alist", so I can
> have my buffer-indexed window layouts persisted.
>
> How could I go and write a serialization/deserialization function pair? And
> where would I integrate it?


Is something like winsav.el in nXhtml what you want? If not, could you
please explain what you are missing there?

Re: Saving a elisp data structure into string and evaluating it back into objects

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 783 bytes --]

I want to save the memory state for layout-restore.el, and this happens to
be in the layout-configuration-alist variable. That's all, actually.


On Wed, Oct 7, 2009 at 10:33 AM, Lennart Borgman
<lennart.borgman@gmail.com>wrote:

> On Wed, Oct 7, 2009 at 5:09 PM, Marcelo de Moraes Serpa
> <celoserpa@gmail.com> wrote:
> >
> > Well, you are absolutely right -- actually, the only thing that interests
> me
> > is saving and restoring the value of "layout-configuration-alist", so I
> can
> > have my buffer-indexed window layouts persisted.
> >
> > How could I go and write a serialization/deserialization function pair?
> And
> > where would I integrate it?
>
>
> Is something like winsav.el in nXhtml what you want? If not, could you
> please explain what you are missing there?
>

[-- Attachment #2: Type: text/html, Size: 1189 bytes --]

Re: Saving a elisp data structure into string and evaluating it back into objects

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Oct 07, 2009 at 10:05:26AM +0200, Pascal J. Bourguignon wrote:
> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:

[...]

> This is not feasible using print and read, since not all lisp objects
> are printable readably.
      ^^^^^^^^^^^^^^^^^^

Wow, Pascal. Nice phrase. May I use it from time to time?

;-)

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFKzMVSBcgs9XrR2kYRAmQfAJsHIlJR2YLPJEkhEnRLcvGl5pFdMQCeKSbP
jnX0vmppvE/NCU+rY1tCw+s=
=evf9
-----END PGP SIGNATURE-----

RE: Saving a elisp data structure into string and evaluating itback into objects

[Drew Adams]

1. There have been several examples of libraries that save and then restore
various sets of Lisp objects, typically in order to restore the state of an
Emacs session, or at least part of it. Some have already been mentioned here.

FWIW, I use Savehist, to restore both history variables and other variables that
I choose (via option `savehist-additional-variables'):
http://www.emacswiki.org/emacs/SaveHist

But it too doesn't help with Emacs objects whose print form is not
Lisp-readable.

(I also have my own code that serializes completion alists, in such a way that
it records and restores markers. But the serialization format I use is
particular to my code.)


2. AFAIK, there is no general serialize/deserialize feature, as such, in Emacs
Lisp. That is, there are no predefined functions that do just that: (1) You
provide a set of objects and a file name to a `serialize' function, and it
writes the objects to the file in a way that captures their current state. (2)
You call a `deserialize' function to read the file (e.g. using the Lisp reader)
and thus re-create the objects in the state in which they were saved.


3. It might be worthwhile looking at this thread:
http://lists.gnu.org/archive/html/emacs-devel/2009-09/msg00846.html

The thread is too short, IMO (it drew no real interest, for some reason), and it
got side-tracked by a discussion of licensing. But the idea is simple:

Thierry Volpiatto noticed that, at least for recent Emacs versions,
byte-compiling serializes Emacs objects in a fairly general way, so reading
(loading) a byte-compiled (*.elc) file then restores the objects as they were.

You can use this feature as is. Or you (or Emacs development) could use it to
define `serialize' and `deserialize' functions for general use.

Thierry pointed out the following URL to me in connection with this. It mentions
using `#.' in Common Lisp, whereas for Emacs Lisp it is `eval-when-compile' that
does the trick.
http://www.cs.cmu.edu/Groups/AI/html/faqs/lang/lisp/part5/faq-doc-5.html

HTH.

Re: Saving a elisp data structure into string and evaluating itback into objects

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 2610 bytes --]

@Drew: Thanks a lot for the comprehensive answer, I really appreciate that!

So, I have two choices, as far as I can see:

1) Write my own serialize/deserialize function. Sounds feasible, but could
someone give me a sample of that? I'm just beginning in elisp and all still
sounds a little bit out of place for me;
2) Use the byte-complie thing. Also, I know I could use that, but how ?

Thank you,

Marcelo.

On Wed, Oct 7, 2009 at 12:41 PM, Drew Adams <drew.adams@oracle.com> wrote:

> 1. There have been several examples of libraries that save and then restore
> various sets of Lisp objects, typically in order to restore the state of an
> Emacs session, or at least part of it. Some have already been mentioned
> here.
>
> FWIW, I use Savehist, to restore both history variables and other variables
> that
> I choose (via option `savehist-additional-variables'):
> http://www.emacswiki.org/emacs/SaveHist
>
> But it too doesn't help with Emacs objects whose print form is not
> Lisp-readable.
>
> (I also have my own code that serializes completion alists, in such a way
> that
> it records and restores markers. But the serialization format I use is
> particular to my code.)
>
>
> 2. AFAIK, there is no general serialize/deserialize feature, as such, in
> Emacs
> Lisp. That is, there are no predefined functions that do just that: (1) You
> provide a set of objects and a file name to a `serialize' function, and it
> writes the objects to the file in a way that captures their current state.
> (2)
> You call a `deserialize' function to read the file (e.g. using the Lisp
> reader)
> and thus re-create the objects in the state in which they were saved.
>
>
> 3. It might be worthwhile looking at this thread:
> http://lists.gnu.org/archive/html/emacs-devel/2009-09/msg00846.html
>
> The thread is too short, IMO (it drew no real interest, for some reason),
> and it
> got side-tracked by a discussion of licensing. But the idea is simple:
>
> Thierry Volpiatto noticed that, at least for recent Emacs versions,
> byte-compiling serializes Emacs objects in a fairly general way, so reading
> (loading) a byte-compiled (*.elc) file then restores the objects as they
> were.
>
> You can use this feature as is. Or you (or Emacs development) could use it
> to
> define `serialize' and `deserialize' functions for general use.
>
> Thierry pointed out the following URL to me in connection with this. It
> mentions
> using `#.' in Common Lisp, whereas for Emacs Lisp it is `eval-when-compile'
> that
> does the trick.
> http://www.cs.cmu.edu/Groups/AI/html/faqs/lang/lisp/part5/faq-doc-5.html
>
> HTH.
>
>

[-- Attachment #2: Type: text/html, Size: 3328 bytes --]

Re: Saving a elisp data structure into string and evaluating it back into objects

[Pascal J. Bourguignon]

tomas@tuxteam.de writes:

> On Wed, Oct 07, 2009 at 10:05:26AM +0200, Pascal J. Bourguignon wrote:
>> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>
> [...]
>
>> This is not feasible using print and read, since not all lisp objects
>> are printable readably.
>       ^^^^^^^^^^^^^^^^^^
>
> Wow, Pascal. Nice phrase. May I use it from time to time?

Sure.  It is standard lisp jargon.

http://www.lispworks.com/documentation/HyperSpec/Body/v_pr_rda.htm
http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_r.htm#readably


-- 
__Pascal Bourguignon__

Re: Saving a elisp data structure into string and evaluating it back into objects

[Lennart Borgman]

Oh, I thought that was window and frame configuration you wanted to
save and restore. My bad if that is not what you want.


On Wed, Oct 7, 2009 at 6:23 PM, Marcelo de Moraes Serpa
<celoserpa@gmail.com> wrote:
> I want to save the memory state for layout-restore.el, and this happens to
> be in the layout-configuration-alist variable. That's all, actually.
>
>
> On Wed, Oct 7, 2009 at 10:33 AM, Lennart Borgman <lennart.borgman@gmail.com>
> wrote:
>>
>> On Wed, Oct 7, 2009 at 5:09 PM, Marcelo de Moraes Serpa
>> <celoserpa@gmail.com> wrote:
>> >
>> > Well, you are absolutely right -- actually, the only thing that
>> > interests me
>> > is saving and restoring the value of "layout-configuration-alist", so I
>> > can
>> > have my buffer-indexed window layouts persisted.
>> >
>> > How could I go and write a serialization/deserialization function pair?
>> > And
>> > where would I integrate it?
>>
>>
>> Is something like winsav.el in nXhtml what you want? If not, could you
>> please explain what you are missing there?
>
>

Re: Saving a elisp data structure into string and evaluating it back into objects

[Kevin Rodgers]

Pascal J. Bourguignon wrote:
> tomas@tuxteam.de writes:
> 
>> On Wed, Oct 07, 2009 at 10:05:26AM +0200, Pascal J. Bourguignon wrote:
>>> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>> [...]
>>
>>> This is not feasible using print and read, since not all lisp objects
>>> are printable readably.
>>       ^^^^^^^^^^^^^^^^^^
>>
>> Wow, Pascal. Nice phrase. May I use it from time to time?

As a native speaker, I think that should be "readably printable".

> Sure.  It is standard lisp jargon.
> 
> http://www.lispworks.com/documentation/HyperSpec/Body/v_pr_rda.htm
> http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_r.htm#readably

Thanks for the links!

-- 
Kevin Rodgers
Denver, Colorado, USA

Re: Saving a elisp data structure into string and evaluating it back into objects

[David Kastrup]

Kevin Rodgers <kevin.d.rodgers@gmail.com> writes:

> Pascal J. Bourguignon wrote:
>> tomas@tuxteam.de writes:
>>
>>> On Wed, Oct 07, 2009 at 10:05:26AM +0200, Pascal J. Bourguignon wrote:
>>>> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>>> [...]
>>>
>>>> This is not feasible using print and read, since not all lisp objects
>>>> are printable readably.
>>>       ^^^^^^^^^^^^^^^^^^
>>>
>>> Wow, Pascal. Nice phrase. May I use it from time to time?
>
> As a native speaker, I think that should be "readably printable".

I don't think so, because readability is the result, not the manner of
printing.  "readily printable", but "printable readably".  It is
"printable in a readable manner", not "in a readable manner printable".

-- 
David Kastrup

Re: Saving a elisp data structure into string and evaluating it back into objects

[Ted Zlatanov]

On Fri, 09 Oct 2009 07:50:09 -0600 Kevin Rodgers <kevin.d.rodgers@gmail.com> wrote: 

KR> As a native speaker, I think that should be "readably printable".

"printably readably printable"? ;)

Ted

Re: Saving a elisp data structure into string and evaluating it back into objects

[Kevin Rodgers]

David Kastrup wrote:
> Kevin Rodgers <kevin.d.rodgers@gmail.com> writes:
> 
>> Pascal J. Bourguignon wrote:
>>> tomas@tuxteam.de writes:
>>>
>>>> On Wed, Oct 07, 2009 at 10:05:26AM +0200, Pascal J. Bourguignon wrote:
>>>>> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>>>> [...]
>>>>
>>>>> This is not feasible using print and read, since not all lisp objects
>>>>> are printable readably.
>>>>       ^^^^^^^^^^^^^^^^^^
>>>>
>>>> Wow, Pascal. Nice phrase. May I use it from time to time?
>> As a native speaker, I think that should be "readably printable".
> 
> I don't think so, because readability is the result, not the manner of
> printing.  "readily printable", but "printable readably".  It is
> "printable in a readable manner", not "in a readable manner printable".

Ah, yes!

-- 
Kevin Rodgers
Denver, Colorado, USA