savehist-mode: invalid-read-syntax "#"

savehist-mode: invalid-read-syntax "#"

[Reiner Steib]

Hi,

`savehist-mode' is a very nice new feature in Emacs 22.  I like it
very much.  Thanks, Hrvoje!

Alas, from time to time `savehist-mode' saves entries containing "#"
like...

  (setq command-history '((describe-key '[(mouse-6 (#<window 14 on
  site-start.el> 41046 (293 . 354) 5470111 nil 41046 (48 . 27)
  nil (221 . 3) (6 . 13)))] 1 nil)))

... to `~/.emacs.d/history'.  The mouse-6 entry came from using `C-h
k'.  I've also seen offending entries after using "#" in with `M-!'
(shell-command-history).  When starting Emacs next, I got...

Debugger entered--Lisp error: (invalid-read-syntax "#")
  signal(invalid-read-syntax ("#"))
  byte-code("STRIPPED" [savehist-mode errvar nil savehist-uninstall signal] 3)
  savehist-mode(nil)
  call-interactively(savehist-mode)
  execute-extended-command(nil)
  call-interactively(execute-extended-command)

Probably `savehist-mode' shouldn't try to save describe-key entries in
the first place.  And it should try to make sure that the history file
is syntactically correct.

Bye, Reiner.


Version  information follows...

In GNU Emacs 22.0.50.2 (i686-pc-linux-gnu, GTK+ Version 2.4.9)
 of 2005-12-29 on shrubbery
X server distributor `The X.Org Foundation', version 11.0.60802000
configured using `configure '--prefix=/import/xtra/emacs/HEAD' '--with-gtk' '--exec-prefix=/import/xtra/emacs/HEAD-i686''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: POSIX
  value of $LC_CTYPE: de_DE@euro
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US
  locale-coding-system: iso-8859-15
  default-enable-multibyte-characters: t

Major mode: Message

Minor modes in effect:
  flyspell-mode: t
  mml-mode: t
  iswitchb-mode: t
  msb-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  show-paren-mode: t
  which-function-mode: t
  recentf-mode: t
  tooltip-mode: t
  auto-compression-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  unify-8859-on-decoding-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: identity
  abbrev-mode: t

Re: savehist-mode: invalid-read-syntax "#"

[Juri Linkov]

> Alas, from time to time `savehist-mode' saves entries containing "#"

Perhaps Emacs should provide a general purpose function (like
`desktop-value-to-string') to print Lisp-readable expressions.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: savehist-mode: invalid-read-syntax "#"

[Hrvoje Niksic]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> Alas, from time to time `savehist-mode' saves entries containing "#"
> like...
>
>   (setq command-history '((describe-key '[(mouse-6 (#<window 14 on
>   site-start.el> 41046 (293 . 354) 5470111 nil 41046 (48 . 27)
>   nil (221 . 3) (6 . 13)))] 1 nil)))

If non-string values can end up in this variable, then savehist
shouldn't be trying to save it.  For me, command-history is not even
part of savehist-minibuffer-history-variables.  Did you explicitly ask
savehist to save this variable?

Re: savehist-mode: invalid-read-syntax "#"

[David Kastrup]

Juri Linkov <juri@jurta.org> writes:

>> Alas, from time to time `savehist-mode' saves entries containing "#"
>
> Perhaps Emacs should provide a general purpose function (like
> `desktop-value-to-string') to print Lisp-readable expressions.

You can't make buffers, windows, overlays, yes even markers
"Lisp-readable".  They are just not restorable by the Lisp reader and
so one should find out what causes them to be in the history in the
first place.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: savehist-mode: invalid-read-syntax "#"

[Reiner Steib]

On Wed, Jan 04 2006, Hrvoje Niksic wrote:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
>> Alas, from time to time `savehist-mode' saves entries containing "#"
>> like...
>>
>>   (setq command-history '((describe-key '[(mouse-6 (#<window 14 on
>>   site-start.el> 41046 (293 . 354) 5470111 nil 41046 (48 . 27)
>>   nil (221 . 3) (6 . 13)))] 1 nil)))
>
> If non-string values can end up in this variable, then savehist
> shouldn't be trying to save it.  For me, command-history is not even
> part of savehist-minibuffer-history-variables.  

It's not present on all machines I use (newlines added in this mail):

$ grep setq.savehist-minibuffer-history-variables ~/.emacs.d/history
(setq savehist-minibuffer-history-variables
 '(gnus-group-web-search-history gnus-group-web-type-history
 bibtex-key-history reftex-cite-regexp-hist TeX-macro-history
 shell-command-history locate-history-list w3m-input-url-history
 gnus-mail-method-history LaTeX-environment-history regexp-history
 string-rectangle-history TeX-command-history Info-search-history
 compile-history set-variable-value-history query-replace-history
 buffer-name-history grep-history read-expression-history
 gnus-method-history coding-system-history iswitchb-history
 woman-topic-history gnus-group-history file-name-history
 minibuffer-history extended-command-history))

$ grep setq.savehist-minibuffer-history-variables ~/.emacs.d/history
(setq savehist-minibuffer-history-variables '(TeX-macro-history
 input-method-history set-variable-value-history
 string-rectangle-history locate-history-list woman-file-history
 command-history gnus-method-history latex-help-hist
 shell-command-history coding-system-history compile-history
 gnus-mail-method-history LaTeX-environment-history TeX-command-history
 Info-search-history woman-topic-history query-replace-history
 regexp-history grep-history buffer-name-history
 extended-command-history read-expression-history file-name-history
 iswitchb-history minibuffer-history gnus-group-history))

> Did you explicitly ask savehist to save this variable?

No, I don't have any settings for savehist-mode in my init files
beside (savehist-mode 1) to enable the mode.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: savehist-mode: invalid-read-syntax "#"

[Reiner Steib]

On Wed, Jan 04 2006, David Kastrup wrote:

> Juri Linkov <juri@jurta.org> writes:
>
>>> Alas, from time to time `savehist-mode' saves entries containing "#"
>>
>> Perhaps Emacs should provide a general purpose function (like
>> `desktop-value-to-string') to print Lisp-readable expressions.
>
> You can't make buffers, windows, overlays, yes even markers
> "Lisp-readable".  

Mentioning "markers" reminds me of another failure when reading the
history file.  I had the following entry in ~/.emacs.d/history (I
didn't C&P in which variable it was) after editing entries in my BBDB:

(bbdb-insert-new-field '["J..." "H..." ("J H...") nil
 (["Home" "NNNNN/NNNNN"] ["Office" "NNNNN/NNNNNN"] ["Mobile" "0"])
 (["Home" ("...weg N") "...." "" "NNNNN" ""])
 ("J....H...@invalid.invalid" "j..._h...@invalid.invalid")
 ((gnokii . "t") (birthday . "NNNN-NN-NN")
  (www . "http://www.invalid.invalid/") (category . "...")
  (creation-date . "NNNN-NN-NN") (timestamp . "NNNN-NN-NN")) 
 ["J... H..." nil #<marker at 22775 in rs.bbdb> nil]]
 'phone '["Mobile" "0"])

> They are just not restorable by the Lisp reader and so one should
> find out what causes them to be in the history in the first place.

AFAICS, it's the following code...

(defun savehist-install ()
  "Hook savehist into Emacs.
Normally invoked by calling `savehist-mode' to set the minor mode.
Installs `savehist-autosave' in `kill-emacs-hook' and on a timer.   To
undo this, call `savehist-uninstall'."
  (add-hook 'minibuffer-setup-hook 'savehist-minibuffer-hook)
[...]

(defun savehist-minibuffer-hook ()
  ;; XEmacs sets minibuffer-history-variable to t to mean "no history
  ;; is being recorded".
  (unless (eq minibuffer-history-variable t)
    (add-to-list 'savehist-minibuffer-history-variables
		 minibuffer-history-variable)))

Maybe there should be a list (say `savehist-ignores-variables') which
should contain `command-history' (and maybe other variables).

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: savehist-mode: invalid-read-syntax "#"

[Richard M. Stallman]

    Perhaps Emacs should provide a general purpose function (like
    `desktop-value-to-string') to print Lisp-readable expressions.

Doesn't prin1-to-string do that?

Re: savehist-mode: invalid-read-syntax "#"

[Juri Linkov]

>     Perhaps Emacs should provide a general purpose function (like
>     `desktop-value-to-string') to print Lisp-readable expressions.
>
> Doesn't prin1-to-string do that?

`read' still can fail with "invalid-read-syntax #" on a string returned
by `prin1-to-string', e.g.:

  (read (prin1-to-string (current-buffer)))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: savehist-mode: invalid-read-syntax "#"

[Juri Linkov]

>> Perhaps Emacs should provide a general purpose function (like
>> `desktop-value-to-string') to print Lisp-readable expressions.
>
> You can't make buffers, windows, overlays, yes even markers
> "Lisp-readable".  They are just not restorable by the Lisp reader and
> so one should find out what causes them to be in the history in the
> first place.

Please look at the functions `desktop-value-to-string' and
`desktop-internal-v2s'.  Surely, they can't keep and restore the
values of buffers, windows, overlays and markers, but at least
they don't create unreadable strings.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: savehist-mode: invalid-read-syntax "#"

[Reiner Steib]

On Wed, Jan 04 2006, Reiner Steib wrote:

> Maybe there should be a list (say `savehist-ignores-variables') which
> should contain `command-history' (and maybe other variables).

What about the following patch?  I will install it in Emacs CVS unless
someone comes up with a better solution.

--8<---------------cut here---------------start------------->8---
--- savehist.el	04 Jan 2006 15:58:31 +0100	1.14
+++ savehist.el	08 Jan 2006 20:23:19 +0100	
@@ -56,6 +56,7 @@
 
 (defgroup savehist nil
   "Save minibuffer history."
+  :version "22.1"
   :group 'minibuffer)
 
 ;;;###autoload
@@ -91,6 +92,11 @@
   :type '(repeat variable)
   :group 'savehist)
 
+(defcustom savehist-ignored-variables '(command-history)
+  "*List of additional variables not to save."
+  :type '(repeat variable)
+  :group 'savehist)
+
 (defcustom savehist-file
   (cond
    ;; Backward compatibility with previous versions of savehist.
@@ -371,9 +377,10 @@
 	(error nil))))))
 
 (defun savehist-minibuffer-hook ()
-  ;; XEmacs sets minibuffer-history-variable to t to mean "no history
-  ;; is being recorded".
-  (unless (eq minibuffer-history-variable t)
+  (unless (or (eq minibuffer-history-variable t)
+	      ;; XEmacs sets minibuffer-history-variable to t to mean "no
+	      ;; history is being recorded".
+	      (memq minibuffer-history-variable savehist-ignored-variables))
     (add-to-list 'savehist-minibuffer-history-variables
 		 minibuffer-history-variable)))
 
--8<---------------cut here---------------end--------------->8---

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: savehist-mode: invalid-read-syntax "#"

[Hrvoje Niksic]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Wed, Jan 04 2006, Reiner Steib wrote:
>
>> Maybe there should be a list (say `savehist-ignores-variables') which
>> should contain `command-history' (and maybe other variables).
>
> What about the following patch?  I will install it in Emacs CVS
> unless someone comes up with a better solution.

Are you sure that is a better idea than tracing the problem?  I use
Emacs 21, and I don't have `command-history' in
savehist-minibuffer-history-variables.  IMHO if a variable is not a
minibuffer history (i.e. a list of Lisp strings), it shouldn't end up
in minibuffer-history-variable.  If this is the case in CVS Emacs,
then the problem should be fixed rather than worked around.

Re: savehist-mode: invalid-read-syntax "#"

[Reiner Steib]

On Sun, Jan 08 2006, Hrvoje Niksic wrote:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>> What about the following patch?  I will install it in Emacs CVS
>> unless someone comes up with a better solution.
[ defcustom savehist-ignored-variables '(command-history) ]

> Are you sure that is a better idea than tracing the problem?

Unfortunately I don't know how to reproduce the problem.  But the
problem is quite serious.  It is quite hard for ordinary users to find
out how to solve the problem so we need a fix.  Additionally, I think
this feature (not saving some variables) might be a good idea anyhow.

I have added the following to `savehist-minibuffer-hook' in my copy
and will see if it happens again:

  (when (eq minibuffer-history-variable 'command-history)
    (debug nil minibuffer-history-variable))

> I use Emacs 21, and I don't have `command-history' in
> savehist-minibuffer-history-variables.  IMHO if a variable is not a
> minibuffer history (i.e. a list of Lisp strings), it shouldn't end
> up in minibuffer-history-variable.  If this is the case in CVS
> Emacs, then the problem should be fixed rather than worked around.

I don't know if this is intended behavior in CVS Emacs.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: savehist-mode: invalid-read-syntax "#"

[Hrvoje Niksic]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

>> Are you sure that is a better idea than tracing the problem?
>
> Unfortunately I don't know how to reproduce the problem.

You could modify savehist-minibuffer-hook (the function) to enter the
debugger when command-history is added to the hook.  It doesn't happen
for me in Emacs 21.  (I now see that you have done this already -- I'd
be interested in your findings.)

> But the problem is quite serious.  It is quite hard for ordinary
> users to find out how to solve the problem so we need a fix.

If my understanding is correct, you refer to the CVS Emacs, which has
not yet been released so should not be used by ordinary users.  If we
install this fix, it will be that much harder to trace the underlying
problem.

Re: savehist-mode: invalid-read-syntax "#"

[Reiner Steib]

On Mon, Jan 09 2006, Hrvoje Niksic wrote:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
>>> Are you sure that is a better idea than tracing the problem?
>>
>> Unfortunately I don't know how to reproduce the problem.
>
> You could modify savehist-minibuffer-hook (the function) to enter the
> debugger when command-history is added to the hook.  It doesn't happen
> for me in Emacs 21.  (I now see that you have done this already -- I'd
> be interested in your findings.)

I've been using this debug statement since Jan 09 and it wasn't
triggered anymore.  A Heisenbug...  ;-)

>> But the problem is quite serious.  It is quite hard for ordinary
>> users to find out how to solve the problem so we need a fix.
>
> If my understanding is correct, you refer to the CVS Emacs, which has
> not yet been released so should not be used by ordinary users.  If we
> install this fix, it will be that much harder to trace the underlying
> problem.

We could add the variable `savehist-ignored-variables' defaulting to
nil.  It might be useful on it's own independent on my problem.  And
if the problem happens, the user can add the history variable there
without patching `savehist.el'.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: savehist-mode: invalid-read-syntax "#"

[Reiner Steib]

On Fri, Mar 03 2006, Reiner Steib wrote:

> We could add the variable `savehist-ignored-variables' defaulting to
> nil.  It might be useful on it's own independent on my problem.  And
> if the problem happens, the user can add the history variable there
> without patching `savehist.el'.

Nobody objected, so I installed this.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/