From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Chris Findeisen Newsgroups: gmane.emacs.bugs Subject: bug#30943: save-hist creates massive cache file Date: Tue, 27 Mar 2018 02:50:57 +0000 Message-ID: References: <83o9jag9wg.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000dd35c605685bf7a8" X-Trace: blaine.gmane.org 1522119014 26322 195.159.176.226 (27 Mar 2018 02:50:14 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 27 Mar 2018 02:50:14 +0000 (UTC) Cc: 30943@debbugs.gnu.org To: eliz@gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Tue Mar 27 04:50:10 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1f0egb-0006li-F3 for geb-bug-gnu-emacs@m.gmane.org; Tue, 27 Mar 2018 04:50:09 +0200 Original-Received: from localhost ([::1]:60068 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f0eie-0002x1-Uv for geb-bug-gnu-emacs@m.gmane.org; Mon, 26 Mar 2018 22:52:16 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58332) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f0eiT-0002wo-42 for bug-gnu-emacs@gnu.org; Mon, 26 Mar 2018 22:52:06 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f0eiQ-0007av-HG for bug-gnu-emacs@gnu.org; Mon, 26 Mar 2018 22:52:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:47097) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f0eiQ-0007ap-EO for bug-gnu-emacs@gnu.org; Mon, 26 Mar 2018 22:52:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1f0eiQ-0002QV-51 for bug-gnu-emacs@gnu.org; Mon, 26 Mar 2018 22:52:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Chris Findeisen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 27 Mar 2018 02:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 30943 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 30943-submit@debbugs.gnu.org id=B30943.15221191029303 (code B ref 30943); Tue, 27 Mar 2018 02:52:02 +0000 Original-Received: (at 30943) by debbugs.gnu.org; 27 Mar 2018 02:51:42 +0000 Original-Received: from localhost ([127.0.0.1]:54994 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1f0ei5-0002Py-Tw for submit@debbugs.gnu.org; Mon, 26 Mar 2018 22:51:42 -0400 Original-Received: from mail-it0-f41.google.com ([209.85.214.41]:38918) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1f0ei3-0002Pl-MX for 30943@debbugs.gnu.org; Mon, 26 Mar 2018 22:51:40 -0400 Original-Received: by mail-it0-f41.google.com with SMTP id e98-v6so13333184itd.4 for <30943@debbugs.gnu.org>; Mon, 26 Mar 2018 19:51:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Ix0ZeWZBjpMPXcrKbCJNMqvsqvQJSYPcxW+y0nixtPg=; b=qSIKGSjOT/gwyjI0J717Htm+4tUVXjGnGdQyhVv70rcuvY57suZol4/vZxCFYczyRW nzTnU5MNc+uRvG6LHQWYHwQIAAdAClcKSLidmvKfjoEKYJyanDHTqN3nhaPHPDlpONCc peIK7723X1EFFFh44UbHBI88EsqP4KqraufN48riTG/KadsSlFfyoJEGcZEGObOaUHcf u9tLaYcM7xf4WIpXa/xeM8ee/1ak3GjYCKZQA1YnJgrXP79gKhuQ2dESva9LXt+WQGvq nrjWgM0tk6vPP1oMnex622WRhVhwTGVtn6ZpKKyOZkKZgDKWB8u+bnpAH3qsAzUDvTjw H3Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Ix0ZeWZBjpMPXcrKbCJNMqvsqvQJSYPcxW+y0nixtPg=; b=NSHw+77+q7wLEZtSA6QaKWBQ5eh3GpgTGoXK0irrKmxkvgXZuoLXxcS9frNnH/Rxay YvbP6DsllAe1m3/Bzv33TawIMweGmKpNQP458Qi/Ak8vaEJRpAjq4SfCY7uwfCGP0tAX OHHXhf+DCj4HVb0P8nuXk7/XLdbXwmqDoUezmmYehKgRjNFgpfn8HKcmGXObvPOhsMBl ANBE3xAa7krut+npGirDvR1YgJtCdCRH+uRWDJ5Ody1raWiCdK/5ril3FGPq3Vkf3NO7 N7yzZstEOb2YOS6m8hVLxLkAjMZxDBHBcFnbzYHRCZuZofYRR2QdxZ0z8taxJbh2Gjkd g+EA== X-Gm-Message-State: AElRT7GmpPOWvMT5nQkhE1xJ0mpc0GEy8Q2c8YrPHsOGgJN3UU9vTceb mIrQtyUsIKN+vHRlqZbWSUowme//DJ6zVdWSLqUeBQ== X-Google-Smtp-Source: AIpwx49cdGLWM21m44kcTHOzSLIzm5zzThec1lzERc8GhOYZalJkcQSwd672YNzZiVnfCdTAAn7iyWMeGe2pzaVaRAA= X-Received: by 2002:a24:1f15:: with SMTP id d21-v6mr24949574itd.71.1522119093305; Mon, 26 Mar 2018 19:51:33 -0700 (PDT) In-Reply-To: <83o9jag9wg.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:144644 Archived-At: --000000000000dd35c605685bf7a8 Content-Type: text/plain; charset="UTF-8" Hey Eli, Correct, the issue is that the variable is growing from session to session without limitation. For instance, some of the culprits from my particular configuration: grep -E -b -o '^\(setq [^ ]+' ~/.emacs.d/.cache/savehist .... 51869:(setq evil-jumps-history 53265:(setq mark-ring 53287:(setq search-ring 53311:(setq regexp-search-ring 53579:(setq extended-command-history I've also seen online this issue crop up for others: https://emacs.stackexchange.com/questions/12086/abnormally-large-savehist-file Save-hist doesn't truncate any variables except the ones it knows > about, because it cannot be sure that truncating some list might leave > the variable with an invalid value. For variables like > minibuffer-history, it knows that truncating the history will produce > a history that is still valid, but how can it do the same with other variables? For example, it could be that some other variable needs to > be truncated from the other end of the list, otherwise the value will > be useless. Hmm, I understand the dilemma about not knowing the format, however there ought to be an alternative way of adding variables that we can "inform" save-hist about. Whether that's demanding/documenting a valid format, "Hey if you add this variable to be tracked by save-hist, it must be formatted like so" or providing a secondary argument that takes a function specifying custom handling, I'm not sure. I think it makes sense to provide some method of honoring the history-length variable. If the list is never culled, any emacs session will just grow over time. That is almost never desired(at least for me), and I would see no point to have additional-variables to be track (I would have to worry about my buffer silently slowing me down). Let me know if I'm missing something, or if you have alternate ideas. Thanks! Regards, Chris Findeisen On Mon, Mar 26, 2018 at 7:59 AM Eli Zaretskii wrote: > > From: Chris Findeisen > > Date: Sun, 25 Mar 2018 19:11:21 +0000 > > > > save-hist-additional-variables never get truncated by save-hist, leading > > to a massive cache file and slowdown. Practically, this matters when the > cache file silently grows to 1/2 a > > GB, and emacs begins randomly freezing. > > > > history-length is supposed to keep a limit on the max history for > save-hist-additional-variables, but it doesn't. > > > > In the savehist-save function: > > > > (dolist (symbol savehist-minibuffer-history-variables) > > (when (and (boundp symbol) > > (not (memq symbol savehist-ignored-variables))) > > (let ((value (savehist-trim-history (symbol-value symbol))) > > ;;.... > > )))) > > > > (dolist (symbol savehist-additional-variables) > > (when (boundp symbol) > > (let ((value (symbol-value symbol))) > > (when (savehist-printable value) > > ;; ... > > )))) > > > > As you can see, the save-hist-trim-history fn is not called in the > > second code block. > > I'm not sure I understand the situation. Is the variable you are > trying to save using save-hist a long list or something, and it keeps > growing from session to session without any limitation? > > Save-hist doesn't truncate any variables except the ones it knows > about, because it cannot be sure that truncating some list might leave > the variable with an invalid value. For variables like > minibuffer-history, it knows that truncating the history will produce > a history that is still valid, but how can it do the same with other > variables? For example, it could be that some other variable needs to > be truncated from the other end of the list, otherwise the value will > be useless. > > If I'm missing something about your use case, please tell the details, > and perhaps show an example of a variable with which this happens. > > Thanks. > --000000000000dd35c605685bf7a8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hey Eli,=C2=A0

Correct, the issue is th= at the variable is growing from session to session without limitation.

For instance, some of the culprits from my particular = configuration:
grep -E -b -o '^\(setq [^ ]+' ~/.emac= s.d/.cache/savehist
....
51869:(setq evil-jumps-h= istory
53265:(setq mark-ring
53287:(setq search-ring
53311:(setq regexp-search-ring
53579:(setq extended-= command-history

I've also seen online th= is issue crop up for others: https://emac= s.stackexchange.com/questions/12086/abnormally-large-savehist-file
<= br class=3D"m_2272368822607271724gmail-m_636073064985831169gmail-Apple-inte= rchange-newline">
Save-hist doesn't truncate any variables except the on= es it knows
about, because it cannot be sure t= hat truncating some list might leave
the varia= ble with an invalid value.=C2=A0 For variables like
minibuffer-history, it knows that truncating the history will produ= ce
a history that is still valid, but how can = it do the same with other
variables?=C2=A0
For example, it could be that some other variable needs to
<= span style=3D"color:rgb(34,34,34);font-family:sans-serif;font-size:13px;fon= t-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-= weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-tran= sform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255= ,255);text-decoration-style:initial;text-decoration-color:initial;float:non= e;display:inline">be truncated from the other end of the list, otherwise th= e value will
be useless.

Hmm, I understand the dilemma about no= t knowing the format, however there ought to be an alternative way o= f adding variables that we can "inform" save-hist about. Whether = that's demanding/documenting a valid format, "Hey if you add this = variable to be tracked by save-hist, it must be formatted like so" or = providing a secondary argument that takes a function specifying custom hand= ling, I'm not sure.=C2=A0

I think it makes sen= se to provide some method of honoring the history-length variable. If the l= ist is never culled, any emacs session will just grow over time. That is al= most never desired(at least for me), and I would see no point to have addit= ional-variables to be track (I would have to worry about my buffer silently= slowing me down).=C2=A0

Let me know if I'm mi= ssing something, or if you have alternate ideas. Thanks!

Regard= s,
Chris Findeisen

On Mon, Mar 26, 2018= at 7:59 AM Eli Zaretskii <eliz@gnu.org> wrote:
> From: Chris Findeisen <cfindeisen@google.com>
> Date: Sun, 25 Mar 2018 19:11:21 +0000
>
> save-hist-additional-variables never get truncated by save-hist, leadi= ng
> to a massive cache file and slowdown. Practically, this matters when t= he cache file silently grows to 1/2 a
> GB, and emacs begins randomly freezing.
>
> history-length is supposed to keep a limit on the max history for save= -hist-additional-variables, but it doesn't.
>
> In the savehist-save function:
>
> (dolist (symbol savehist-minibuffer-history-variables)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(when (and (boundp symbol)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 (not (memq symbol savehist-ignored-variables)))
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(let ((value (savehist-= trim-history (symbol-value symbol)))
>=C2=A0 =C2=A0 =C2=A0 =C2=A0;;....
>=C2=A0 =C2=A0))))
>
> (dolist (symbol savehist-additional-variables)
>=C2=A0 =C2=A0(when (boundp symbol)
>=C2=A0 =C2=A0 =C2=A0(let ((value (symbol-value symbol)))
>=C2=A0 =C2=A0 =C2=A0(when (savehist-printable value)
>=C2=A0 =C2=A0 =C2=A0;; ...
>=C2=A0 =C2=A0 =C2=A0))))
>
> As you can see, the save-hist-trim-history fn is not called in the
> second code block.

I'm not sure I understand the situation.=C2=A0 Is the variable you are<= br> trying to save using save-hist a long list or something, and it keeps
growing from session to session without any limitation?

Save-hist doesn't truncate any variables except the ones it knows
about, because it cannot be sure that truncating some list might leave
the variable with an invalid value.=C2=A0 For variables like
minibuffer-history, it knows that truncating the history will produce
a history that is still valid, but how can it do the same with other
variables?=C2=A0 For example, it could be that some other variable needs to=
be truncated from the other end of the list, otherwise the value will
be useless.

If I'm missing something about your use case, please tell the details,<= br> and perhaps show an example of a variable with which this happens.

Thanks.
--000000000000dd35c605685bf7a8--