Turning on savehist-mode by default

Turning on savehist-mode by default

[sbaugh]

savehist-mode is a useful mode which is turned on by many Emacs
users. [1] It matches the default behavior of programs like bash and
vim, which save command history by default.  I suggest that we should
find some way to enable savehist by default.

By default savehist creates a timer which saves the history every 5
minutes.  I suggest that by default we should just have behavior like:
(add-hook 'kill-emacs-hook #'savehist-autosave)
If the user explicitly runs (savehist-mode 1) then that would enable the
timer-based saving.

I'm not sure what other issues have prevented savehist from being turned
on by default, but I'm happy to do the work to get it into good shape.

This proposal is somewhat motivated by bug#66993, because project.el
already has some custom code which runs by default which is kind of like
savehist-mode, but which only save project--list.  It may be possible to
get rid of that code in favor of the generic savehist logic, but to do
that we'd need to turn savehist on by default in some form.

[1]: At least, savehist is turned on by every Emacs configuration
framework I've checked, in a brief search of the most popular ones:
https://git.sr.ht/~technomancy/better-defaults/tree/main/item/better-defaults.el
https://github.com/doomemacs/doomemacs/blob/986398504d09e585c7d1a8d73a6394024fe6f164/lisp/doom-editor.el#L356
https://github.com/syl20bnr/spacemacs/blob/a6a834f08af61b5a4dd80e337449f0f7026f6a0c/layers/%2Bspacemacs/spacemacs-defaults/packages.el#L401
https://github.com/bbatsov/prelude/blob/b57ff48e0985a6ef0f1ed9b279ec487c55982334/core/prelude-editor.el#L108
https://github.com/pprevos/emacs-writing-studio/blob/master/init.el#L247
https://systemcrafters.net/emacs-from-scratch/the-best-default-settings/

RE: [External] : Turning on savehist-mode by default

[Drew Adams]

> savehist-mode is a useful mode which is turned on by many Emacs
> users. It matches the default behavior of programs like bash and
> vim, which save command history by default.  I suggest that we
> should find some way to enable savehist by default.

Why?  It's trivial to turn it on.  As you
say, many users (moi aussi) do so.

There are a zillion minor modes that many
users find useful to turn on by default.
It doesn't follow that `emacs -Q' should
turn on any of them by default.

`kill-emacs-hook' should definitely be nil
by default.

> what other issues have prevented savehist
> from being turned on by default

Wrong question.

Re: [External] : Turning on savehist-mode by default

[Philip Kaludercic]

Drew Adams <drew.adams@oracle.com> writes:

>> savehist-mode is a useful mode which is turned on by many Emacs
>> users. It matches the default behavior of programs like bash and
>> vim, which save command history by default.  I suggest that we
>> should find some way to enable savehist by default.
>
> Why?  It's trivial to turn it on.  As you
> say, many users (moi aussi) do so.
>
> There are a zillion minor modes that many
> users find useful to turn on by default.
> It doesn't follow that `emacs -Q' should
> turn on any of them by default.

If something is done by (practically) everyone, especially when it is
something that (practically) all beginners would be interested in, I
wouldn't dismiss the proposal to enable it by default.  How many
features satisfy this condition shouldn't matter.  After all, there are
already a number of features that are enabled by default, like
show-paren-mode, that individually are easy to enable, if you know how
to do it.  The issue is that beginners neither know how to do it, nor
what all the options are that they might be interested in.

IMO the more important question when considering to enable a feature
OOTB, is if it will inconvenience any existing users.  I don't think
there should be any issues with savehist-mode, but I might not know of
some issue?  Been using it myself as well for a while.

On a related topic, I have been running a configuration generator that
has been mentioned here on the list for a while now
(https://emacs.amodernist.com/), that could provide opt-in participation
in statistics of what features people are interested in.  That might
help ground matters like these in somewhat more empirical data, or least
a different perspective.

-- 
Philip Kaludercic

RE: [External] : Turning on savehist-mode by default

[Drew Adams]

> > There are a zillion minor modes that many
> > users find useful to turn on by default.
> > It doesn't follow that `emacs -Q' should
> > turn on any of them by default.
> 
> If something is done by (practically) everyone,

Is it?

> especially when it is something that (practically)
> all beginners would be interested in,

How is that known?

> I wouldn't dismiss the proposal to enable it by default.

I don't dismiss it.

> The issue is that beginners neither know how to do it, nor
> what all the options are that they might be interested in.

And yet it's "done by (practically) everyone"?

___

Let's enable `delete-selection-mode' by default.

It took decades to get `transient-mark-mode' ON
by default.  `delete-selection-mode' completes
that job.  It welcomes new users with the same
type-to-replace behavior they're used to outside
Emacs (everywhere).

Persists nothing.  Is easy for anyone not who
doesn't want it ON to turn OFF.

Re: [External] : Turning on savehist-mode by default

[Philip Kaludercic]

Drew Adams <drew.adams@oracle.com> writes:

>> > There are a zillion minor modes that many
>> > users find useful to turn on by default.
>> > It doesn't follow that `emacs -Q' should
>> > turn on any of them by default.
>> 
>> If something is done by (practically) everyone,
>
> Is it?

Given that my sentence was a hypothetical, I cannot answer this question
since the term "something" cannot be unambiguously instantiated as to be
answered concretely.

>> especially when it is something that (practically)
>> all beginners would be interested in,
>
> How is that known?

It is not known, otherwise there wouldn't really be a discussion.  Our
knowledge can only approximate reality through experience and talking
with different kinds of Emacs users.

Taking the example of savehist-mode, then my experience, which takes
different kinds of users, of different experience levels, people I have
met online and in-person, appears to indicate that this is a popular and
useful feature.

>> I wouldn't dismiss the proposal to enable it by default.
>
> I don't dismiss it.

So you don't oppose enabling savehist-mode by default?

>> The issue is that beginners neither know how to do it, nor
>> what all the options are that they might be interested in.
>
> And yet it's "done by (practically) everyone"?

Let us say, "(practically) everyone" who manages to stay along, by
finding the right options to create a comfortable and productive
environment for themselves.  There are certainly many beginners that
never change this user option; but I suspect that these are also the
ones that never get to taking a look at any user options, because they
give up too soon.

> ___
>
> Let's enable `delete-selection-mode' by default.
>
> It took decades to get `transient-mark-mode' ON
> by default.  `delete-selection-mode' completes
> that job.  It welcomes new users with the same
> type-to-replace behavior they're used to outside
> Emacs (everywhere).
>
> Persists nothing.  Is easy for anyone not who
> doesn't want it ON to turn OFF.

One has to keep in mind that there are a lot of people who use Emacs,
and are familiar with the "feel" of the default key bindings or at least
some subset of these, without having much of an understanding of how to
do things or what is going on.  These are users that should nevertheless
be respected -- hence my point that enabling a feature has to take the
workflow of people into account, for whom a change would break an
expectation.

Note that I am *not* saying that the goal should be to accommodate
newcomers (following whatever current trends may be) at any cost,
especially when this comes at the expense of long-term users.

To make this argument with savehist-mode, one would have to make the
use-case believable, that someone expects the history of mini-buffer
input to not persist between sessions.  I think that is a claim that it
a lot harder to justify, than that inserting a key while a selection is
active, replaces the selection.

-- 
Philip Kaludercic

RE: [External] : Turning on savehist-mode by default

[Drew Adams]

> >> I wouldn't dismiss the proposal to enable it by default.
> >
> > I don't dismiss it.
> 
> So you don't oppose enabling savehist-mode by default?

I don't _dismiss_ it.

___

https://www.wordwebonline.com/search.pl?w=dismiss

WordWeb tells you:

Verb: dismiss

Bar from attention or consideration

"She dismissed his advances";
- disregard, brush aside, brush off [informal], discount, push aside, ignore
 
Cease to consider; put out of judicial consideration

"This case is dismissed!";
- throw out
 
...
 
Declare void

"The President dismissed the parliament and called for new elections";
- dissolve

> >> The issue is that beginners neither know how to do it, nor
> >> what all the options are that they might be interested in.
> >
> > And yet it's "done by (practically) everyone"?
> 
> Let us say, "(practically) everyone" who manages to stay along, by
> finding the right options to create a comfortable and productive
> environment for themselves.  There are certainly many beginners that
> never change this user option; but I suspect that these are also the
> ones that never get to taking a look at any user options, because they
> give up too soon.

So nearly all of those who (1) tried savehist
and (2) didn't give up on getting comfortable
and productive with it by fiddling with its
options.

That's a far cry from "(practically) everyone".
But OK, it's clear now: Most who persisted at
trying to use savehist continue to use it -
that's the claim now.  Might be the case.

> > ___
> >
> > Let's enable `delete-selection-mode' by default.
> >
> > It took decades to get `transient-mark-mode' ON
> > by default.  `delete-selection-mode' completes
> > that job.  It welcomes new users with the same
> > type-to-replace behavior they're used to outside
> > Emacs (everywhere).
> >
> > Persists nothing.  Is easy for anyone not who
> > doesn't want it ON to turn OFF.
> 
> One has to keep in mind that there are a lot of people who use Emacs,
> and are familiar with the "feel" of the default key bindings or at least
> some subset of these, without having much of an understanding of how to
> do things or what is going on.  These are users that should nevertheless
> be respected -- hence my point that enabling a feature has to take the
> workflow of people into account, for whom a change would break an
> expectation.

Not too worry.  One has been keeping this in mind
for many decades...

One kept it in mind about `transient-mark-mode',
yet that was eventually turned ON by default.

(There are still users, today, who turn off
`transient-mark-mode', but likely many fewer than
if the default behavior hadn't changed to ON.)

> Note that I am *not* saying that the goal should be to accommodate
> newcomers (following whatever current trends may be) at any cost,
> especially when this comes at the expense of long-term users.

Nor am I.  Nor was that behind turning ON
`transient-mark-mode' by default.

> To make this argument with savehist-mode, one would have to make the
> use-case believable, that someone expects the history of mini-buffer
> input to not persist between sessions.  I think that is a claim that it
> a lot harder to justify, than that inserting a key while a selection is
> active, replaces the selection.

No, I don't think making that use case believable
is a requirement.  Nor does anyone need to justify
it as a claim.

But FWIW, I do bet that someone - likely even most
users! - currently expect minibuffer input history
to NOT persist.  If it isn't persisted then, hey,
why would someone expect that it is?

To me that expectation _is_ believable.  But I'm
not assuming that "(practically) everyone" turns
on saving of minibuffer histories.  I'm guessing
that more people don't than do.  Just a guess.

__

FWIW, `savehist.el' is a general way to save variable
values, not just a way to save values of minibuffer
history variables.  It's often recommended (including
by me) as a way to persist any variables you like.

Why isn't it written with that foremost in mind, i.e.,
as a general variable-persisting feature?  Why make
users fiddle option `savehist-additional-variables'
to be able to use its (real) functionality?  Why have
pure-Boolean option `savehist-save-minibuffer-history'
instead of just an option whose value is a list of
variables to persist?

Did it start with just the aim of saving minibuffer
histories, and later grafted on the more general
functionality as a wart?  I guess not:

It's not in Emacs 20; I don't have Emacs 21; and in
Emacs 22 there's already the current design, with
variable `savehist-additional-variables'.  The code
comment expressing the motivation says it provides
Emacs with what "many editors (e.g. Vim)" already
had: minibuffer-history persistence.  (Presumably
they had something similar to a minibuffer and its
input histories.)

I understand that it treats minibuffer histories
specially, in that when you use a new history var
that automatically gets included in the list of
vars to persist.  Still, the general functionality
is to persist a set of vars.

__

FWIW, I repeat that I'm a user/fan of savehist.
My value of `savehist-additional-variables':
`(search-ring regexp-search-ring)'.

Re: [External] : Turning on savehist-mode by default

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "sbaugh@catern.com" <sbaugh@catern.com>, "emacs-devel@gnu.org"
>  <emacs-devel@gnu.org>
> Date: Sat, 18 Nov 2023 23:01:28 +0000
> 
> > >> I wouldn't dismiss the proposal to enable it by default.
> > >
> > > I don't dismiss it.
> > 
> > So you don't oppose enabling savehist-mode by default?
> 
> I don't _dismiss_ it.

You are splitting hair.  Philip is not a native English speaker, and
he obviously meant "reject" when he said "dismiss".

RE: [External] : Turning on savehist-mode by default

[Drew Adams]

> > > >> I wouldn't dismiss the proposal to enable it by default.
> > > >
> > > > I don't dismiss it.
> > >
> > > So you don't oppose enabling savehist-mode by default?
> >
> > I don't _dismiss_ it.
> 
> You are splitting hair.  Philip is not a native English speaker, and
> he obviously meant "reject" when he said "dismiss".

Did he, obviously?  Then my showing the definition
of `dismiss' might hopefully have helped.

I don't _reject_ it either.  (Not that it's in my
power to do that.  The most I could do is disagree
with it.)

I may in fact be the only person who's even been
considering it.  No one else has spoken up so far.

I haven't seen a good reason to turn it ON by
default - so far.  But maybe there is one.  I've
made clear that I myself use savehist and I often
recommend it.

Persisting minibuffer input has been opt-in, so
far.  I do think it's generally advisable for
users to be aware that what they type will be
persisted, when that's the case.

That applies to input history, search history,
bookmarks, mail drafts, etc.  Automatically
saving predefined thingies (e.g. Customize
settings) is less problematic in this regard
than saving arbitrary typed text/input.

This might be only a minor consideration -
dunno.  What do you think should be considered
when weighing the proposal?

Re: Turning on savehist-mode by default

[Po Lu]

sbaugh@catern.com writes:

> savehist-mode is a useful mode which is turned on by many Emacs
> users. [1] It matches the default behavior of programs like bash and
> vim, which save command history by default.  I suggest that we should
> find some way to enable savehist by default.
>
> By default savehist creates a timer which saves the history every 5
> minutes.  I suggest that by default we should just have behavior like:
> (add-hook 'kill-emacs-hook #'savehist-autosave)
> If the user explicitly runs (savehist-mode 1) then that would enable the
> timer-based saving.
>
> I'm not sure what other issues have prevented savehist from being turned
> on by default, but I'm happy to do the work to get it into good shape.
>
> This proposal is somewhat motivated by bug#66993, because project.el
> already has some custom code which runs by default which is kind of like
> savehist-mode, but which only save project--list.  It may be possible to
> get rid of that code in favor of the generic savehist logic, but to do
> that we'd need to turn savehist on by default in some form.

There is a grave disadvantage to introducing kill-emacs-hooks or saving
files periodically from a timer.  In the case of the former, Emacs will
behave inconsistently, subject to whether it's been killed such that the
hook cannot run.  And the latter runs the risk of files being corrupted
if the file is being saved, by chance, the instant Emacs is killed.

You might object that people are always meant to shut Emacs down in a
way that prompts it to execute these hooks, but I've always considered
it a virtue of Emacs's that it can be so terminated without consequence.

Departing from this in our default configuration would be a step
backwards, inasmuch as we constantly hear of users suffering from long
startup times, implying that Emacs is frequently started for momentary
editing sessions.  And in such scenarios users are apt to kill Emacs by
whatever means are then at hand: xkill, pkill, and the like, this being
behavior that I have myself engaged in on occasion.

As regards enabling savehist-mode by default, I have no strong
objections.  Provided that you can pull it off without either timers or
entries in kill-emacs-hook.

Re: [External] : Turning on savehist-mode by default

[Juri Linkov]

> FWIW, `savehist.el' is a general way to save variable
> values, not just a way to save values of minibuffer
> history variables.  It's often recommended (including
> by me) as a way to persist any variables you like.

Please don't forget there is also `desktop.el'
that is a superset of `savehist.el'

> Why isn't it written with that foremost in mind, i.e.,
> as a general variable-persisting feature?

A general variable-persisting feature already exists:
`define-multisession-variable'.

Re: Turning on savehist-mode by default

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Sun, 19 Nov 2023 14:59:47 +0800
> 
> There is a grave disadvantage to introducing kill-emacs-hooks or saving
> files periodically from a timer.

I disagree that there's a disadvantage in that, let alone a grave one.
I'm using desktop.el, which does both of the above, for many years,
and have yet to see any problems with that.

Re: Turning on savehist-mode by default

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> I disagree that there's a disadvantage in that, let alone a grave one.
> I'm using desktop.el, which does both of the above, for many years,
> and have yet to see any problems with that.

I know this is a disadvantage for one subset of our users, because I
often witness them kill Emacs in precisely the fashion I mentioned
earlier.  And that's if the lock files strewn everywhere belonging to
long-dead Emacs sessions aren't evidence enough.

Re: [External] : Turning on savehist-mode by default

[Yuri Khan]

On Sun, 19 Nov 2023 at 14:11, Juri Linkov <juri@linkov.net> wrote:

> Please don't forget there is also `desktop.el'
> that is a superset of `savehist.el'

Is it though? I’m using ‘desktop-save-mode’, not observing
preservation of histories, and not seeing any user option in the
Desktop group that would shout to me “enable this to save all
histories”. I see ‘file-name-history’ in ‘desktop-globals-to-save’ but
I wouldn’t like to whitelist all the histories I care about.

Re: Turning on savehist-mode by default

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: sbaugh@catern.com,  emacs-devel@gnu.org
> Date: Sun, 19 Nov 2023 15:27:29 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I disagree that there's a disadvantage in that, let alone a grave one.
> > I'm using desktop.el, which does both of the above, for many years,
> > and have yet to see any problems with that.
> 
> I know this is a disadvantage for one subset of our users, because I
> often witness them kill Emacs in precisely the fashion I mentioned
> earlier.  And that's if the lock files strewn everywhere belonging to
> long-dead Emacs sessions aren't evidence enough.

People are free to shoot themselves in the foot, but they shouldn't
then come back to use crying that it hurts.  If someone kills Emacs
with a fatal signal, they should not be surprised that their files are
sometimes corrupted.

Anyway, IME, the files are usable in the next session even if the
previous session was terminated abnormally, like when AC power goes
down.  I don't remember even a single instance of having unusable
files saved by a timer or kill-emacs-hook even though my sessions are
not always terminated normally.

Re: [External] : Turning on savehist-mode by default

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 19 Nov 2023 14:32:57 +0700
> Cc: Drew Adams <drew.adams@oracle.com>, Philip Kaludercic <philipk@posteo.net>,
>  "sbaugh@catern.com" <sbaugh@catern.com>,
>  "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> 
> On Sun, 19 Nov 2023 at 14:11, Juri Linkov <juri@linkov.net> wrote:
> 
> > Please don't forget there is also `desktop.el'
> > that is a superset of `savehist.el'
> 
> Is it though? I’m using ‘desktop-save-mode’, not observing
> preservation of histories, and not seeing any user option in the
> Desktop group that would shout to me “enable this to save all
> histories”. I see ‘file-name-history’ in ‘desktop-globals-to-save’ but
> I wouldn’t like to whitelist all the histories I care about.

Why not?  I do precisely that, FWIW.  It's a one-time configuration,
so not a lot of effort.

Re: [External] : Turning on savehist-mode by default

[Yuri Khan]

On Sun, 19 Nov 2023 at 15:26, Eli Zaretskii <eliz@gnu.org> wrote:

> > > Please don't forget there is also `desktop.el'
> > > that is a superset of `savehist.el'
> >
> > Is it though? I’m using ‘desktop-save-mode’, not observing
> > preservation of histories, and not seeing any user option in the
> > Desktop group that would shout to me “enable this to save all
> > histories”. I see ‘file-name-history’ in ‘desktop-globals-to-save’ but
> > I wouldn’t like to whitelist all the histories I care about.
>
> Why not?

Why would I? Histories are user data. User data is precious. Losing
user data on a restart is a bug. (Slightly exaggerating.)

I can see three reasons for *not* saving histories: (1) risk of
information disclosure if sensitive data such as passwords is stored
in histories; (2) reduced startup performance if too much history gets
accumulated; and (3) cross-process interference if the application is
typically running in multiple instances. (1) is solved by a blacklist
(opt-out for a few specific histories), (2) by pruning saved histories
to some configurable maximum depth, and (3) by saying Emacs is not
typically used like that[citation needed].

(I remember seeing an implementation of input histories that also
allowed pinning important entries so they would never be pruned, and
explicitly deleting specific entries, right from the UI that displayed
history entries.)

> I do precisely that, FWIW.  It's a one-time configuration,
> so not a lot of effort.

It’s only a one-time configuration if your workflow is constant. If I
start using a feature I had not been using previously, its histories
will initially not be saved, and I’ll have to interrupt my work and
amend my configuration. So it’s a recurring cost.

Re: [External] : Turning on savehist-mode by default

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 19 Nov 2023 16:06:17 +0700
> Cc: juri@linkov.net, drew.adams@oracle.com, philipk@posteo.net, 
> 	sbaugh@catern.com, emacs-devel@gnu.org
> 
> On Sun, 19 Nov 2023 at 15:26, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > > > Please don't forget there is also `desktop.el'
> > > > that is a superset of `savehist.el'
> > >
> > > Is it though? I’m using ‘desktop-save-mode’, not observing
> > > preservation of histories, and not seeing any user option in the
> > > Desktop group that would shout to me “enable this to save all
> > > histories”. I see ‘file-name-history’ in ‘desktop-globals-to-save’ but
> > > I wouldn’t like to whitelist all the histories I care about.
> >
> > Why not?
> 
> Why would I?

Because some histories you care about, whereas others you don't?
That's the situation here, FWIW.

> Histories are user data. User data is precious. Losing
> user data on a restart is a bug. (Slightly exaggerating.)

The "slightly" part is already exaggerated.

Not all histories are precious enough to save them.  They don't come
for free: you pay with memory for them, if not with longer search
times.

> I can see three reasons for *not* saving histories: (1) risk of
> information disclosure if sensitive data such as passwords is stored
> in histories; (2) reduced startup performance if too much history gets
> accumulated; and (3) cross-process interference if the application is
> typically running in multiple instances. (1) is solved by a blacklist
> (opt-out for a few specific histories), (2) by pruning saved histories
> to some configurable maximum depth, and (3) by saying Emacs is not
> typically used like that[citation needed].

Well, I think you just explained why whitelisting history variables,
and thus using desktop.el for that, is not such an outlandish idea.

One other argument for using desktop.el is if you already use it for
its other features.

> > I do precisely that, FWIW.  It's a one-time configuration,
> > so not a lot of effort.
> 
> It’s only a one-time configuration if your workflow is constant. If I
> start using a feature I had not been using previously, its histories
> will initially not be saved, and I’ll have to interrupt my work and
> amend my configuration. So it’s a recurring cost.

If you start using new features frequently enough, you haven't been
using Emacs long enough ;-)

Let's agree to disagree about the advantages and disadvantages of
whitelisting vs blacklisting.  Basically, both serve the same purpose,
so the decision which one to use depends on the length of the white
list vs the black list.  Which means either method is fine, and
there's no need to represent the other choice as something completely
senseless.

Re: Turning on savehist-mode by default

[Dmitry Gutov]

On 19/11/2023 08:59, Po Lu wrote:
> There is a grave disadvantage to introducing kill-emacs-hooks or saving
> files periodically from a timer.  In the case of the former, Emacs will
> behave inconsistently, subject to whether it's been killed such that the
> hook cannot run.  And the latter runs the risk of files being corrupted
> if the file is being saved, by chance, the instant Emacs is killed.

That would be a real problem if the information saved by savehist-mode 
were of grave importance.

Completion input histories are not usually in that category. People can 
repeat their inputs manually; it's just a time-saving feature.

Re: Turning on savehist-mode by default

[Po Lu]

Dmitry Gutov <dmitry@gutov.dev> writes:

> That would be a real problem if the information saved by savehist-mode
> were of grave importance.
>
> Completion input histories are not usually in that category. People
> can repeat their inputs manually; it's just a time-saving feature.

It is a nuisance for Emacs to display outdated completion history if it
is abruptly killed: anyone who has run multiple bash shells in parallel,
and thus played the guessing game of "which shell's history has been
preserved?" will concur.  Corruption is worse still.

Re: Turning on savehist-mode by default

[Dmitry Gutov]

On 19/11/2023 16:38, Po Lu wrote:
> Dmitry Gutov<dmitry@gutov.dev>  writes:
> 
>> That would be a real problem if the information saved by savehist-mode
>> were of grave importance.
>>
>> Completion input histories are not usually in that category. People
>> can repeat their inputs manually; it's just a time-saving feature.
> It is a nuisance for Emacs to display outdated completion history if it
> is abruptly killed: anyone who has run multiple bash shells in parallel,
> and thus played the guessing game of "which shell's history has been
> preserved?" will concur.  Corruption is worse still.

Yes. But when you play stupid games (e.g. pkill), you win stupid prizes.

Saving the history synchronously has other problems, e.g. exemplified in 
bug#66993.

Re: [External] : Turning on savehist-mode by default

[Spencer Baugh]

Yuri Khan <yuri.v.khan@gmail.com> writes:
> I can see three reasons for *not* saving histories:
>
> (1) risk of information disclosure if sensitive data such as passwords
> is stored in histories;
> (2) reduced startup performance if too much
> history gets accumulated; and
> (3) cross-process interference if the application is typically running
> in multiple instances.
>
> (1) is solved by a blacklist (opt-out for a few specific histories),
> (2) by pruning saved histories to some configurable maximum depth, and
> (3) by saying Emacs is not typically used like that[citation needed].

I agree with your problem statements and solutions.

To add further suggestions: maybe part of a solution for (2) would be
for emacs -Q to not load or save history (e.g. it doesn't enable
savehist-mode).  Possibly even also emacs -q.

That may also help with (3), because 99% of times I have multiple
Emacsen running, it's because I'm running a test emacs -Q or emacs -q.

(3) could also be solved by some intelligent merging of histories; it
would be relatively straightforward to merge the histories when saving
them.  Although probably unnecessary complexity - everyone has been
getting along fine without it.

Re: Turning on savehist-mode by default

[Spencer Baugh]

Po Lu <luangruo@yahoo.com> writes:
> Dmitry Gutov <dmitry@gutov.dev> writes:
>
>> That would be a real problem if the information saved by savehist-mode
>> were of grave importance.
>>
>> Completion input histories are not usually in that category. People
>> can repeat their inputs manually; it's just a time-saving feature.
>
> It is a nuisance for Emacs to display outdated completion history if it
> is abruptly killed:

I agree that outdated completion history is a nuisance.  However, isn't
outdated completion history better than none?

> anyone who has run multiple bash shells in parallel,
> and thus played the guessing game of "which shell's history has been
> preserved?" will concur.

I definitely find the bash history experience extremely annoying.  It
happens even when bash is exited cleanly - it's just a consequence of
the fact that bash overwrites the previous history when exiting, rather
than combining them in some way.

Emacs isn't commonly run in parallel, and when it is, at least for me
there's always one main Emacs instance which outlives the others.  So we
shouldn't have the bash experience.  But still: I'd be happy to
implement some history merging behavior so we don't get the bash history
experience, if that's a requirement for savehist-mode to be turned on by
default.

Re: [External] : Turning on savehist-mode by default

[Visuwesh]

[சனி நவம்பர் 18, 2023] Philip Kaludercic wrote:

>>> especially when it is something that (practically)
>>> all beginners would be interested in,
>>
>> How is that known?
>
> It is not known, otherwise there wouldn't really be a discussion.  Our
> knowledge can only approximate reality through experience and talking
> with different kinds of Emacs users.
>
> Taking the example of savehist-mode, then my experience, which takes
> different kinds of users, of different experience levels, people I have
> met online and in-person, appears to indicate that this is a popular and
> useful feature.
>
> [...]
>>> The issue is that beginners neither know how to do it, nor
>>> what all the options are that they might be interested in.
>>
>> And yet it's "done by (practically) everyone"?
>
> Let us say, "(practically) everyone" who manages to stay along, by
> finding the right options to create a comfortable and productive
> environment for themselves.  There are certainly many beginners that
> never change this user option; but I suspect that these are also the
> ones that never get to taking a look at any user options, because they
> give up too soon.

FWIW, I came to know of savehist-mode fairly late into my Emacs usage
and had no motivation to turn on since most of the time the minibuffer
history is filled with garbage that can be discarded without causing
pain.  If savehist-mode were to be turned on by default, I would
promptly turn it off in my config and call it a day.

I think I would use the mode if it had a whitelist of histories to track
rather than a blacklist, and/or if the history influenced the completion
sorting a la prescient.el.

RE: [External] : Turning on savehist-mode by default

[Drew Adams]

> > FWIW, `savehist.el' is a general way to save variable
> > values, not just a way to save values of minibuffer
> > history variables.  It's often recommended (including
> > by me) as a way to persist any variables you like.
> 
> Please don't forget there is also `desktop.el'
> that is a superset of `savehist.el'

Oh, I don't forget.  That has advantages and
disadvantages compared to savehist.  You can
have any number of desktop files, which can
be in different directories (+).  But it
(always?) saves a lot more than just vars
(-, if you just want to save vars).

> > Why isn't it written with that foremost in mind, i.e.,
> > as a general variable-persisting feature?
> 
> A general variable-persisting feature already exists:
> `define-multisession-variable'.

Looks like that Emacs 29+ feature (like
desktop but in other ways) is also more
than, or different from, just saving vars.

You need to define those vars - and even
access them - in special ways - they're
not ordinary vars.  And it looks like (?)
each such var is stored in its own file
(or in a DB).

I looked at it superficially, however.

It really looks like overkill for just
saving variables.  The use cases we're
talking about, I think, include saving
ordinary variables, of any kind, anytime.

When I mentioned having a simple feature
for just persisting/restoring variables
I meant a simple feature - something like
what savehist does, without separating
out `savehist-additional-variables'.
___

But the existence of multiple ways of
persisting variables suggests that maybe
we should have a doc section about the
topic in the Elisp manual.

That could include simple, ad hoc, saving:
write one or more vars to a file (e.g.
`with-temp-file'); std ways to save a set
of vars (e.g. savehist), std way to define,
save, access `multisession' vars; std ways
to save vars along with other stuff (e.g.
desktop).
___

There are also bookmarks (in Bookmark+, at
least) that just save sets of vars.  You
can have multiple bookmark files, anywhere,
and they can, but need not, be limited to
variable-set bookmarks.  Just jump to a
bookmark to restore a set of vars.

(It's also Bookmark+ that lets you create
desktop files anywhere, including multiple
ones in the same directory.  And lets you
bookmark desktops - just jump to restore.)

RE: [External] : Turning on savehist-mode by default

[Drew Adams]

> > Please don't forget there is also `desktop.el'
> > that is a superset of `savehist.el'
> 
> Is it though? I’m using ‘desktop-save-mode’, not observing
> preservation of histories, and not seeing any user option in the
> Desktop group that would shout to me “enable this to save all
> histories”. I see ‘file-name-history’ in ‘desktop-globals-to-save’ but
> I wouldn’t like to whitelist all the histories I care about.

The two are ... different.  Neither
is a subset of the other.

And even to the extent that part of
one is included in the other, it's
not possible/simple (is it?) to use
the other to get only that subset
behavior.  E.g., you can't (?) use
desktop to only save vars.

Re: Turning on savehist-mode by default

[sbaugh]

Visuwesh <visuweshm@gmail.com> writes:
> I think I would use the mode if it had a whitelist of histories to track
> rather than a blacklist, and/or if the history influenced the completion
> sorting a la prescient.el.

I recently posted a patch which adds an option to sort completion
candidates by the order they appear in the history, in fact, so that
might be available soon (once I rework it for some feedback).  Probably
can't turn that on by default, though.

Re: Turning on savehist-mode by default

[Po Lu]

Spencer Baugh <sbaugh@catern.com> writes:

> I agree that outdated completion history is a nuisance.  However,
> isn't outdated completion history better than none?

Not as I see it.  When M-p is typed and Emacs cannot supply a proper
history entry, it should ding rather than begin iterating over hundreds
of outdated entries.

> I definitely find the bash history experience extremely annoying.  It
> happens even when bash is exited cleanly - it's just a consequence of
> the fact that bash overwrites the previous history when exiting,
> rather than combining them in some way.
>
> Emacs isn't commonly run in parallel, and when it is, at least for me
> there's always one main Emacs instance which outlives the others.  So
> we shouldn't have the bash experience.  But still: I'd be happy to
> implement some history merging behavior so we don't get the bash
> history experience, if that's a requirement for savehist-mode to be
> turned on by default.

Yes.  Despite this not being completely orthodox practice, we know many
users follow it; search for threads on major Emacs forums concerning
startup time, and you will recognize the pervasiveness of short,
ephemeral Emacs sessions.

Re: Turning on savehist-mode by default

[Po Lu]

Dmitry Gutov <dmitry@gutov.dev> writes:

> Yes. But when you play stupid games (e.g. pkill), you win stupid
> prizes.

There are't such "prizes" in the default installation now, so we should
strive not to introduce any.

Re: Turning on savehist-mode by default

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > savehist-mode is a useful mode which is turned on by many Emacs
  > users. [1] It matches the default behavior of programs like bash and
  > vim, which save command history by default.  I suggest that we should
  > find some way to enable savehist by default.

I suppose this functionality would not bother anyone, but would it
cause annoying pauses sometimes?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Turning on savehist-mode by default

[sbaugh]

Po Lu <luangruo@yahoo.com> writes:
> Spencer Baugh <sbaugh@catern.com> writes:
>
>> I agree that outdated completion history is a nuisance.  However,
>> isn't outdated completion history better than none?
>
> Not as I see it.  When M-p is typed and Emacs cannot supply a proper
> history entry, it should ding rather than begin iterating over hundreds
> of outdated entries.

So do you yourself not use savehist-mode?  For these reasons?  Can you
describe your usage pattern that makes savehist-mode not work for you?

Re: Turning on savehist-mode by default

[Spencer Baugh]

Richard Stallman <rms@gnu.org> writes:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > savehist-mode is a useful mode which is turned on by many Emacs
>   > users. [1] It matches the default behavior of programs like bash and
>   > vim, which save command history by default.  I suggest that we should
>   > find some way to enable savehist by default.
>
> I suppose this functionality would not bother anyone, but would it
> cause annoying pauses sometimes?

Probably not, but to be extra sure, I suggest only turning on the
kill-emacs-hook part of savehist-mode by default, not the timer.

Re: Turning on savehist-mode by default

[Po Lu]

sbaugh@catern.com writes:

> So do you yourself not use savehist-mode?  For these reasons?  Can you
> describe your usage pattern that makes savehist-mode not work for you?

I thought I already related why: it doesn't accommodate running multiple
Emacs sessions at once or Emacs processes being abruptly killed.  If it
is to be enabled by default, both these deficiencies must be addressed.

Re: [OT] Not clobbering bash history

[brickviking]

[-- Attachment #1: Type: text/plain, Size: 1900 bytes --]

On Mon, 20 Nov 2023 at 04:18, Spencer Baugh <sbaugh@catern.com> wrote:

> Po Lu <luangruo@yahoo.com> writes:
> > anyone who has run multiple bash shells in parallel,
> > and thus played the guessing game of "which shell's history has been
> > preserved?" will concur.
>
> I definitely find the bash history experience extremely annoying.  It
> happens even when bash is exited cleanly - it's just a consequence of
> the fact that bash overwrites the previous history when exiting, rather
> than combining them in some way.
>
>
Just as a complete aside for preventing bash history erasure, I got around
it like this:

* shopt -s histappend # append to the history file, don't overwrite it
* HISTIGNORE='cd "`printf "%b"*' # Need this so tab filename completion
lines don't get stored
* export HISTCONTROL=ignoreboth # I could also have used erasedups here but
felt this would be unacceptably slow
* export HISTSIZE=30000000 # Yes, it's huge, but I get history going back
years

I prevent history erasure by the simple process of defining history to be
huge (30Mb) and save each line as I hit enter without trimming the
.bash_history shorter. The one disadvantage I've struck so far is that I
have very long bash startup times. I get around that to a degree by
periodically trimming history down to a more sane 10,000 lines after
archiving to a master file. I also load 1k lines of history each time I
start up a shell, hence the longer startup times.

Relating to history searches, I tend to use grep "my pattern"
~/.bash_history first, and if I don't find what I want there (a really
unusual occurrence now) I grep the master file as well. I have history
going back several years, and that has saved my time more times than I can
remember.

I hope that's a useful diversion from the intricacies of emacs history
saving, and might give a tip or two to those bash users among us.

Regards, brickviking

[-- Attachment #2: Type: text/html, Size: 2397 bytes --]

Re: [External] : Turning on savehist-mode by default

[Manuel Giraud via Emacs development discussions.]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Yuri Khan <yuri.v.khan@gmail.com>
>> Date: Sun, 19 Nov 2023 14:32:57 +0700
>> Cc: Drew Adams <drew.adams@oracle.com>, Philip Kaludercic <philipk@posteo.net>,
>>  "sbaugh@catern.com" <sbaugh@catern.com>,
>>  "emacs-devel@gnu.org" <emacs-devel@gnu.org>
>> 
>> On Sun, 19 Nov 2023 at 14:11, Juri Linkov <juri@linkov.net> wrote:
>> 
>> > Please don't forget there is also `desktop.el'
>> > that is a superset of `savehist.el'
>> 
>> Is it though? I’m using ‘desktop-save-mode’, not observing
>> preservation of histories, and not seeing any user option in the
>> Desktop group that would shout to me “enable this to save all
>> histories”. I see ‘file-name-history’ in ‘desktop-globals-to-save’ but
>> I wouldn’t like to whitelist all the histories I care about.
>
> Why not?  I do precisely that, FWIW.  It's a one-time configuration,
> so not a lot of effort.

I'm curious.  What desktop save option did you set for this?  I am
currently using desktop-save *and* savehist but I could use just
desktop-save?
-- 
Manuel Giraud

Re: [External] : Turning on savehist-mode by default

[Eli Zaretskii]

> From: Manuel Giraud <manuel@ledu-giraud.fr>
> Cc: Yuri Khan <yuri.v.khan@gmail.com>,  juri@linkov.net,
>   drew.adams@oracle.com,  philipk@posteo.net,  sbaugh@catern.com,
>   emacs-devel@gnu.org
> Date: Mon, 20 Nov 2023 10:53:46 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Is it though? I’m using ‘desktop-save-mode’, not observing
> >> preservation of histories, and not seeing any user option in the
> >> Desktop group that would shout to me “enable this to save all
> >> histories”. I see ‘file-name-history’ in ‘desktop-globals-to-save’ but
> >> I wouldn’t like to whitelist all the histories I care about.
> >
> > Why not?  I do precisely that, FWIW.  It's a one-time configuration,
> > so not a lot of effort.
> 
> I'm curious.  What desktop save option did you set for this?

I customize desktop-globals-to-save.  It has file-name-history by
default, and I add other history variables to it.

> I am currently using desktop-save *and* savehist but I could use
> just desktop-save?

I didn't study savehist too deeply, but it sounds like these
functionalities are equivalent, yes.

Re: [External] : Turning on savehist-mode by default

[Manuel Giraud via Emacs development discussions.]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Manuel Giraud <manuel@ledu-giraud.fr>
>> Cc: Yuri Khan <yuri.v.khan@gmail.com>,  juri@linkov.net,
>>   drew.adams@oracle.com,  philipk@posteo.net,  sbaugh@catern.com,
>>   emacs-devel@gnu.org
>> Date: Mon, 20 Nov 2023 10:53:46 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Is it though? I’m using ‘desktop-save-mode’, not observing
>> >> preservation of histories, and not seeing any user option in the
>> >> Desktop group that would shout to me “enable this to save all
>> >> histories”. I see ‘file-name-history’ in ‘desktop-globals-to-save’ but
>> >> I wouldn’t like to whitelist all the histories I care about.
>> >
>> > Why not?  I do precisely that, FWIW.  It's a one-time configuration,
>> > so not a lot of effort.
>> 
>> I'm curious.  What desktop save option did you set for this?
>
> I customize desktop-globals-to-save.  It has file-name-history by
> default, and I add other history variables to it.

Thanks, I'll try to do that.  BTW, maybe this requires a paragraph in
info.  The "(emacs) Saving Emacs Sessions" node, only refers to the
savehist library.

>> I am currently using desktop-save *and* savehist but I could use
>> just desktop-save?
>
> I didn't study savehist too deeply, but it sounds like these
> functionalities are equivalent, yes.

Looking at my ".emacs.d/history" file, it seems that savehist comes with
lots of history saved by default: erc, eww, gnus-search, query-replace…
just to name a few.
-- 
Manuel Giraud

Re: [External] : Turning on savehist-mode by default

[Eli Zaretskii]

> From: Manuel Giraud <manuel@ledu-giraud.fr>
> Cc: yuri.v.khan@gmail.com,  juri@linkov.net,  drew.adams@oracle.com,
>   philipk@posteo.net,  sbaugh@catern.com,  emacs-devel@gnu.org
> Date: Mon, 20 Nov 2023 14:15:24 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I customize desktop-globals-to-save.  It has file-name-history by
> > default, and I add other history variables to it.
> 
> Thanks, I'll try to do that.  BTW, maybe this requires a paragraph in
> info.  The "(emacs) Saving Emacs Sessions" node, only refers to the
> savehist library.

Maybe.  Let me think about it.

> >> I am currently using desktop-save *and* savehist but I could use
> >> just desktop-save?
> >
> > I didn't study savehist too deeply, but it sounds like these
> > functionalities are equivalent, yes.
> 
> Looking at my ".emacs.d/history" file, it seems that savehist comes with
> lots of history saved by default: erc, eww, gnus-search, query-replace…
> just to name a few.

If you really want all or most of the history variables, then savehist
is easier and more convenient, since you don't have to name every
variable you want to save.  Me, I want to save only a small number of
them, the rest are useless for me.

Re: Turning on savehist-mode by default

[Spencer Baugh]

Po Lu <luangruo@yahoo.com> writes:
> sbaugh@catern.com writes:
>
>> So do you yourself not use savehist-mode?  For these reasons?  Can you
>> describe your usage pattern that makes savehist-mode not work for you?
>
> I thought I already related why: it doesn't accommodate running multiple
> Emacs sessions at once or Emacs processes being abruptly killed.  If it
> is to be enabled by default, both these deficiencies must be addressed.

Yes, I understand this point.

What I mean is: Can you describe, concretely, when you personally are
running multiple Emacs sessions at once, or abruptly killing an Emacs
process?

I ask just to make sure I have at least one concrete example of the kind
of usage should be supported.  Otherwise we might come up with a
solution that doesn't actually work for you.

Plus, since such usage seems to be considered uncommon, a concrete
example will further support your point.

RE: [External] : Turning on savehist-mode by default

[Drew Adams]

> I customize desktop-globals-to-save.  It has file-name-history by
> default, and I add other history variables to it.

FWIW, that option shouldn't be named `...globals...'.
It should be named something like `...variables...'
or `...vars...'.

Re: Not clobbering bash history

[Juri Linkov]

> Just as a complete aside for preventing bash history erasure, I got around
> it like this:
>
> * export HISTCONTROL=ignoreboth # I could also have used erasedups here but
> felt this would be unacceptably slow

Actually this is not quite off-topic.  In Emacs, the corresponding option
is 'history-delete-duplicates' that corresponds to 'ignoredups'.
But unfortunately still no option corresponding to 'ignorespace'.

Instead of using such advice

  (define-advice add-to-history (:around (ofun history-var newelt &rest args) ignorespace)
    (unless (string-match-p "\\` " newelt)
      (apply ofun history-var newelt args)))

maybe 'history-delete-duplicates' should support a new value 'ignorespace'?

> * export HISTSIZE=30000000
> # Yes, it's huge, but I get history going back years

I found that it helps to reduce the size of the shell history
by splitting the history file by different file trees using advice:

  (define-advice shell (:around (ofun &rest args) per-tree-history)
    (let ((process-environment (copy-sequence process-environment))
          (root (locate-dominating-file default-directory ".bash_history")))
      (when root
        (setenv "HISTFILE" (expand-file-name ".bash_history" root)))
      (apply ofun args)))

but no idea how this could be more customizable.

Re: [External] : Turning on savehist-mode by default

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "yuri.v.khan@gmail.com" <yuri.v.khan@gmail.com>,
>         "juri@linkov.net"
> 	<juri@linkov.net>,
>         "philipk@posteo.net" <philipk@posteo.net>,
>         "sbaugh@catern.com" <sbaugh@catern.com>,
>         "emacs-devel@gnu.org"
> 	<emacs-devel@gnu.org>
> Date: Mon, 20 Nov 2023 16:57:16 +0000
> 
> > I customize desktop-globals-to-save.  It has file-name-history by
> > default, and I add other history variables to it.
> 
> FWIW, that option shouldn't be named `...globals...'.
> It should be named something like `...variables...'
> or `...vars...'.

There's also desktop-locals-to-save.

RE: [External] : Turning on savehist-mode by default

[Drew Adams]

> > > ... desktop-globals-to-save ...
> >
> > FWIW, that option shouldn't be named `...globals...'.
> > It should be named something like `...variables...'
> > or `...vars...'.
> 
> There's also desktop-locals-to-save.

"globals" and "locals" isn't clear, IMHO, and
isn't very discoverable (e.g. with apropos).

They should be named something like
`...global-vars...' and `...buffer-local-vars...'.

"Local" variables as opposed to global doesn't
necessarily (or even normally) mean _buffer_
local.  (But saving other kinds of local vars
doesn't make sense, admittedly.)

More generally, anything having to do with
"buffer local" should include "buffer" in the
name, especially if the context doesn't make
clear that this is what's meant.

Same for directory-local, file-local, and
connection-local vars.

Note that at least the options etc. for those
use the word "variables" in the name.  We don't
use a name such as just `ignored-locals'.

It is, however, true that we've omitted `file'
from the names of several file-local names,
such as `ignored-local-variables'.  It would
be better for those too to include `file' in
the name. 

Re: [External] : Turning on savehist-mode by default

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "manuel@ledu-giraud.fr" <manuel@ledu-giraud.fr>,
>         "yuri.v.khan@gmail.com"
> 	<yuri.v.khan@gmail.com>,
>         "juri@linkov.net" <juri@linkov.net>,
>         "philipk@posteo.net" <philipk@posteo.net>,
>         "sbaugh@catern.com"
> 	<sbaugh@catern.com>,
>         "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Mon, 20 Nov 2023 18:54:47 +0000
> 
> > > > ... desktop-globals-to-save ...
> > >
> > > FWIW, that option shouldn't be named `...globals...'.
> > > It should be named something like `...variables...'
> > > or `...vars...'.
> > 
> > There's also desktop-locals-to-save.
> 
> "globals" and "locals" isn't clear, IMHO,

They are crystal clear IMO.

> and isn't very discoverable (e.g. with apropos).

I don't see why they should be discoverable via apropos.  They are
minor features of desktop.el, and therefore should be discovered
either via customize-group or by simply searching desktop.el for
"defcustom".

Re: [OT] Not clobbering bash history

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I definitely find the bash history experience extremely annoying.  It
  > > happens even when bash is exited cleanly - it's just a consequence of
  > > the fact that bash overwrites the previous history when exiting, rather
  > > than combining them in some way.

Should we suggest that the Bash developer add a feature to handle this
case (multiple shells in parallel) the "right" way?  If many users
would like it, that could make it worth building in.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [OT] Not clobbering bash history

[Arsen Arsenović]

[-- Attachment #1: Type: text/plain, Size: 911 bytes --]


Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > I definitely find the bash history experience extremely annoying.  It
>   > > happens even when bash is exited cleanly - it's just a consequence of
>   > > the fact that bash overwrites the previous history when exiting, rather
>   > > than combining them in some way.
>
> Should we suggest that the Bash developer add a feature to handle this
> case (multiple shells in parallel) the "right" way?  If many users
> would like it, that could make it worth building in.

That would be very nice.  This issue has almost been prolific enough to
force me to switch shells.

Thanks, have a lovely night.
-- 
Arsen Arsenović

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 251 bytes --]

Re: [External] : Turning on savehist-mode by default

[Eli Zaretskii]

> Date: Mon, 20 Nov 2023 16:05:09 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: yuri.v.khan@gmail.com, juri@linkov.net, drew.adams@oracle.com,
>  philipk@posteo.net, sbaugh@catern.com, emacs-devel@gnu.org
> 
> > From: Manuel Giraud <manuel@ledu-giraud.fr>
> > Cc: yuri.v.khan@gmail.com,  juri@linkov.net,  drew.adams@oracle.com,
> >   philipk@posteo.net,  sbaugh@catern.com,  emacs-devel@gnu.org
> > Date: Mon, 20 Nov 2023 14:15:24 +0100
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > > I customize desktop-globals-to-save.  It has file-name-history by
> > > default, and I add other history variables to it.
> > 
> > Thanks, I'll try to do that.  BTW, maybe this requires a paragraph in
> > info.  The "(emacs) Saving Emacs Sessions" node, only refers to the
> > savehist library.
> 
> Maybe.  Let me think about it.

Now done on the emacs-29 branch.

Re: [External] : Turning on savehist-mode by default

[Manuel Giraud via Emacs development discussions.]

Eli Zaretskii <eliz@gnu.org> writes:

[...]

>> > Thanks, I'll try to do that.  BTW, maybe this requires a paragraph in
>> > info.  The "(emacs) Saving Emacs Sessions" node, only refers to the
>> > savehist library.
>> 
>> Maybe.  Let me think about it.
>
> Now done on the emacs-29 branch.

Thanks!
-- 
Manuel Giraud

Re: [OT] Not clobbering bash history

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Should we suggest that the Bash developer add a feature to handle this
  > > case (multiple shells in parallel) the "right" way?  If many users
  > > would like it, that could make it worth building in.

  > That would be very nice.  This issue has almost been prolific enough to
  > force me to switch shells.

How about if you (or someone) write a description of the behavior you
would like Bash to offer (at least as an option)?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [OT] Not clobbering bash history

[Arsen Arsenović]

[-- Attachment #1: Type: text/plain, Size: 1679 bytes --]


Richard Stallman <rms@gnu.org> writes:

>   > > Should we suggest that the Bash developer add a feature to handle this
>   > > case (multiple shells in parallel) the "right" way?  If many users
>   > > would like it, that could make it worth building in.
>
>   > That would be very nice.  This issue has almost been prolific enough to
>   > force me to switch shells.
>
> How about if you (or someone) write a description of the behavior you
> would like Bash to offer (at least as an option)?

Sure.

libhistory should, on recall attempt, try to re-read the history file
from the last point it was on in order to catch new histories, it should
append to the file, and it should attempt to lock the file via flock or
similar if such facilities are available (just in case).

Bash already implements this partially, lacking the update-on-recall
functionality (I believe ZSH calls its version of this "share_history"),
and lacking a way to append a history command *before* it is executed,
to my awareness.

In the event of truncation, libhistory needs to be careful not to lose
any histories that were to be submitted in between the moment of
determination of truncation and commitment of the truncation to disk.

On occasion, I wish histories were stored in a SQLite3 database :-)

As a QoL feature, bash should prevent history truncation if ran with
--norc or other flags that would inhibit HISTFILE being set potentially.
I suspect that this latter change alone would prevent the issue I was
seeing (but I can't be sure, since I never caught bash in the act of
history file truncation).

Thanks, have a lovely day.
-- 
Arsen Arsenović

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 381 bytes --]

Re: Turning on savehist-mode by default

[Thanos Apollo]

[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]


sbaugh@catern.com writes:

> savehist-mode is a useful mode which is turned on by many Emacs
> users. [1] It matches the default behavior of programs like bash and
> vim, which save command history by default.  I suggest that we should
> find some way to enable savehist by default.
>

What is the recommended way for package authors to disable
savehist-mode?

If savehist-mode is enabled by default, it is advisable to also consider
a simple way to disable it for a specific package mode.

for example I'm using emacs to answer a MCQ, such as:

Q:Which one of the following statements concerning the
ability of acidosis to precipitate a crisis in sickle cell
anemia is correct?

```emacs-lisp
(completing-read "Answer: "
'("Fetal blood has a higher affinity for oxygen than does
adult blood because Hb F has a decreased affinity for
2,3-BPG."
"Purified Hb F (stripped of 2,3-BPG) has a higher affin-
ity for oxygen than does purified Hb A."
"The globin chain composition of Hb F is α2δ2."))
```
Saving the latest and usually correct answer defeats the purpose of an MCQ.

One workaround for this is to set minibuffer-history to nil, although it
is unclear whether this is advised, even with a let statement.

-- 
Thanos Apollo
https://thanosapollo.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 865 bytes --]

Re: Turning on savehist-mode by default

[Thanos Apollo]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]


Thanos Apollo <public@thanosapollo.org> writes:

> [[PGP Signed Part:Undecided]]
>
> sbaugh@catern.com writes:
>
>> savehist-mode is a useful mode which is turned on by many Emacs
>> users. [1] It matches the default behavior of programs like bash and
>> vim, which save command history by default.  I suggest that we should
>> find some way to enable savehist by default.
>>
>
> What is the recommended way for package authors to disable
> savehist-mode?

Ignore this email, my question had nothing to do with savehist.

-- 
Thanos Apollo
https://thanosapollo.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 865 bytes --]

Re: Turning on savehist-mode by default

[Eli Zaretskii]

> From: Thanos Apollo <public@thanosapollo.org>
> Date: Tue, 28 Nov 2023 13:04:41 +0200
> 
> > savehist-mode is a useful mode which is turned on by many Emacs
> > users. [1] It matches the default behavior of programs like bash and
> > vim, which save command history by default.  I suggest that we should
> > find some way to enable savehist by default.
> >
> 
> What is the recommended way for package authors to disable
> savehist-mode?
> 
> If savehist-mode is enabled by default, it is advisable to also consider
> a simple way to disable it for a specific package mode.
> 
> for example I'm using emacs to answer a MCQ, such as:
> 
> Q:Which one of the following statements concerning the
> ability of acidosis to precipitate a crisis in sickle cell
> anemia is correct?
> 
> ```emacs-lisp
> (completing-read "Answer: "
> '("Fetal blood has a higher affinity for oxygen than does
> adult blood because Hb F has a decreased affinity for
> 2,3-BPG."
> "Purified Hb F (stripped of 2,3-BPG) has a higher affin-
> ity for oxygen than does purified Hb A."
> "The globin chain composition of Hb F is α2δ2."))
> ```
> Saving the latest and usually correct answer defeats the purpose of an MCQ.
> 
> One workaround for this is to set minibuffer-history to nil, although it
> is unclear whether this is advised, even with a let statement.

You could instead bind history-add-new-input to nil around calls to
completing-read.

Re: Turning on savehist-mode by default

[Adam Porter]

FWIW, IME, it may not be a good idea to enable savehist-mode by default. 
  As an example, see this issue report on Ement.el's repository:

https://github.com/alphapapa/ement.el/issues/216

The gist is that, for users who enabled savehist, it was saving the 
variable command-history, which included some complex commands whose 
arguments (from interactive completion) included large data structures 
(which are hidden from the user when called interactively).  This 
resulted in savehist files hundreds of MB in size and very noticeable 
pauses whenever savehist saved its data, from timers and upon Emacs exit.

The worst part was how long it took to discover the source of the 
problem.  There were intermittent reports of weird pauses for months, 
without any discernible reason, until a user finally narrowed it down to 
using savehist.  Then we had to add this hacky workaround in Ement to 
try to automatically avoid the problem.

Such cases may be rare (or not--I can't say), but when it happens, it 
seems to be a bear to deal with, being very unintuitive.  For myself, 
even having written a number of Emacs packages, I wasn't yet aware of 
the command-history variable and how it saves arguments, and certainly 
not the implications of using savehist as well.

--Adam

Re: Turning on savehist-mode by default

[Dmitry Gutov]

On 28/11/2023 23:07, Adam Porter wrote:
> FWIW, IME, it may not be a good idea to enable savehist-mode by default. 
>   As an example, see this issue report on Ement.el's repository:
> 
> https://github.com/alphapapa/ement.el/issues/216
> 
> The gist is that, for users who enabled savehist, it was saving the 
> variable command-history, which included some complex commands whose 
> arguments (from interactive completion) included large data structures 
> (which are hidden from the user when called interactively).  This 
> resulted in savehist files hundreds of MB in size and very noticeable 
> pauses whenever savehist saved its data, from timers and upon Emacs exit.

savehist-mode is very popular, so even if it's not on by default, such 
issues might be encountered by a lot of people anyway.

Could we try to fix it for those users? Either by filtering out 
"expensive" entries before saving (with some heuristic), or by entirely 
blacklisting command-history by default -- through 
savehist-ignored-variables.

Re: Turning on savehist-mode by default

[Eli Zaretskii]

> Date: Tue, 28 Nov 2023 15:07:46 -0600
> Cc: emacs-devel@gnu.org, public@thanosapollo.org
> From: Adam Porter <adam@alphapapa.net>
> 
> FWIW, IME, it may not be a good idea to enable savehist-mode by default. 
>   As an example, see this issue report on Ement.el's repository:
> 
> https://github.com/alphapapa/ement.el/issues/216
> 
> The gist is that, for users who enabled savehist, it was saving the 
> variable command-history, which included some complex commands whose 
> arguments (from interactive completion) included large data structures 
> (which are hidden from the user when called interactively).  This 
> resulted in savehist files hundreds of MB in size and very noticeable 
> pauses whenever savehist saved its data, from timers and upon Emacs exit.
> 
> The worst part was how long it took to discover the source of the 
> problem.  There were intermittent reports of weird pauses for months, 
> without any discernible reason, until a user finally narrowed it down to 
> using savehist.  Then we had to add this hacky workaround in Ement to 
> try to automatically avoid the problem.
> 
> Such cases may be rare (or not--I can't say), but when it happens, it 
> seems to be a bear to deal with, being very unintuitive.  For myself, 
> even having written a number of Emacs packages, I wasn't yet aware of 
> the command-history variable and how it saves arguments, and certainly 
> not the implications of using savehist as well.

savehist provides a customizable ability to limit the number of
history entries saved for each variable.

We could also add a new feature which limits the size of individual
entry: entries longer than that will not be saved.

Patches welcome.

Re: Turning on savehist-mode by default

[Björn Bidar]

Would undelete-frame-mode be a similar candidate for such a change?
I think so since it would improve the out of the box experience for most
users without bothering anyone else since the saved information doesn't
take up much memory and new users would have less issues when closing a
frame by accident.

Re: [OT] Not clobbering bash history

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I am trying to understand the words you wrote.

  > libhistory should, on recall attempt,

What does "recall attempt" mean?  I don't know that term.

                                          try to re-read the history file
  > from the last point it was on

What does "last point it was on" mean?

                                  in order to catch new histories, it should
  > append to the file,

What should it append, and in what circumstances?

                        and it should attempt to lock the file via flock or
  > similar if such facilities are available (just in case).

If two different shells will try to write history into one single file,
are they doomed to give bad results, one way or another>  It seems to me
that the crucial thing is for them to use two different files.

  > In the event of truncation, 

What truncates the file?  When does that occur?

Is such truncation _supposed_ to occur, or is it a bug that it occurs?

                                libhistory needs to be careful not to lose
  > any histories that were to be submitted in between the moment of
  > determination of truncation and commitment of the truncation to disk.

I don't follow.

  > As a QoL feature, bash should prevent history truncation if ran with
  > --norc or other flags that would inhibit HISTFILE being set potentially.

I don't understand this point -- can you explain?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [OT] Not clobbering bash history

[Arsen Arsenović]

[-- Attachment #1: Type: text/plain, Size: 3840 bytes --]

Hi,

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> I am trying to understand the words you wrote.
>
>   > libhistory should, on recall attempt,
>
> What does "recall attempt" mean?  I don't know that term.

I mean the action usually bound to C-p (I believe the libhistory
function is previous_history).

>                                           try to re-read the history file
>   > from the last point it was on
>
> What does "last point it was on" mean?

Say that the history has eight entries, and a ninth was added by a
parallel process, the current process would be at the point after the
eighth element and should continue reading from there next time it wants
a history entry.

>                                   in order to catch new histories, it should
>   > append to the file,
>
> What should it append, and in what circumstances?

New history entries, always (i.e. it should only ever append, as if via
O_APPEND, to ~/.bash_history and friends, at least while not holding
locks).

>                         and it should attempt to lock the file via flock or
>   > similar if such facilities are available (just in case).
>
> If two different shells will try to write history into one single file,
> are they doomed to give bad results, one way or another>  It seems to me
> that the crucial thing is for them to use two different files.

Not necessarily.  If both shells use a single write() syscall on an
O_APPEND file, they should work as expected to my awareness.  But, if
not, then locks would help (flock-ing the file before adding an entry
ensures only one shell is ever writing to enable correct history
storage).

>   > In the event of truncation, 
>
> What truncates the file?  When does that occur?

bash/libhistory would need to if it'd like to rotate the history file
due to HISTSIZE.

> Is such truncation _supposed_ to occur, or is it a bug that it occurs?

Depends.  If the user sets HISTSIZE to, say, 500, and enters a new
command, and it'd have been the 501st, the history file would need to be
truncated to be rotated.  In that scenario, it is intended.  If the user
did not intend to have HISTSIZE set to a positive value, then any
truncation is a bug (as history would be lost).

>                                 libhistory needs to be careful not to lose
>   > any histories that were to be submitted in between the moment of
>   > determination of truncation and commitment of the truncation to disk.
>
> I don't follow.

If a bash process decides to rotate the history file as a result of
HISTSIZE, and another bash process decides to do the same, one of their
new history entries would be lost due to the other one overriding it.
This would be a bug.

Hence, libhistory needs to be careful to prevent this.

>   > As a QoL feature, bash should prevent history truncation if ran with
>   > --norc or other flags that would inhibit HISTFILE being set potentially.
>
> I don't understand this point -- can you explain?

I meant to say HISTSIZE here.  My apologies.

If HISTSIZE is set in my RC file to a negative value (per (bash) Bash
Variables, this should inhibit history file rotation and/or truncation),
but if a shell is started with --norc (by, say, TRAMP, perhaps), then
bash would not be aware of the users desired HISTSIZE.  In that event,
it should be conservative and *not* overwrite the users' history.

I suspect that the latter is what was causing sporadic truncation in my
case (but I haven't confirmed so unfortunately).

Thanks, have a lovely day.
-- 
Arsen Arsenović

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 381 bytes --]

Re: [OT] Not clobbering bash history

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Since this is about Bash, ot Emacs, I moved it to emacs-tangents@gnu.org.
-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)