Re: master 927b885 1/3: Disable filtering of commands in M-x completion

Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Stefan Kangas]

eliz@gnu.org (Eli Zaretskii) writes:

> branch: master
> commit 927b88571cebb4f64aca360fbfa5d15a1f922ad6
> Author: Eli Zaretskii <eliz@gnu.org>
> Commit: Eli Zaretskii <eliz@gnu.org>
>
>     Disable filtering of commands in M-x completion
>
>     This makes the default behavior like it was before:
>     M-x completion doesn't filter out any commands.  To
>     have commands filtered based on their relevance to the
>     current buffer's modes, customize the option
>     'read-extended-command-predicate' to call
>     'command-completion-default-include-p'.

I can understand this decision for Emacs 28, but hope that we can
consider enabling it by default in Emacs 29.  (I would have preferred it
as the default in Emacs 28, but I can see the benefit in having a period
to let the feature stabilize and mature a bit first.)

I understand that sometimes a package developer will get filtering wrong
(i.e. a bug), and that at other times the user will know best if a
command is relevant or not.  I would expect this to overwhelmingly not
be the most common case.  But even if I am right about that, it is still
important that we get this situation right.

I think we should take one of the below actions:

a) Add a key to show the unfiltered list of matches in `M-x'.
   (We could use any key, but how about just using `M-x M-x' to show the
   unfiltered list?  It would affect recursive minibuffers, but we could
   just require a third `M-x' for that.)

b) Add a new command, e.g. on `C-x x x', that always acts like the old
   `M-x'.

c) Both.

And then we should consider making this feature the default, preferably
already in Emacs 28 but otherwise in Emacs 29.

Here is why:

I have never understood why Emacs suggests commands for execution in a
context where they will obviously fail.  I think this is a glaring
deficiency in our default UI -- you are proposed useless commands (that
won't work there, will screw up your buffer, etc.).  No longer doing
that is in my view a big step forward for Emacs usability.

Having this feature on provides additional safety that is most sorely
needed by "regular" users.  It will also make it easier to learn Emacs,
as there is less intimidating, often useless cruft to filter through.

Finding the correct command, often even a relevant one, was something I
personally struggled a lot with when I started out with Emacs.  My own
conclusion was that `M-x' was often practically useless as a discovery
mechanism.  This of course largely depends on what you are trying to do,
but consider finding a relevant Gnus command using `M-x'... not fun.

Finally, I note that Emacs is *not* more powerful because users can say
`M-x mwheel-scroll' -- it just has a lower signal to noise ratio.[1]

OTOH, it seems to me that the target audience for disabling this feature
is (or will be) mostly "hardcore"/"veteran"/"advanced" users who are
very used to and like the old behavior.

From the discussions we've had so far, it is my understanding that some
like to use it for searching for and discovering commands.  That is fine
and valid, and reason for having an option to opt-out of this behavior.
(I also think we should add a `describe-command' to try to better cover
this use-case.)

Footnotes:
[1] Some modes are prudent in saying:

       (unless (derived-p 'foo-mode) (user-error "Nope"))

     I think up until now this has often been the right and safe thing
     to do, but it has unfortunately been severely underused.  Yet it
     still has the deficiency that the command will show up in 'M-x`,
     and it provides no fire-escape.

Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Juri Linkov]

> I think we should take one of the below actions:
>
> a) Add a key to show the unfiltered list of matches in `M-x'.
>    (We could use any key, but how about just using `M-x M-x' to show the
>    unfiltered list?  It would affect recursive minibuffers, but we could
>    just require a third `M-x' for that.)
>
> b) Add a new command, e.g. on `C-x x x', that always acts like the old
>    `M-x'.
>
> c) Both.

Also maybe a key to toggle filtering in the already active completion
(and a key to toggle sorting).

> I have never understood why Emacs suggests commands for execution in a
> context where they will obviously fail.  I think this is a glaring
> deficiency in our default UI -- you are proposed useless commands (that
> won't work there, will screw up your buffer, etc.).  No longer doing
> that is in my view a big step forward for Emacs usability.

I have no problem with unrelated commands displayed by M-x,
but for the new option I'd like also to suggest to filter out
commands having "r" in their interactive specs - they signal
an error when the region is not active, so no need to show them.
Also filter out commands with "*" in read-only buffers.

> From the discussions we've had so far, it is my understanding that some
> like to use it for searching for and discovering commands.  That is fine
> and valid, and reason for having an option to opt-out of this behavior.
> (I also think we should add a `describe-command' to try to better cover
> this use-case.)

`describe-command' is a very good idea for searching and discovering commands.

> Footnotes:
> [1] Some modes are prudent in saying:
>
>        (unless (derived-p 'foo-mode) (user-error "Nope"))
>
>      I think up until now this has often been the right and safe thing
>      to do, but it has unfortunately been severely underused.  Yet it
>      still has the deficiency that the command will show up in 'M-x`,
>      and it provides no fire-escape.

Isearch mode uses such idiom that when an internal command is called
when Isearch is not active, then it's activated before the command
is executed, with such a guard clause:

  (unless isearch-mode (isearch-mode t))

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> Also maybe a key to toggle filtering in the already active completion
> (and a key to toggle sorting).

For filtering, why toggle instead of cycle, or
completion-choose a sort order?  I.e., why allow
for only two choices?

> I'd like also to suggest to filter out
> commands having "r" in their interactive specs - they signal
> an error when the region is not active, so no need to show them.

Are you sure they raise an error?

(defun foo (&optional beg end)
  (interactive "r")
  (message "FOO"))

I also think it's misleading to base filtering on
an `interactive' form that uses a string.  That's
inherently fragile/limiting.  It won't apply to a
form that evaluates a sexp to provide the args.
The resulting incoherence will confuse users.

> Also filter out commands with "*" in read-only buffers.

Same problem.  Relying on a string arg to `interactive'
is not a great idea, I think.

> `describe-command' is a very good idea for searching
> and discovering commands.

I mentioned that I defined that long ago.  But I'll
also mention that `describe-function' with a prefix
arg already gives you that behavior.  Oops!  Nope,
that too is something I defined long ago in
`help-fns+.el'.

Similarly, I defined `describe-option', and a prefix
arg to (my definition of) `describe-variable' limits
choices to user options (like `describe-option').

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Juri Linkov]

>> Also maybe a key to toggle filtering in the already active completion
>> (and a key to toggle sorting).
>
> For filtering, why toggle instead of cycle, or
> completion-choose a sort order?  I.e., why allow
> for only two choices?

I agree, cycling is better.

>> I'd like also to suggest to filter out
>> commands having "r" in their interactive specs - they signal
>> an error when the region is not active, so no need to show them.
>
> Are you sure they raise an error?
>
> (defun foo (&optional beg end)
>   (interactive "r")
>   (message "FOO"))

Yes, I'm sure.  'M-x foo RET' without an active region raises an error:

  command-execute: The mark is not active now

> I also think it's misleading to base filtering on
> an `interactive' form that uses a string.  That's
> inherently fragile/limiting.  It won't apply to a
> form that evaluates a sexp to provide the args.
> The resulting incoherence will confuse users.

This means that more complex interactive specs need
manual tagging using a new tag:

  (declare (predicate (use-region-p)))

>> Also filter out commands with "*" in read-only buffers.
>
> Same problem.  Relying on a string arg to `interactive'
> is not a great idea, I think.

Same tag:

  (declare (predicate (not buffer-read-only)))

>> `describe-command' is a very good idea for searching
>> and discovering commands.
>
> I mentioned that I defined that long ago.  But I'll
> also mention that `describe-function' with a prefix
> arg already gives you that behavior.  Oops!  Nope,
> that too is something I defined long ago in
> `help-fns+.el'.
>
> Similarly, I defined `describe-option', and a prefix
> arg to (my definition of) `describe-variable' limits
> choices to user options (like `describe-option').

Using a prefix arg for 'C-h f' and 'C-h v' to limit the list
of completions would be nice.

But I guess M-x can't use a prefix arg to limit completions?

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> >> I'd like also to suggest to filter out commands
> >> having "r" in their interactive specs - they signal
> >> an error when the region is not active, so no
> >> need to show them.
> >
> > Are you sure they raise an error?
> >
> > (defun foo (&optional beg end)
> >   (interactive "r")
> >   (message "FOO"))
> 
> Yes, I'm sure.  'M-x foo RET' without an active region
> raises an error:
>   command-execute: The mark is not active now

I don't see that in Emacs 27.1 or prior (with
`emacs -Q').  Is this perhaps new for 28?
If so, why would we do that?

`interactive' code `r' gives you the region
limits, regardless of whether the region is
active - which is TRT.

I don't understand why this should now raise an
error (even if `r' gave you the limits only when
the region were active - which it shouldn't).

And certainly in a case like the example I gave,
where the args are optional.  I don't see the
motivation for now raising an error.

> > I also think it's misleading to base filtering on
> > an `interactive' form that uses a string.  That's
> > inherently fragile/limiting.  It won't apply to a
> > form that evaluates a sexp to provide the args.
> > The resulting incoherence will confuse users.
> 
> This means that more complex interactive specs
> need manual tagging using a new tag:
>   (declare (predicate (use-region-p)))

That's not what `r' has done forever (up through
Emacs 27.1, at least).  It's never been only about
the _active_ region.

Anyway, that's only for `r'.  You can no doubt
propose something for other `interactive' codes.
But even that won't handle arbitrary `interactive'
sexps, right?

`interactive' can accept a string arg, which, in
effect, has its own, parsable language.  So you
can try to parse, and try to do some somewhat
relevant tagging based on that parsing.

But `interactive' is much more general than what's
represented by that string-arg language.  Trying
to base your tagging on `interactive' seems
fundamentally flawed/misguided.

> >> Also filter out commands with "*" in read-only buffers.
> > Same problem.  Relying on a string arg to `interactive'
> > is not a great idea, I think.
> 
> Same tag:
>   (declare (predicate (not buffer-read-only)))

And what about commands that might be usable
interactively, but whose `interactive' spec
doesn't encapsulate all that's involved?  In that
case, manual addition of a declaration will need
to look into the logic of the command body as
well.  (Admittedly, such commands are uncommon.)

> >> `describe-command' is a very good idea for searching
> >> and discovering commands.
> >
> > I mentioned that I defined that long ago.  But I'll
> > also mention that `describe-function' with a prefix
> > arg already gives you that behavior.  Oops!  Nope,
> > that too is something I defined long ago in
> > `help-fns+.el'.
> >
> > Similarly, I defined `describe-option', and a prefix
> > arg to (my definition of) `describe-variable' limits
> > choices to user options (like `describe-option').
> 
> Using a prefix arg for 'C-h f' and 'C-h v' to limit the list
> of completions would be nice.
> 
> But I guess M-x can't use a prefix arg to limit completions?

Why do you think so?  `C-u M-x describe-function TAB'
(with my definition from `help-fns+.el') shows only
commands as candidates.

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Juri Linkov]

>> > (defun foo (&optional beg end)
>> >   (interactive "r")
>> >   (message "FOO"))
>>
>> Yes, I'm sure.  'M-x foo RET' without an active region
>> raises an error:
>>   command-execute: The mark is not active now
>
> I don't see that in Emacs 27.1 or prior (with
> `emacs -Q').  Is this perhaps new for 28?
> If so, why would we do that?

It raises an error because the value of
mark-even-if-inactive was changed to nil,
so the region exists only when explicitly activated.

>> This means that more complex interactive specs
>> need manual tagging using a new tag:
>>   (declare (predicate (use-region-p)))
>
> But even that won't handle arbitrary `interactive'
> sexps, right?

`declare' could use the same logic used in `interactive'
that detects region boundaries.

>> Same tag:
>>   (declare (predicate (not buffer-read-only)))
>
> And what about commands that might be usable
> interactively, but whose `interactive' spec
> doesn't encapsulate all that's involved?  In that
> case, manual addition of a declaration will need
> to look into the logic of the command body as
> well.  (Admittedly, such commands are uncommon.)

Yes, here it should share the same logic as well.

>> Using a prefix arg for 'C-h f' and 'C-h v' to limit the list
>> of completions would be nice.
>>
>> But I guess M-x can't use a prefix arg to limit completions?
>
> Why do you think so?  `C-u M-x describe-function TAB'
> (with my definition from `help-fns+.el') shows only
> commands as candidates.

The docstring of `execute-extended-command' says:

  To pass a prefix argument to the command you are invoking,
  give a prefix argument to ‘execute-extended-command’.

And indeed `C-u M-x forward-char RET' moves 4 chars forwards.
So you can't use `C-u M-x TAB' to limit the number of completions.
Maybe then use another prefix like `C-x', i.e. `C-x M-x'?
This is similar to how `C-x M-:' combines the prefix `C-x'
to run `repeat-complex-command'.

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> >> > (defun foo (&optional beg end)
> >> >   (interactive "r")
> >> >   (message "FOO"))
> >>
> >> Yes, I'm sure.  'M-x foo RET' without an active
> >> region raises an error:
> >>   command-execute: The mark is not active now
> >
> > I don't see that in Emacs 27.1 or prior (with
> > `emacs -Q').  Is this perhaps new for 28?
> > If so, why would we do that?
> 
> It raises an error because the value of
> mark-even-if-inactive was changed to nil,
> so the region exists only when explicitly activated.

Really?  That's a horrible change in default behavior
(IMHO).  Why was that done?

And that simple `foo' definition is enough to point
to regressive behavior.  That command should just
work - across all Emacs versions.  `M-x foo' should
not raise an error suddenly in Emacs 28.  (IMHO.)

This backward-incompatible change will mean that
code such as that for the simple `foo' command will
need to bind that user option to non-nil (and binding
a user option is supposedly a no-no for vanilla Emacs).

> >> This means that more complex interactive specs
> >> need manual tagging using a new tag:
> >>   (declare (predicate (use-region-p)))
> >
> > But even that won't handle arbitrary `interactive'
> > sexps, right?
> 
> `declare' could use the same logic used in 
> `interactive' that detects region boundaries.

Nearly anything is possible, of course.  Once you
head down a rabbit hole like this you end up
fiddling to change/fix/adapt stuff left & right.

This ought to be a sign that we're headed in the
wrong direction.  (Yet another sign.)

> >> Same tag:
> >>   (declare (predicate (not buffer-read-only)))
> >
> > And what about commands that might be usable
> > interactively, but whose `interactive' spec
> > doesn't encapsulate all that's involved?  In that
> > case, manual addition of a declaration will need
> > to look into the logic of the command body as
> > well.  (Admittedly, such commands are uncommon.)
> 
> Yes, here it should share the same logic as well.

(Yet another sign.  Left & right...)

> >> Using a prefix arg for 'C-h f' and 'C-h v' to limit the list
> >> of completions would be nice.
> >>
> >> But I guess M-x can't use a prefix arg to limit completions?
> >
> > Why do you think so?  `C-u M-x describe-function TAB'
> > (with my definition from `help-fns+.el') shows only
> > commands as candidates.
> 
> The docstring of `execute-extended-command' says:
> 
>   To pass a prefix argument to the command you are invoking,
>   give a prefix argument to ‘execute-extended-command’.
> 
> And indeed `C-u M-x forward-char RET' moves 4 chars forwards.
> So you can't use `C-u M-x TAB' to limit the number of completions.
> Maybe then use another prefix like `C-x', i.e. `C-x M-x'?
> This is similar to how `C-x M-:' combines the prefix `C-x'
> to run `repeat-complex-command'.

1. The context here was using a prefix arg for `C-h f'
and `C-h v'.

2. I mentioned that _my_ version of `describe-function'
lets a prefix arg limit candidates to commands.

3. So `C-u M-x describe-function' limits candidates to
commands.

(Similarly for `C-h v'.)

Why are you changing the subject here to invocation of
non-`describe-*' commands with `M-x'?  What's the point?

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Juri Linkov]

>> >> > (defun foo (&optional beg end)
>> >> >   (interactive "r")
>> >> >   (message "FOO"))
>> >>
>> >> Yes, I'm sure.  'M-x foo RET' without an active
>> >> region raises an error:
>> >>   command-execute: The mark is not active now
>> >
>> > I don't see that in Emacs 27.1 or prior (with
>> > `emacs -Q').  Is this perhaps new for 28?
>> > If so, why would we do that?
>> 
>> It raises an error because the value of
>> mark-even-if-inactive was changed to nil,
>> so the region exists only when explicitly activated.
>
> Really?  That's a horrible change in default behavior
> (IMHO).  Why was that done?

No, not by default, only customized to nil.
But I don't think that changing the default
would be "a horrible change".

> 1. The context here was using a prefix arg for `C-h f'
> and `C-h v'.
>
> 2. I mentioned that _my_ version of `describe-function'
> lets a prefix arg limit candidates to commands.
>
> 3. So `C-u M-x describe-function' limits candidates to
> commands.
>
> (Similarly for `C-h v'.)
>
> Why are you changing the subject here to invocation of
> non-`describe-*' commands with `M-x'?  What's the point?

Because this is the subject of this thread.

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> > 1. The context here was using a prefix arg for `C-h f'
> >    and `C-h v'.
> > 2. I mentioned that _my_ version of `describe-function'
> >    lets a prefix arg limit candidates to commands.
> > 3. So `C-u M-x describe-function' limits candidates to
> >    commands.
> >
> > Why are you changing the subject here to invocation of
> > non-`describe-*' commands with `M-x'?  What's the point?
> 
> Because this is the subject of this thread.

OK, but the change you made there was abrupt
and not called out.  This was the _immediate_
context:

 JL> Using a prefix arg for 'C-h f' and 'C-h v'
 JL> to limit the list of completions would be nice.
 JL> 
 JL> But I guess M-x can't use a prefix arg to
 JL> limit completions?
DA>
DA> Why do you think so?
DA> `C-u M-x describe-function TAB' (with my
DA> definition from `help-fns+.el') shows only
DA> commands as candidates.

The context was about using a prefix arg with
`describe-function', to have it use only commands.
You seemed to say that that wouldn't work with `M-x'.

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> Date: Thu, 18 Feb 2021 18:24:38 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Kangas <stefankangas@gmail.com>,
>  "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> 
> > It raises an error because the value of
> > mark-even-if-inactive was changed to nil,
> > so the region exists only when explicitly activated.
> 
> Really?  That's a horrible change in default behavior
> (IMHO).  Why was that done?
> 
> And that simple `foo' definition is enough to point
> to regressive behavior.  That command should just
> work - across all Emacs versions.  `M-x foo' should
> not raise an error suddenly in Emacs 28.  (IMHO.)

I don't understand what you are saying.  Commands with "r" interactive
spec always signaled an error if mark wasn't set, it's a very old and
documented behavior.  What do you mean by "just work" here?

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> I don't understand what you are saying.  Commands with "r" interactive
> spec always signaled an error if mark wasn't set, it's a very old and
> documented behavior.  What do you mean by "just work" here?

Dunno what the mystery is.
For me it just works; no error raised.

(defun foo (&optional beg end)
 (interactive "r")
 (message "FOO"))

1. `emacs -Q'
2. Eval that defun.
3. `M-x foo'

Just works.
No mark set, no active region, no error.
What am I missing?

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "juri@linkov.net" <juri@linkov.net>,
>         "stefankangas@gmail.com"
> 	<stefankangas@gmail.com>,
>         "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Thu, 18 Feb 2021 19:47:37 +0000
> 
> (defun foo (&optional beg end)
>  (interactive "r")
>  (message "FOO"))
> 
> 1. `emacs -Q'
> 2. Eval that defun.
> 3. `M-x foo'
> 
> Just works.
> No mark set, no active region, no error.
> What am I missing?

I have no idea.  Here it signals an error, in Emacs 26, 27, and 28.

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> > (defun foo (&optional beg end)
> >  (interactive "r")
> >  (message "FOO"))
> >
> > 1. `emacs -Q'
> > 2. Eval that defun.
> > 3. `M-x foo'
> >
> > Just works.
> > No mark set, no active region, no error.
> > What am I missing?
> 
> I have no idea.  Here it signals an error, in Emacs 26, 27, and 28.

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> > (defun foo (&optional beg end)
> >  (interactive "r")
> >  (message "FOO"))
> >
> > 1. `emacs -Q'
> > 2. Eval that defun.
> > 3. `M-x foo'
> >
> > Just works.
> > No mark set, no active region, no error.
> > What am I missing?
> 
> I have no idea.  Here it signals an error,
> in Emacs 26, 27, and 28.

OK, interesting.  Here it doesn't.
MS Windows 10
Emacs 23 through 27.1
emacs -Q

`mark-even-if-inactive' = t
(default in Emacs 23 and later)

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "juri@linkov.net" <juri@linkov.net>,
>         "stefankangas@gmail.com"
> 	<stefankangas@gmail.com>,
>         "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Thu, 18 Feb 2021 20:22:37 +0000
> 
> > I have no idea.  Here it signals an error,
> > in Emacs 26, 27, and 28.
> 
> OK, interesting.  Here it doesn't.
> MS Windows 10
> Emacs 23 through 27.1
> emacs -Q
> 
> `mark-even-if-inactive' = t
> (default in Emacs 23 and later)

Are you sure you have no mark?

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> > OK, interesting.  Here it doesn't.
> > MS Windows 10
> > Emacs 23 through 27.1
> > emacs -Q
> >
> > `mark-even-if-inactive' = t
> > (default in Emacs 23 and later)
> 
> Are you sure you have no mark?

No, there's a mark.  (I guess just from
defining the command or invoking it.)

But even if `mark-even-if-inactive' is nil,
the region _exists_, no?

The doc string of `mark-even-if-inactive'
doesn't actually say what the behavior is
when nil.

It says only that if NON-nil then "you can
use the mark" even when inactive.  A guess
is that if nil then you CANNOT use the mark
when inactive.  But what does "use the mark"
really mean, here?

The region exists, even when that var is nil,
no?  The region (and mark) just might not be
_active_.  But if you ever had a mark in that
buffer then the region _exists_.

And the doc for `interactive' code `r' says
nothing about activity - it just says that
`r' returns point and mark.

I see that the Elisp manual does say that
an error is raised for `r' if the mark is not
SET.  But even the manual says nothing about
what `r' does for an inactive region/mark.

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "juri@linkov.net" <juri@linkov.net>,
>         "stefankangas@gmail.com"
> 	<stefankangas@gmail.com>,
>         "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Thu, 18 Feb 2021 20:45:55 +0000
> 
> > Are you sure you have no mark?
> 
> No, there's a mark.  (I guess just from
> defining the command or invoking it.)

No, defining a command doesn't set the mark, nor does invoking it.
maybe you've pasted the definition from somewhere?

> But even if `mark-even-if-inactive' is nil,
> the region _exists_, no?

If the mark is set, then the region exists, but it is not necessarily
"active".

> The doc string of `mark-even-if-inactive'
> doesn't actually say what the behavior is
> when nil.

It does here:

  Non-nil means you can use the mark even when inactive.
  This option makes a difference in Transient Mark mode.
  When the option is non-nil, deactivation of the mark
  turns off region highlighting, but commands that use the mark
  behave as if the mark were still active.

As usual, when the description is about what happens when something is
non-nil means that those things do NOT happen when that thing is nil.

> It says only that if NON-nil then "you can
> use the mark" even when inactive.  A guess
> is that if nil then you CANNOT use the mark
> when inactive.  But what does "use the mark"
> really mean, here?

It mentions Transient Mark mode, and that should tell you what that
means.

> I see that the Elisp manual does say that
> an error is raised for `r' if the mark is not
> SET.  But even the manual says nothing about
> what `r' does for an inactive region/mark.

It does now.

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> > > Are you sure you have no mark?
> >
> > No, there's a mark.  (I guess just from
> > defining the command or invoking it.)
> 
> No, defining a command doesn't set the mark, nor does invoking it.
> maybe you've pasted the definition from somewhere?

No, I didn't paste it.  I typed it.  I didn't
use `C-SPC', but I clearly must have done
something that resulted in setting the mark.
 
> > I see that the Elisp manual does say that
> > an error is raised for `r' if the mark is not
> > SET.  But even the manual says nothing about
> > what `r' does for an inactive region/mark.
> 
> It does now.

OK.  And the doc string?

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "juri@linkov.net" <juri@linkov.net>,
>         "stefankangas@gmail.com"
> 	<stefankangas@gmail.com>,
>         "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Fri, 19 Feb 2021 17:53:15 +0000
> 
> > > I see that the Elisp manual does say that
> > > an error is raised for `r' if the mark is not
> > > SET.  But even the manual says nothing about
> > > what `r' does for an inactive region/mark.
> > 
> > It does now.
> 
> OK.  And the doc string?

The doc string didn't say anything about signaling an error to begin
with, so I didn't see any reason to make any changes there.

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[martin rudalics]

 >> (defun foo (&optional beg end)
 >>   (interactive "r")
 >>   (message "FOO"))
 >>
 >> 1. `emacs -Q'
 >> 2. Eval that defun.
 >> 3. `M-x foo'
 >>
 >> Just works.
 >> No mark set, no active region, no error.
 >> What am I missing?
 >
 > I have no idea.  Here it signals an error, in Emacs 26, 27, and 28.

Presumably Drew yanks in the definition of 'foo' which does push the
mark.  Hence the "mark will have been set" at the time he does "eval
that defun".  The recipe above is then probably botched.

martin

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> Presumably Drew yanks in the definition of 'foo' which does push the
> mark.  Hence the "mark will have been set" at the time he does "eval
> that defun".  The recipe above is then probably botched.

More or less, yes.  I was not yanking anything
but I typed it in *scratch* and used `C-M-x'.

Just using `M-:', I see that the error is always
raised (regardless of the value of
`mark-even-if-inactive').

But see my remarks about the doc of `interactive'
code `r'.  This isn't covered at all by that.

And the doc of `mark-even-if-inactive' says
nothing about the mark existing - it's only about
whether the mark (assuming it exists) is active.

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "emacs-devel@gnu.org" <emacs-devel@gnu.org>,
>         "stefankangas@gmail.com"
> 	<stefankangas@gmail.com>,
>         "juri@linkov.net" <juri@linkov.net>
> Date: Thu, 18 Feb 2021 23:32:54 +0000
> 
> And the doc of `mark-even-if-inactive' says
> nothing about the mark existing - it's only about
> whether the mark (assuming it exists) is active.

Precisely.  So what exactly is the problem with the docs?  Are you
saying you don't know what "active mark" or "active region" means?
I'd be surprised.

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> > And the doc of `mark-even-if-inactive' says
> > nothing about the mark existing - it's only about
> > whether the mark (assuming it exists) is active.
> 
> Precisely.  So what exactly is the problem with the docs?  Are you
> saying you don't know what "active mark" or "active region" means?
> I'd be surprised.

No, I'm not saying that.  To me, that doc describes
sensible behavior.

I sloppily wrote "No mark set, no active region."
in my recipe to you, when I should have said
"Mark not active".  And that mistake set off some
confused back & forth about whether there was a
mark (i.e., mark set).

The problem I raised from the beginning is with
the doc of `interactive' (for code `r').

The doc string says only:

 r -- Region: point and mark as 2 numeric args,
      smallest first.  Does no I/O.

It says nothing about raising an error if
not active and `mark-even*' is nil.  The
expectation from reading that is that
`interactive' just provides the region
limits as args, always - no error.

The Elisp manual says this for `r':

 Point and the mark, as two numeric arguments,
 smallest first.  This is the only code letter
 that specifies two successive arguments rather
 than one.  This will signal an error if the
 mark is not set in the buffer which is current
 when the command is invoked.  No I/O.

It says that an error is raised when there is
no mark (which is true, but incomplete).  It
should perhaps instead say (and so should the
doc string, if we want to be complete), that an
error is raised if either (1) there is no mark
or (2) the mark is not active and
`mark-even-if-inactive' is nil.

Another possibility is to say nothing at all
about raising an error.  The `mark-even*'
behavior doesn't really have anything to do
with `interactive' specifically.

You might say sure, we need not say anything
about an error from inactive with `mark-even*'
= nil, but we should mention an error if no
mark.  IOW, just what we say now.

I'd argue that it can be confusing to just
call out that special (not too common) case,
and not mention the more common case where
an error is raised for an inactive region
(modulo the option value).

But I recognize that that more common error
is not, strictly speaking, about `r'.  And I
don't feel strongly about this.

I'll point out that this all started because
Juri wrote:

  I'd like also to suggest to filter out commands
  having "r" in their interactive specs - they
  signal an error when the region is not active,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  so no need to show them.

And yes, we can assume that Juri (like I) knows
"what 'active mark' or 'active region' means."

What he said is perhaps further demonstration
that the `interactive' `r' doc could be clearer
or more helpful.

Had he not been mistaken about that, there would
have been no follow-up by me, pointing out that
an inactive region _doesn't_ necessarily raise
an error for `interactive' `r'.
___

As for whether commands with `r' in their specs
should be filtered out, presumably the filtering
condition should be this:

(1) no mark OR
    [(2) `mark-even*' = nil
     AND
     (3) inactive region]

(Personally, I'm not in favor of such filtering,
as I've said, but at least it should be accurate.)
___

One source of confusion in our exchange was no
doubt due to my likely using `M-x set-variable'
to set option `mark-even*' to nil.  I shouldn't
have done that.

There's apparently the equivalent of a :set
function in the C definition of the defcustom.
Bypassing that (which `set-variable' does)
results in no error being raised when the region
is inactive.

(This happens if you use `set-variable' at the
outset or after using Customize to "Revert this
session's customization".  It doesn't happen if
you use `set-variable' after using Customize
just to toggle the value.)

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> Date: Thu, 18 Feb 2021 18:24:38 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Kangas <stefankangas@gmail.com>,
>  "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> 
> > It raises an error because the value of
> > mark-even-if-inactive was changed to nil,
> > so the region exists only when explicitly activated.
> 
> Really?  That's a horrible change in default behavior
> (IMHO).  Why was that done?

Btw, in the current master branch mark-even-if-inactive is t, not nil.
So I really don't understand what is the above about.  What did I
miss?

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> > > It raises an error because the value of
> > > mark-even-if-inactive was changed to nil,
> > > so the region exists only when explicitly activated.
>
> Btw, in the current master branch mark-even-if-inactive is t, not nil.
> So I really don't understand what is the above about.  What did I
> miss?

I think Juri corrected himself, afterward.
Apparently he meant that IF you customize
that option to nil THEN...

Re: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "juri@linkov.net" <juri@linkov.net>,
>         "stefankangas@gmail.com"
> 	<stefankangas@gmail.com>,
>         "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Thu, 18 Feb 2021 19:49:01 +0000
> 
> > > > It raises an error because the value of
> > > > mark-even-if-inactive was changed to nil,
> > > > so the region exists only when explicitly activated.
> >
> > Btw, in the current master branch mark-even-if-inactive is t, not nil.
> > So I really don't understand what is the above about.  What did I
> > miss?
> 
> I think Juri corrected himself, afterward.
> Apparently he meant that IF you customize
> that option to nil THEN...

No, the error is signaled even if that variable is non-nil.
That's what happens in "emacs -Q".

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> the error is signaled even if that variable is non-nil.
> That's what happens in "emacs -Q".

Not here.

Emacs 27.1, emacs -Q
If `mark-even-if-inactive' is non-nil then no error
If it's nil then the error is raised.

Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Wed, 17 Feb 2021 11:27:11 -0800
> 
> > commit 927b88571cebb4f64aca360fbfa5d15a1f922ad6
> > Author: Eli Zaretskii <eliz@gnu.org>
> > Commit: Eli Zaretskii <eliz@gnu.org>
> >
> >     Disable filtering of commands in M-x completion
> >
> >     This makes the default behavior like it was before:
> >     M-x completion doesn't filter out any commands.  To
> >     have commands filtered based on their relevance to the
> >     current buffer's modes, customize the option
> >     'read-extended-command-predicate' to call
> >     'command-completion-default-include-p'.
> 
> I can understand this decision for Emacs 28

There's no decision yet.  We are still discussing how best to do it,
and useful ideas are still being brought up.  Including by you, in the
message to which I'm replying.

There's no rush to make that decision, as Emacs 28 is not around the
corner yet.

Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> There's no decision yet.  We are still discussing how best to do it,
> and useful ideas are still being brought up.  Including by you, in the
> message to which I'm replying.

I am happy to hear that, thank you for clarifying.

> There's no rush to make that decision, as Emacs 28 is not around the
> corner yet.

Fully agreed.

RE: [External] : Re: master 927b885 1/3: Disable filtering of commands in M-x completion

[Drew Adams]

> I think we should take one of the below actions:
> 
> a) Add a key to show the unfiltered list of matches in `M-x'.

Add a key _to_ filter, instead.  (I mentioned
that my own code provides several keys for
filtering according to different criteria.)

I think Lars proposed (also?) providing another
command, which people could use instead of (or
in addition to) the command bound by default to
`M-x'.

Users could bind that new command to keys.  We
could later decide whether it should have a key
binding by default, or whether it should take
over `M-x' by default.  All of that should be
decided after more user experience and after
discussion of the pros & cons (like the current
discussion, but with benefit of experience). 

> (We could use any key, but how about just 
> using `M-x M-x' to show the unfiltered list?

That would be a particularly bad key choice, for
the reason you mention next:

> It would affect recursive minibuffers, but we
> could just require a third `M-x' for that.)

We could "just"?  Why would we choose a key that
requires jumping through such extra hoops?  Why
would we choose a key that _otherwise_ makes
sense within `M-x'?

And this is only one kind of filtering, even for
command completions.  And command completions
are only one kind of completion.  More general
completion-filtering keys are what are called
for, in addition, perhaps to some command-specific
ones (e.g. some, like what's currently proposed,
that are for command completion).

> And then we should consider making this feature
> the default, preferably already in Emacs 28 but
> otherwise in Emacs 29.

What's the hurry?  Emacs has used `M-x' for 35+
years.  Suddenly this new shiny object is a MUST,
and needs to replace the longstanding behavior?

> Here is why:
> 
> I have never understood why Emacs suggests
> commands for execution in a context where they
> will obviously fail.

1. "Obviously fail" has recently been argued by
Lars and others as being an insufficient test.

2. People have explained why.  They've said how
it can be useful for `M-x' to suggest also
commands that might fail.

3. You haven't understood why.  OK.  And why
is that lack of understanding a reason why the
behavior should change ("Here is why")?

> I think this is a glaring deficiency in our
> default UI

Glaring?  35+ years...

> OTOH, it seems to me that the target audience for disabling this
> feature is (or will be) mostly "hardcore"/"veteran"/"advanced"
> users who are very used to and like the old behavior.

Oh, here we go again.  Another long-time,
hardcore, advanced user trying to speak for
the unwashed masses of potential newbies who
are claimed to suffer from some longstanding
behavior.

Yes, the status quo gets more support, by
default.  If the opposite were true we
wouldn't have Emacs as we know it.  Every
fly-by-night or passing fad would have been
immediately adopted.  Churn, churn, churn.

Emacs's design-change conservatism is
recognized too little as one of its strengths.
Change comes with difficulty.  Emacs has a
large, solid user base.  A tiny fraction of
its habits, experience, and preferences get
expressed here.

> having an option to opt-out of this behavior.

What's so special about this behavior that
it should become the new default behavior?

Why not add it as an opt-in, and wait and see
how much support it gains?  What's the hurry?

> (I also think we should add a `describe-command'
> to try to better cover this use-case.)

FWIW, I added `describe-command' to my library
`help-fns+.el' decades ago.  I bind it to
`C-h c' (and I bind `describe-key-briefly' to
`C-h C-c' - I find it less useful than
`describe-command').

https://www.emacswiki.org/emacs/download/help-fns%2b.el