bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

The completing-read function in Emacs allows you to create a simple 
interactive prompt where the user can type to complete a string based
on a given collection.  REQUIRE-MATCH determines whether the input 
must match an item in the collection.

I find the design of completing-read as half-baked because INITIAL-VALUE 
always works as an independent entity, always unaware of COLLECTION and 
REQUIRE-MATCH

Thus, there exists no capabilities for possible interactions that could 
influence how the completion works.

For instance, suppose you want INITIAL-INPUT to be takes from collection,
and then have cycling start from a certain element after INITIAL-INPUT
is displayed.  You can't.  The only thing you can do is select an option
by automatically completing as much as possible.  

As for the cycling part, no proper functionality has been introduced.  INITIAL-INPUT
is there just to insert something to the minibuffer without any regard to possible 
restriction that one might want imposed on it and what gets displayed after INITIAL-VALUE.  

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Eli Zaretskii]

> Date: Tue, 22 Aug 2023 22:04:06 +0000
> From:  Heime via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> 
> The completing-read function in Emacs allows you to create a simple 
> interactive prompt where the user can type to complete a string based
> on a given collection.  REQUIRE-MATCH determines whether the input 
> must match an item in the collection.
> 
> I find the design of completing-read as half-baked because INITIAL-VALUE 
> always works as an independent entity, always unaware of COLLECTION and 
> REQUIRE-MATCH

"Half-baked" is a harsh qualification.  I don't think it makes your
arguments more attractive or convincing, so my advice is to avoid such
derogatory epithets on issues which you have just recently learned and
about which you might still be missing quite a lot.

In any case, from where I stand, any significant changes in the API of
completing-read that are backward incompatible are out of the question
at this stage, and if this discussion leads to a conclusion that an
incompatible API could be useful, it will have to be a separate API.

> Thus, there exists no capabilities for possible interactions that could 
> influence how the completion works.
> 
> For instance, suppose you want INITIAL-INPUT to be takes from collection,
> and then have cycling start from a certain element after INITIAL-INPUT
> is displayed.  You can't.  The only thing you can do is select an option
> by automatically completing as much as possible.  
> 
> As for the cycling part, no proper functionality has been introduced.  INITIAL-INPUT
> is there just to insert something to the minibuffer without any regard to possible 
> restriction that one might want imposed on it and what gets displayed after INITIAL-VALUE.  

Adding Stefan.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Wednesday, August 23rd, 2023 at 11:29 PM, Eli Zaretskii <eliz@gnu.org> wrote:


> > Date: Tue, 22 Aug 2023 22:04:06 +0000
> > From: Heime via "Bug reports for GNU Emacs,
> > the Swiss army knife of text editors" bug-gnu-emacs@gnu.org
> > 
> > The completing-read function in Emacs allows you to create a simple
> > interactive prompt where the user can type to complete a string based
> > on a given collection. REQUIRE-MATCH determines whether the input
> > must match an item in the collection.
> > 
> > I find the design of completing-read as half-baked because INITIAL-VALUE
> > always works as an independent entity, always unaware of COLLECTION and
> > REQUIRE-MATCH
> 
> 
> "Half-baked" is a harsh qualification. I don't think it makes your
> arguments more attractive or convincing, so my advice is to avoid such
> derogatory epithets on issues which you have just recently learned and
> about which you might still be missing quite a lot.

How can we describe it - primitive - then.  What lots am I missing about 
the behaviour of INITIAL-VALUE being independent about the state of the 
other variables ?  And that entries in collection always start from index
zero when cycling is used.  I have found its capability limited, having
actually used it.

Nevertheless, if I am missing on things, I would be glad to be corrected 
so that I can change my code accordingly. 
 
> In any case, from where I stand, any significant changes in the API of
> completing-read that are backward incompatible are out of the question
> at this stage, and if this discussion leads to a conclusion that an
> incompatible API could be useful, it will have to be a separate API.

I expected that as the functionalities get enhanced, some deficiencies
also get pumped up.  Giving more control to the programmer about what 
gets displayed in the minibuffer.


 
> > Thus, there exists no capabilities for possible interactions that could
> > influence how the completion works.
> > 
> > For instance, suppose you want INITIAL-INPUT to be takes from collection,
> > and then have cycling start from a certain element after INITIAL-INPUT
> > is displayed. You can't. The only thing you can do is select an option
> > by automatically completing as much as possible.
> > 
> > As for the cycling part, no proper functionality has been introduced. INITIAL-INPUT
> > is there just to insert something to the minibuffer without any regard to possible
> > restriction that one might want imposed on it and what gets displayed after INITIAL-VALUE.
> 
> 
> Adding Stefan.
>

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> How can we describe it - primitive - then.  What lots am I missing about
> the behaviour of INITIAL-VALUE being independent about the state of the
> other variables ?

I can't really answer that because I don't fully understand what it is
you're trying to do and feel that you can't do with this API.

Maybe you're missing what other people usually miss:
`completing-read` is also used for completion of things that have
structure, like file-names, and where the INITIAL-INPUT may be a good
starting point for the user while not being a valid end point
(i.e. a choice rejected by REQUIRE-MATCH).

> And that entries in collection always start from index
> zero when cycling is used.

Don't know what you mean by that.  Could you clarify?

By and large the COLLECTION argument is treated as a *set*, i.e. without
any ordering.  Instead, the ordering is chosen by the UI (i.e. the
completion code) and can depend on various user config choices, so
I don't know what you mean by "always start from index zero".

I'm also not sure what you mean by "when cycling is used".  Are you
referring to the first choice offered by `minibuffer-complete` when
`completion-cycle-threshold` is set to something like t?

> I have found its capability limited, having actually used it.

Are you talking about "used it" as an end-user or as a coder?

> I expected that as the functionalities get enhanced, some deficiencies
> also get pumped up.  Giving more control to the programmer about what
> gets displayed in the minibuffer.

The intention is to try and make room for 3 parts:

- what the programmer provides to `completing-read`.
- what the end-user sees.
- between those two, the specific completion UI chosen by the end-user.

So indeed, we don't want to offer too much control to the programmer who
calls `completing-read`, so as to give more freedom to the
completion UI.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

------- Original Message -------
On Thursday, August 24th, 2023 at 1:07 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > How can we describe it - primitive - then. What lots am I missing about
> > the behaviour of INITIAL-VALUE being independent about the state of the
> > other variables ?
> 
> I can't really answer that because I don't fully understand what it is
> you're trying to do and feel that you can't do with this API.
> 
> Maybe you're missing what other people usually miss:
> `completing-read` is also used for completion of things that have
> structure, like file-names, and where the INITIAL-INPUT may be a good
> starting point for the user while not being a valid end point
> (i.e. a choice rejected by REQUIRE-MATCH).

I am aware and have no problem with the points mentioned - structured completion and
INITIAL-INPUT not being a valid end point with regards to REQUIRE-MATCH as 't'.
 
> > And that entries in collection always start from index
> > zero when cycling is used.
> 
> Don't know what you mean by that. Could you clarify?
> 
> By and large the COLLECTION argument is treated as a set, i.e. without
> any ordering. Instead, the ordering is chosen by the UI (i.e. the
> completion code) and can depend on various user config choices, so
> I don't know what you mean by "always start from index zero".

Consider simple cycling through completion options using the down arrow key (<down>) in 
the completing-read function.  When using completing-read, you can cycle through the 
available completion options by pressing the 'down' arrow key.

Consider

  (interactive
    (let ( (cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")) )
      (list
        (completing-read "Grapheme: " cseq nil t "alpha")) ))

I have no problem with possibility of having INITIAL not in COLLECTION.
But, suppose that I pre-insert the first element in collection as INITIAL
value in minibuffer.
 
Then, repeatedly hit <down>.  You get

alpha alpha beta gamma delta epsilon zeta eta alpha beta gamma and-so-on

You do actually get that the consecutive entries appearing in the minibuffer
have direct correspondence to to order found in COLLECTION.

Once INITIAL is inserted, I cannot instruct completing-read to continue
sequentially starting from a particular element.

For instance, I cannot get the following result when using simple cycling 
through collection options when repeatedly hitting the <down> key.

To achieve

alpha epsilon zeta eta alpha beta gamma delta epsilon and-so-on

> I'm also not sure what you mean by "when cycling is used". Are you
> referring to the first choice offered by `minibuffer-complete` when
> `completion-cycle-threshold` is set to something like t?
> 
> > I have found its capability limited, having actually used it.
> 
> 
> Are you talking about "used it" as an end-user or as a coder?

As a coder and after some tests mimicking a user using only simple cycling. 
 
> > I expected that as the functionalities get enhanced, some deficiencies
> > also get pumped up. Giving more control to the programmer about what
> > gets displayed in the minibuffer.
> 
> 
> The intention is to try and make room for 3 parts:
> 
> - what the programmer provides to `completing-read`.
> - what the end-user sees.
> - between those two, the specific completion UI chosen by the end-user.
> 
> So indeed, we don't want to offer too much control to the programmer who
> calls `completing-read`, so as to give more freedom to the
> completion UI. - Stefan

My school of thought is always allow the programmer to define how much freedom
he intends to give the completion UI.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Then, repeatedly hit <down>.  You get
>
> alpha alpha beta gamma delta epsilon zeta eta alpha beta gamma and-so-on

Ah, OK, yes, that sounds like a problem (one I've heard before).
I think it's a problem in the cycling code rather than in INITIAL-VALUE
or COLLECTION.  E.g. the same problem occurs if the initial input is
empty and the user types `alpha C-n`.

> You do actually get that the consecutive entries appearing in the
> minibuffer have direct correspondence to to order found in COLLECTION.

For that form of cycling, apparently so, yes.  I don't categorize it as
"completion" so I haven't looked very closely at that code :-)
There are other ways to do cycling which use a different ordering.

The cycling through completions via `next-line-or-history-element` is
not very configurable, AFAICT.

> Once INITIAL is inserted, I cannot instruct completing-read to continue
> sequentially starting from a particular element.

Indeed.

> For instance, I cannot get the following result when using simple cycling 
> through collection options when repeatedly hitting the <down> key.
>
> To achieve
>
> alpha epsilon zeta eta alpha beta gamma delta epsilon and-so-on

Could you give a bit more context about why/where you'd want to do that
so I can better imagine how to attack the problem?

Hopefully it would explain why you want "alpha epsilon ..." first but
"alpha beta .." later.  Also for such uses of `completing-read` we
usually recommend against a non-nil value for INITIAL-INPUT, so that
context should try to explain why you use "alpha" as INITIAL-INPUT.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

------- Original Message -------
On Thursday, August 24th, 2023 at 4:05 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > Then, repeatedly hit <down>. You get
> > 
> > alpha alpha beta gamma delta epsilon zeta eta alpha beta gamma and-so-on
> 
> Ah, OK, yes, that sounds like a problem (one I've heard before).
> I think it's a problem in the cycling code rather than in INITIAL-VALUE
> or COLLECTION. E.g. the same problem occurs if the initial input is
> empty and the user types `alpha C-n`.

That has been my assessment as well.  It is not a problem of INITIAL-VALUE,
but a problem in the cycling mechanism.
 
> > You do actually get that the consecutive entries appearing in the
> > minibuffer have direct correspondence to the order found in COLLECTION.
> 
> For that form of cycling, apparently so, yes. I don't categorize it as
> "completion" so I haven't looked very closely at that code :-)
> There are other ways to do cycling which use a different ordering.

Correct, it is not categorised as completion.
 
> The cycling through completions via `next-line-or-history-element` is
> not very configurable, AFAICT.
> 
> > Once INITIAL is inserted, I cannot instruct completing-read to continue
> > sequentially starting from a particular element.
> 
> Indeed.
> 
> > For instance, I cannot get the following result when using simple cycling
> > through collection options when repeatedly hitting the <down> key.
> > 
> > To achieve
> > 
> > alpha epsilon zeta eta alpha beta gamma delta epsilon and-so-on
> 
> Could you give a bit more context about why/where you'd want to do that
> so I can better imagine how to attack the problem?
> 
> Hopefully it would explain why you want "alpha epsilon ..." first but
> "alpha beta .." later. 

In such instance one considers collection as a cyclic list for the case of 
simple cycling.  But with possibility of starting from a particular index.

> Also for such uses of `completing-read` we usually recommend against a 
> non-nil value for INITIAL-INPUT, so that context should try to explain 
> why you use "alpha" as INITIAL-INPUT. - Stefan

Because using INITIAL is the only way available to pre-insert a value in 
the minibuffer.

Hence, if I want a pre-insertion candidate for the first element in collection
I get 'alpha' repeated twice.  First from the insertion via INITIAL, and then 
again through the simple cycling of collection (that starts from 'alpha').  

If one can instruct the starting position in collection and subsequently consider
collection as a cyclic list, the problem of ending with the following is solved.

alpha alpha beta gamma delta epsilon zeta eta alpha and-so-on

which means

INITIAL COLLECTION(0) COLLECTION(1) and-so-on.

--------

Starting from entry 1 of collection, you would get

alpha beta gamma delta epsilon zeta eta alpha and-so-on

which means

INITIAL COLLECTION(1) COLLECTION(2) and-so-on

--------

And in general

INITIAL COLLECTION(i) COLLECTION(i+1) ... COLLECTION(0) COLLECTION(1) and-so-on

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Could you give a bit more context about why/where you'd want to do that
>> so I can better imagine how to attack the problem?
>> 
>> Hopefully it would explain why you want "alpha epsilon ..." first but
>> "alpha beta .." later. 
>
> In such instance one considers collection as a cyclic list for the case of 
> simple cycling.  But with possibility of starting from a particular index.

So the initial "alpha" is simply not considered, OK, that makes sense,
so given that the ordering of COLLECTION happens to be preserved, you
can get the above behavior if you reorder COLLECTION before the call, as
in:

    (let ((cseq '("epsilon" "zeta" "eta" "alpha" "beta" "gamma" "delta")))
      (completing-read "Grapheme: " cseq nil t "alpha"))

>> Also for such uses of `completing-read` we usually recommend against a 
>> non-nil value for INITIAL-INPUT, so that context should try to explain 
>> why you use "alpha" as INITIAL-INPUT. - Stefan
> Because using INITIAL is the only way available to pre-insert a value in 
> the minibuffer.

That's a bit tautological.
What I meant is why do you need to preinsert a value?

The normal/recommended call looks like:

    (let ((cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")))
      (completing-read (format-prompt "Grapheme" "alpha")
                       cseq nil t nil nil "alpha"))

[ Yes, it's a bit cumbersome: hysterical raisins :-(  ]


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Thursday, August 24th, 2023 at 4:58 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > > Could you give a bit more context about why/where you'd want to do that
> > > so I can better imagine how to attack the problem?
> > > 
> > > Hopefully it would explain why you want "alpha epsilon ..." first but
> > > "alpha beta .." later.
> > 
> > In such instance one considers collection as a cyclic list for the case of
> > simple cycling. But with possibility of starting from a particular index.
> 
> 
> So the initial "alpha" is simply not considered, OK, that makes sense,
> so given that the ordering of COLLECTION happens to be preserved, you
> can get the above behavior if you reorder COLLECTION before the call, as
> in:
> 
> (let ((cseq '("epsilon" "zeta" "eta" "alpha" "beta" "gamma" "delta")))
> (completing-read "Grapheme: " cseq nil t "alpha"))
 
> > > Also for such uses of `completing-read` we usually recommend against a
> > > non-nil value for INITIAL-INPUT, so that context should try to explain
> > > why you use "alpha" as INITIAL-INPUT. - Stefan
> > > Because using INITIAL is the only way available to pre-insert a value in
> > > the minibuffer.
> 
> That's a bit tautological.
> What I meant is why do you need to preinsert a value?

In particular cases, the pre-insertion indicates to the user
the kind of input structure of collection that can help him
apply a completion strategy for traversing the possibilities.

 
> The normal/recommended call looks like:
> 
> (let ((cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")))
> (completing-read (format-prompt "Grapheme" "alpha")
> cseq nil t nil nil "alpha"))
> 
> [ Yes, it's a bit cumbersome: hysterical raisins :-( ] - Stefan

Correct.  The solution involves an unwieldy parameter configurations, and possibly 
using convoluted workarounds to achieve the desired behavior.  This scenario implies
that using the function as intended requires a level of effort, understanding, and 
manipulation that goes beyond direct usage.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> The normal/recommended call looks like:
>> 
>> (let ((cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")))
>> (completing-read (format-prompt "Grapheme" "alpha")
>> cseq nil t nil nil "alpha"))
>> 
>> [ Yes, it's a bit cumbersome: hysterical raisins :-( ] - Stefan
>
> Correct.  The solution involves an unwieldy parameter configurations, and possibly
> using convoluted workarounds to achieve the desired behavior.  This scenario implies
> that using the function as intended requires a level of effort, understanding, and
> manipulation that goes beyond direct usage.

Agreed.  That argues in favor of introducing a new function.  But that
new function would be so similar to `completing-read` that it will lead
to users wondering when to use which or why there are two unless we mark
`completing-read` as obsolete, which would in turn cause a lot of
warnings in a lot of packages and code churn.

So, we've been hesitating to make this change for a while, not sure
which choice is least bad.

But at this point do you consider the behavior of

    (let ((cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")))
      (completing-read (format-prompt "Grapheme" "alpha")
                       cseq nil t nil nil "alpha"))

to be a good solution for your original problem?

[ BTW, to me the main benefit of a new function is that we could try
  and make it so that the user can choose between a UI where the default
  is not inserted in the initial minibuffer (as in the above code), or
  one where it is (presumably pre-selected so you can delete it with
  a simple DEL), as is common in many other applications.  ]


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Drew Adams]

> [ BTW, to me the main benefit of a new function is that we could try
>   and make it so that the user can choose between a UI where the default
>   is not inserted in the initial minibuffer (as in the above code), or
>   one where it is (presumably pre-selected so you can delete it with
>   a simple DEL), as is common in many other applications.  ]

FWIW, this is what Icicles offers (since the beginning).

https://www.emacswiki.org/emacs/Icicles_-_Customization_and_General_Tips#icicle-default-value

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Gregory Heytings]

>
> But at this point do you consider the behavior of
>
>    (let ((cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")))
>      (completing-read (format-prompt "Grapheme" "alpha")
>                       cseq nil t nil nil "alpha"))
>
> to be a good solution for your original problem?
>

Is this not a better solution to his original problem?

(defun my-completing-read (prompt collection start)
   (minibuffer-with-setup-hook
       (:append
        (lambda ()
          (setq-local actual-minibuffer-default-add-function
                      minibuffer-default-add-function)
          (setq-local minibuffer-default-add-function
                      '(lambda () (setq minibuffer-default-add-done nil)
                         (funcall actual-minibuffer-default-add-function)))
          (next-history-element start)
          (move-end-of-line nil)))
     (completing-read (format-prompt prompt nil) collection nil t)))

(my-completing-read "Grapheme" '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta") 1)

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> (defun my-completing-read (prompt collection start)
>   (minibuffer-with-setup-hook
>       (:append
>        (lambda ()
>          (setq-local actual-minibuffer-default-add-function
>                      minibuffer-default-add-function)
>          (setq-local minibuffer-default-add-function
>                      '(lambda () (setq minibuffer-default-add-done nil)
>                         (funcall actual-minibuffer-default-add-function)))

Why use ' on `lambda`?
BTW, rather than the above two `setq-local`s, I think you can write:

    (add-function :before (local 'minibuffer-default-add-function)
                  (lambda () (setq minibuffer-default-add-done nil)))


-- Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

------- Original Message -------
On Thursday, August 24th, 2023 at 9:27 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > > The normal/recommended call looks like:
> > > 
> > > (let ((cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")))
> > > (completing-read (format-prompt "Grapheme" "alpha")
> > > cseq nil t nil nil "alpha"))
> > > 
> > > [ Yes, it's a bit cumbersome: hysterical raisins :-( ] - Stefan
> > 
> > Correct. The solution involves an unwieldy parameter configurations, and possibly
> > using convoluted workarounds to achieve the desired behavior. This scenario implies
> > that using the function as intended requires a level of effort, understanding, and
> > manipulation that goes beyond direct usage.
> 
> 
> Agreed. That argues in favor of introducing a new function. But that
> new function would be so similar to `completing-read` that it will lead
> to users wondering when to use which or why there are two unless we mark
> `completing-read` as obsolete, which would in turn cause a lot of
> warnings in a lot of packages and code churn.
> 
> So, we've been hesitating to make this change for a while, not sure
> which choice is least bad.
> 
> But at this point do you consider the behavior of
> 
> (let ((cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")))
> (completing-read (format-prompt "Grapheme" "alpha")
> cseq nil t nil nil "alpha"))
> 
> to be a good solution for your original problem?

It is not, because the intention is on prefilling the minibuffer with "alpha"  
rather than considering "alpha" as DEF.  

It appears that the maintainers might not be prioritizing the initial intention
of prefilling the minibuffer.  Currently, their emphasis seems to be on encouraging 
developers to either utilize the "DEF" approach or actively discourage the use of 
"INITIAL".  But, I haven't come across a clear and well-founded reasoning behind 
these shifts in approach.

Let just remember that COLLECTION might already have been defined somewhere else
according to some arrangement rule regarding the order of its entries.  Rearranging
the actual order of elements in collection just to fit the limited requirements
of completing-read is not to be considered good function design.  Apart from the 
reality that rearranging the entries in COLLECTION might be difficult or even
impossible to achieve.  The example I show is straightforward in that you can see 
the entries and you can easily arrange them as you had suggested earlier
with

(let ((cseq '("epsilon" "zeta" "eta" "alpha" "beta" "gamma" "delta")))
  (completing-read "Grapheme: " cseq nil t "alpha"))

But this is only easily done only when collection is actually being constructed
in-place via the 'let' clause.   But once COLLECTION starts getting imported
from somewhere else (via a call to same other function for instance), your suggested
solution is impossible to achieve. 
 
> [ BTW, to me the main benefit of a new function is that we could try
> and make it so that the user can choose between a UI where the default
> is not inserted in the initial minibuffer (as in the above code), or
> one where it is (presumably pre-selected so you can delete it with
> a simple DEL), as is common in many other applications. ] - Stefan

I am neutral towards have two functions or a single one.

One can be considered as enhancing the functionaity of the other, leaving
the former for simpler use cases (and to conform in regards to backward 
compatibility) - if sensible names can be made up such that no confusion 
about their use can arise.  

Failing that, one can have a single function, where the coder can clearly
specify the way in which he wants it used.  Currently the design is for
completing-read to determine by itself the strategy it should use, without
giving the coder actual control on how completing-read should behave when 
one is faced with certain implementational circumstances where workarounds
are burdens to avoid rather than embrace.
 

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Gregory Heytings]

>> (defun my-completing-read (prompt collection start)
>>   (minibuffer-with-setup-hook
>>       (:append
>>        (lambda ()
>>          (setq-local actual-minibuffer-default-add-function
>>                      minibuffer-default-add-function)
>>          (setq-local minibuffer-default-add-function
>>                      '(lambda () (setq minibuffer-default-add-done nil)
>>                         (funcall actual-minibuffer-default-add-function)))
>
> Why use ' on `lambda`?
>

I think my finger slipped.

>
> BTW, rather than the above two `setq-local`s, I think you can write:
>
>    (add-function :before (local 'minibuffer-default-add-function)
>                  (lambda () (setq minibuffer-default-add-done nil)))
>

Right.  I was writing that function as it if were for minibuffer.el, and 
was therefore avoiding advices.  But now I see that there are a couple of 
add-function in core anyway (it's not clear to me where the dividing line 
is).  So here's the final version:

(defun my-completing-read (prompt collection start)
   (minibuffer-with-setup-hook
       (:append
        (lambda ()
          (add-function :before (local 'minibuffer-default-add-function)
                        (lambda () (setq minibuffer-default-add-done nil)))
          (next-history-element start)
          (move-end-of-line nil)))
     (completing-read (format-prompt prompt nil) collection nil t)))

(my-completing-read "Grapheme" '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta") 1)

Heime, can you try that and tell us if it solves your problem?

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

------- Original Message -------
On Friday, August 25th, 2023 at 12:30 AM, Gregory Heytings <gregory@heytings.org> wrote:

> > > (defun my-completing-read (prompt collection start)
> > > (minibuffer-with-setup-hook
> > > (:append
> > > (lambda ()
> > > (setq-local actual-minibuffer-default-add-function
> > > minibuffer-default-add-function)
> > > (setq-local minibuffer-default-add-function
> > > '(lambda () (setq minibuffer-default-add-done nil)
> > > (funcall actual-minibuffer-default-add-function)))
> > 
> > Why use ' on `lambda`?
> 
> 
> I think my finger slipped.
> 
> > BTW, rather than the above two `setq-local`s, I think you can write:
> > 
> > (add-function :before (local 'minibuffer-default-add-function)
> > (lambda () (setq minibuffer-default-add-done nil)))
> 
> 
> Right. I was writing that function as it if were for minibuffer.el, and
> was therefore avoiding advices. But now I see that there are a couple of
> add-function in core anyway (it's not clear to me where the dividing line
> is). So here's the final version:
> 
> (defun my-completing-read (prompt collection start)
> (minibuffer-with-setup-hook
> (:append
> (lambda ()
> (add-function :before (local 'minibuffer-default-add-function)
> (lambda () (setq minibuffer-default-add-done nil)))
> (next-history-element start)
> (move-end-of-line nil)))
> (completing-read (format-prompt prompt nil) collection nil t)))
> 
> (my-completing-read "Grapheme" '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta") 1)
> 
> Heime, can you try that and tell us if it solves your problem?

The behaviour is as it should be.  Although we should keep 'start' to correspond to 
collection index which customarily starts from value 0. 

Although I can use the function provided, I rather have all this incorporated in
'completing-read'.  Because 'completing-read' would benefit from it.  Particularly
now that its use is being extended.  This taking into account that my use case
is more aligned with its original operation, except that certain aspects which I have
shown to be important were not recognised at the time.  After some focused attention,
Stefan concurred how I have been making sense, particularly after seeing how `completing-read` 
actually behaves during the form of cycling I discussed at some length.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> It is not, because the intention is on prefilling the minibuffer with
> "alpha" rather than considering "alpha" as DEF.

Could you explain why this is important in your case?

> It appears that the maintainers might not be prioritizing the initial
> intention of prefilling the minibuffer.  Currently, their emphasis
> seems to be on encouraging  developers to either utilize the "DEF"
> approach or actively discourage the use of  "INITIAL".  But, I haven't
> come across a clear and well-founded reasoning behind  these shifts
> in approach.

Not sure what you mean by "shifts".  AFAIK this behavior has been with
Emacs "forever": most minibuffers start empty (but with an associated
default value) rather than starting with an initial value.

This originally comes presumably from the absence of
`transient-mark-mode` (or associated visual highlighting, both of which
were introduced in Emacs-19) which means that it was annoying having to
erase "alpha" before you can type "beta".

Starting with a non-empty minibuffer does happen occasionally, most
importantly for `read-file-name` where we do expect that this initial
input will very likely be a part of the name the user will end up
typing, so its rare that the users need to erase it before they can type
the file name they want.

> But this is only easily done only when collection is actually being
> constructed in-place via the 'let' clause.  But once COLLECTION
> starts getting imported from somewhere else (via a call to same other
> function for instance), your suggested solution is impossible
> to achieve. 

That's partly why I've asked about a concrete example showing the wider
context :-)


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>>> (defun my-completing-read (prompt collection start)
>>>   (minibuffer-with-setup-hook
>>>       (:append
>>>        (lambda ()
>>>          (setq-local actual-minibuffer-default-add-function
>>>                      minibuffer-default-add-function)
>>>          (setq-local minibuffer-default-add-function
>>>                      '(lambda () (setq minibuffer-default-add-done nil)
>>>                         (funcall actual-minibuffer-default-add-function)))
>>
>> Why use ' on `lambda`?
>>
>
> I think my finger slipped.
>
>>
>> BTW, rather than the above two `setq-local`s, I think you can write:
>>
>>    (add-function :before (local 'minibuffer-default-add-function)
>>                  (lambda () (setq minibuffer-default-add-done nil)))
>>
>
> Right.  I was writing that function as it if were for minibuffer.el, and was
> therefore avoiding advices.  But now I see that there are a couple of
> add-function in core anyway (it's not clear to me where the dividing line
> is).  So here's the final version:

The issue is not "don't use `(n)advice.el`", but "don't modify functions
on the sly".  And by "functions" this refers to those things stored in
the `symbol-function` slot of symbols in the global obarray.  This is
because code that does `(my-foo ...)` usually expects to execute the
code found in `(defun my-foo ...)` and not something else, and readers
of that code often make the same assumption, so it can make debugging
really nasty.

Changing a function with `fset` (or `cl-letf` or `defalias`) is worse
than using `advice-add`, for that reason: at least `advice-add` sets up
the help system such that `C-h f` will (hopefully) warn you about the
presence of an advice.

For variables containing functions (such as
`minibuffer-default-add-function`), there is no such expectation that
calling this function will run some known piece of code, on the
contrary: the whole point of the variable is to run difference pieces of
code in different contexts, so modifying the function is perfectly OK,
regardless whether you do it with `setq` or `add-function`.  Here I'd
use `add-function` because it's simpler.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

------- Original Message -------
On Friday, August 25th, 2023 at 1:36 AM, Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> wrote:

> > It is not, because the intention is on prefilling the minibuffer with
> > "alpha" rather than considering "alpha" as DEF.
> 
> Could you explain why this is important in your case?

There purpose of INITIAL has always been about prefilling the minibuffer.
No other 'completing-read' functionality can do such a thing.  DEF has
always served a different purpose.  For some reason that I cannot understand,
most of the communications I have try to persuade me to set INITIAL to nil.
INITIAL had a purpose, which under certain circumstances has implications
to the way COLLECTION is constructed and used.  Rather than fixing the 
difficulties for certain cases, the answer has always been the same, put 
INITIAL to nil and just don't use it, and use DEF if you want.  Even though
Default Settings and Minibuffer Prefilling result in two completely distinct  
behaviours.
 
> > It appears that the maintainers might not be prioritizing the initial
> > intention of prefilling the minibuffer. Currently, their emphasis
> > seems to be on encouraging developers to either utilize the "DEF"
> > approach or actively discourage the use of "INITIAL". But, I haven't
> > come across a clear and well-founded reasoning behind these shifts
> > in approach.
> 
> Not sure what you mean by "shifts". AFAIK this behavior has been with
> Emacs "forever": most minibuffers start empty (but with an associated
> default value) rather than starting with an initial value.

Right.  Because most minibuffers start empty (but with an associated
default value) rather than starting with an initial value, this particular
use of 'completing-read' in today considered to be the only way to use 
'completing-read'.  Something that is completely wrong because a particular
use case has now became dogma.  One cannot ever disregard the different feature
provided by INITIAL for those who want to use it. 
 
> This originally comes presumably from the absence of
> `transient-mark-mode` (or associated visual highlighting, both of which
> were introduced in Emacs-19) which means that it was annoying having to
> erase "alpha" before you can type "beta".
> 
> Starting with a non-empty minibuffer does happen occasionally, most
> importantly for `read-file-name` where we do expect that this initial
> input will very likely be a part of the name the user will end up
> typing, so its rare that the users need to erase it before they can type
> the file name they want.

Right.  Although it is customarily rare, INITIAL does actually have relevant
use cases.  It just happens that over time, additional use cases have cropped
up.  Rarity seams to have became a reason to discourage use of INITIAL.  Such
school of thought is seriously misguided.  
 
> > But this is only easily done only when collection is actually being
> > constructed in-place via the 'let' clause. But once COLLECTION
> > starts getting imported from somewhere else (via a call to same other
> > function for instance), your suggested solution is impossible
> > to achieve.
> 
> That's partly why I've asked about a concrete example showing the wider
> context :-) - Stefan

I am working on an emacs org library for archeological investigations where
field practitioners can insert specific org templates detailing the progress
of excavations and finds.  Each phase is categorised.

For instance

"Physical_Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"

And there exists a certain order.  It would be difficult to change that order on-the-fly 
just to make 'completing-read' happy. With each exists specific templates that practitioners
can introduce and elaborate.  Once certain aspects are completed, the previous categorisations
would be skipped, because they would no longer be relevant.  What gets shown is then directed
towards improving productivity, particularly when tight deadlines are imposed.  

Tight deadlines in archaeological field excavations arise from a variety of logistical and 
operational factors).  For instance, certain sites are threatened by construction or development
projects, with limited timeframes to excavate and document findings.  Some sites are at risk of
deterioration due to environmental factors (e.g. flooding, collapse), meaning that emergency 
excavations within short timeframes to salvage artifacts and information becomes necessary.

Such application should be concrete enough.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > It is not, because the intention is on prefilling the minibuffer with
>> > "alpha" rather than considering "alpha" as DEF.
>> 
>> Could you explain why this is important in your case?
>
> There purpose of INITIAL has always been about prefilling the minibuffer.
> No other 'completing-read' functionality can do such a thing.  DEF has
> always served a different purpose.  For some reason that I cannot understand,
> most of the communications I have try to persuade me to set INITIAL to nil.
> INITIAL had a purpose, which under certain circumstances has implications
> to the way COLLECTION is constructed and used.  Rather than fixing the 
> difficulties for certain cases, the answer has always been the same, put 
> INITIAL to nil and just don't use it, and use DEF if you want.  Even though
> Default Settings and Minibuffer Prefilling result in two completely distinct  
> behaviours.

My question is not about INITIAL-INPUT but about the behavior that the
user sees: why do you want the users of your code to see a minibuffer
that is prefilled rather than one whose content is initially empty?

That question is not rhetorical.  There can be many different perfectly
valid answers.  Depending on that answer, the best way to code it can be
quite different, tho.

>> That's partly why I've asked about a concrete example showing the wider
>> context :-) - Stefan
>
> I am working on an Emacs org library for archeological investigations where
> field practitioners can insert specific org templates detailing the progress
> of excavations and finds.  Each phase is categorised.
>
> For instance
>
> "Physical_Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"
>
> And there exists a certain order.  It would be difficult to change
> that order on-the-fly  just to make 'completing-read' happy. With each
> exists specific templates that practitioners can introduce and
> elaborate.  Once certain aspects are completed, the previous
> categorisations would be skipped, because they would no longer be
> relevant.  What gets shown is then directed towards improving
> productivity, particularly when tight deadlines are imposed.  

So, IIUC, you have a `completing-read` call asking them which template
to insert, and you want to order the set of completions based on
knowledge of the stage at which they are?

I suspect you'll want to use a COLLECTION that explicitly asks to not be
(re)sorted and which you "manually" re-order before the call, so that
the sort order you choose is obeyed not just by this specific cycling
you're using but also for users who rely on different UIs.

I don't see any part there that explains why the minibuffer needs to be
prefilled, but that is usually handled separately from the
completions anyway.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Friday, August 25th, 2023 at 4:45 AM, Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> wrote:


> > > > It is not, because the intention is on prefilling the minibuffer with
> > > > "alpha" rather than considering "alpha" as DEF.
> > > 
> > > Could you explain why this is important in your case?
> > 
> > There purpose of INITIAL has always been about prefilling the minibuffer.
> > No other 'completing-read' functionality can do such a thing. DEF has
> > always served a different purpose. For some reason that I cannot understand,
> > most of the communications I have try to persuade me to set INITIAL to nil.
> > INITIAL had a purpose, which under certain circumstances has implications
> > to the way COLLECTION is constructed and used. Rather than fixing the
> > difficulties for certain cases, the answer has always been the same, put
> > INITIAL to nil and just don't use it, and use DEF if you want. Even though
> > Default Settings and Minibuffer Prefilling result in two completely distinct
> > behaviours.
> 
> 
> My question is not about INITIAL-INPUT but about the behavior that the
> user sees: why do you want the users of your code to see a minibuffer
> that is prefilled rather than one whose content is initially empty?
> 
> That question is not rhetorical. There can be many different perfectly
> valid answers. Depending on that answer, the best way to code it can be
> quite different, tho.

I was planning something like for 'read-file-name' where one can expect 
that the initial input will very likely be the part the user will end up
typing, otherwise they can quickly do a simple cycling to see the possibilities.
 
> > > That's partly why I've asked about a concrete example showing the wider
> > > context :-) - Stefan
> > 
> > I am working on an Emacs org library for archeological investigations where
> > field practitioners can insert specific org templates detailing the progress
> > of excavations and finds. Each phase is categorised.
> > 
> > For instance
> > 
> > "Physical Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"
> > 
> > And there exists a certain order. It would be difficult to change
> > that order on-the-fly just to make 'completing-read' happy. With each
> > exists specific templates that practitioners can introduce and
> > elaborate. Once certain aspects are completed, the previous
> > categorisations would be skipped, because they would no longer be
> > relevant. What gets shown is then directed towards improving
> > productivity, particularly when tight deadlines are imposed.
> 
> So, IIUC, you have a `completing-read` call asking them which template
> to insert, and you want to order the set of completions based on
> knowledge of the stage at which they are?

No ordering actually happens, a particular element in collection is used
to prefill the minibuffer entry and consecutive elements in simple 
cycling continue through the next stages.

The ordering in 

"Physical Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"

is an order in the operational flow, and it would not be possible to apply any alphabetical
type of sorting.  

> I suspect you'll want to use a COLLECTION that explicitly asks to not be
> (re)sorted and which you "manually" re-order before the call, so that
> the sort order you choose is obeyed not just by this specific cycling
> you're using but also for users who rely on different UIs.

Actually, tho order is only obeyed for the specific cycling I am using, but
users have the possibility to use a different UI such as the usual manipulation
possibilities provided by completing read.
 
> I don't see any part there that explains why the minibuffer needs to be
> prefilled, but that is usually handled separately from the
> completions anyway. - Stefan

Then my conclusion is that the introduction of INITIAL for 'completing-read'
was a mistake.  That completing-read should only be about completion.  And that
prefilling the minibuffer should involve a separate function call.

From my interactions, it seems evident that we cannot get rid of INITIAL either,
and we are stuck with it.  And the reason for discouraging its use.  I am not sure
whether simple cycling and completion should be provided by a single function, 
although the capability of managing both would be very powerful.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> My question is not about INITIAL-INPUT but about the behavior that the
>> user sees: why do you want the users of your code to see a minibuffer
>> that is prefilled rather than one whose content is initially empty?
>
> I was planning something like for 'read-file-name' where one can expect 
> that the initial input will very likely be the part the user will end up
> typing,

You mean if the users will probably select "Chronological Dating",
you want to prefill the buffer so they don't need to type it and just
hit RET?

Using the DEFault arg gives you the same benefit without prefilling the
minibuffer, so I must be missing something.  What would be the advantage
for the users by prefilling the minibuffer with "Chronological Dating"?

>> So, IIUC, you have a `completing-read` call asking them which template
>> to insert, and you want to order the set of completions based on
>> knowledge of the stage at which they are?
> No ordering actually happens, a particular element in collection is used
> to prefill the minibuffer entry and consecutive elements in simple 
> cycling continue through the next stages.

Yes, the ordering I'm talking about is the order in the operational
flow refined such that if the likely next stage is "Composition and
Provenance", then you'll want to use:

    "Composition and Provenance" "Isotope Analysis" "Physical Analysis" "Chronological Dating"

That's what you want, right?

>> I suspect you'll want to use a COLLECTION that explicitly asks to not be
>> (re)sorted and which you "manually" re-order before the call, so that
>> the sort order you choose is obeyed not just by this specific cycling
>> you're using but also for users who rely on different UIs.
> Actually, tho order is only obeyed for the specific cycling I am using, but
> users have the possibility to use a different UI such as the usual manipulation
> possibilities provided by completing read.

You mean, if they use, say, `icomplete-mode` or `vertico-mode`, you'd
prefer that those UIs use an alphabetical ordering rather than the one
based on operational flow?

>> I don't see any part there that explains why the minibuffer needs to be
>> prefilled, but that is usually handled separately from the
>> completions anyway. - Stefan
> Then my conclusion is that the introduction of INITIAL for 'completing-read'
> was a mistake.  That completing-read should only be about completion.  And that
> prefilling the minibuffer should involve a separate function call.

FWIW, I tend to agree and if(?) we introduce a new API I'd replace this
INITIAL-INPUT argument with a function which could then do what we
usually do via the `minibuffer-with-setup-hook` hack.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

------- Original Message -------
On Friday, August 25th, 2023 at 7:35 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > > My question is not about INITIAL-INPUT but about the behavior that the
> > > user sees: why do you want the users of your code to see a minibuffer
> > > that is prefilled rather than one whose content is initially empty?
> > 
> > I was planning something like for 'read-file-name' where one can expect
> > that the initial input will very likely be the part the user will end up
> > typing,
> 
> 
> You mean if the users will probably select "Chronological Dating",
> you want to prefill the buffer so they don't need to type it and just
> hit RET?

Correct.
 
> Using the DEFault arg gives you the same benefit without prefilling the
> minibuffer, so I must be missing something. What would be the advantage
> for the users by prefilling the minibuffer with "Chronological Dating"?

Yes, without prefilling the minibuffer.  Prefilling the minibuffer is quicker
for users, giving them the possibility to see the current stage if they forget 
default key shortcuts.  The more we discuss this the more it seems the prefilling
the minbuffer was something we never wanted users to have. 

> > > So, IIUC, you have a `completing-read` call asking them which template
> > > to insert, and you want to order the set of completions based on
> > > knowledge of the stage at which they are?
> > > No ordering actually happens, a particular element in collection is used
> > > to prefill the minibuffer entry and consecutive elements in simple
> > > cycling continue through the next stages.
> 
> 
> Yes, the ordering I'm talking about is the order in the operational
> flow refined such that if the likely next stage is "Composition and
> Provenance", then you'll want to use:
> 
> "Composition and Provenance" "Isotope Analysis" "Physical Analysis" "Chronological Dating"
> 
> That's what you want, right?

Correct, but the actual collection might still be 

"Physical_Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"

Rather than applying modifications to it, I can just specify the start index.
 
> > > I suspect you'll want to use a COLLECTION that explicitly asks to not be
> > > (re)sorted and which you "manually" re-order before the call, so that
> > > the sort order you choose is obeyed not just by this specific cycling
> > > you're using but also for users who rely on different UIs.
> > > Actually, tho order is only obeyed for the specific cycling I am using, but
> > > users have the possibility to use a different UI such as the usual manipulation
> > > possibilities provided by completing read.
> 
> 
> You mean, if they use, say, `icomplete-mode` or `vertico-mode`, you'd
> prefer that those UIs use an alphabetical ordering rather than the one
> based on operational flow?

I would think that if they use vertico, there is a reason that is convenient
to them to use cempletion, if they are not employing simple cycling (through
repeated use of <down>). 
 
> > > I don't see any part there that explains why the minibuffer needs to be
> > > prefilled, but that is usually handled separately from the
> > > completions anyway. - Stefan
> > > Then my conclusion is that the introduction of INITIAL for 'completing-read'
> > > was a mistake. That completing-read should only be about completion. And that
> > > prefilling the minibuffer should involve a separate function call.
> 
> 
> FWIW, I tend to agree and if(?) we introduce a new API I'd replace this
> INITIAL-INPUT argument with a function which could then do what we
> usually do via the `minibuffer-with-setup-hook` hack.  - Stefan

Quite right, and help programmers from the need to go as low level as 
calling 'minibuffer-with-setup-hook' hacks.  The unfortunate thing is that
the inclusion of INITIAL makes people want to use it, as I did in certain 
circumstances.   Having to go through 'minibuffer-with-setup-hook' hacks
is not something one looks forward to do in the interactive clause of a 
function.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Using the DEFault arg gives you the same benefit without prefilling the
>> minibuffer, so I must be missing something. What would be the advantage
>> for the users by prefilling the minibuffer with "Chronological Dating"?
> Yes, without prefilling the minibuffer.  Prefilling the minibuffer is
> quicker for users, giving them the possibility to see the current
> stage if they forget  default key shortcuts.

The `format-prompt` thingy in my example code is there to show to the
users what the default will be.

> The more we discuss this the more it seems the prefilling
> the minbuffer was something we never wanted users to have.

Indeed, it was never intended for regular use.  It's only meant to be
used in specific cases like `read-file-name` where we do have something
to prefill that is almost always useful.

[ You used the term "dogma" earlier, but I like to think of Emacs
  design as being rather undogmatic in that we go through a lot of
  trouble to allow people to do even those things that we don't want
  them to do.  ]

>> > > So, IIUC, you have a `completing-read` call asking them which template
>> > > to insert, and you want to order the set of completions based on
>> > > knowledge of the stage at which they are?
>> > > No ordering actually happens, a particular element in collection is used
>> > > to prefill the minibuffer entry and consecutive elements in simple
>> > > cycling continue through the next stages.
>> 
>> 
>> Yes, the ordering I'm talking about is the order in the operational
>> flow refined such that if the likely next stage is "Composition and
>> Provenance", then you'll want to use:
>> 
>> "Composition and Provenance" "Isotope Analysis" "Physical Analysis" "Chronological Dating"
>> 
>> That's what you want, right?
>
> Correct, but the actual collection might still be 
>
> "Physical_Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"
>
> Rather than applying modifications to it, I can just specify the start index.

That's an implementation detail.

E.g. you can use

    (defconst my-phases
      '("Physical_Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"))

    (defun my-rotate (collection first)
      (let ((x (member first collection)))
        (if (not x) collection
          (let ((idx (- (length collection) (length x))))
            (append x (seq-take collection idx))))))

     [...]
     (let ((next-phase (my-guess-next-phase)))
       (completing-read (format-prompt "Phase" next-phase)
                        (my-rotate my-phases next-phase)
                        nil t nil nil next-phase))
     [...]

>> You mean, if they use, say, `icomplete-mode` or `vertico-mode`, you'd
>> prefer that those UIs use an alphabetical ordering rather than the one
>> based on operational flow?
>
> I would think that if they use vertico, there is a reason that is convenient
> to them to use cempletion, if they are not employing simple cycling (through
> repeated use of <down>). 

Both `icomplete-mode` and `vertico-mode` offer/encourage the use of
cycling (but their cycling code is completely different from the one
you're using so it doesn't obey `next-history-element`), so I expect
their users would also appreciate if the ordering of the completions is
chosen with the same care as what you do for the
`next-line-or-history-element` case.

> Quite right, and help programmers from the need to go as low level as
> calling 'minibuffer-with-setup-hook' hacks.  The unfortunate thing is that
> the inclusion of INITIAL makes people want to use it, as I did in certain
> circumstances.   Having to go through 'minibuffer-with-setup-hook' hacks
> is not something one looks forward to do in the interactive clause of a
> function.

+1

I guess we really should work on a replacement for `completing-read`, eh?


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Friday, August 25th, 2023 at 9:02 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > > Using the DEFault arg gives you the same benefit without prefilling the
> > > minibuffer, so I must be missing something. What would be the advantage
> > > for the users by prefilling the minibuffer with "Chronological Dating"?
> > > Yes, without prefilling the minibuffer. Prefilling the minibuffer is
> > > quicker for users, giving them the possibility to see the current
> > > stage if they forget default key shortcuts.
> 
> 
> The `format-prompt` thingy in my example code is there to show to the
> users what the default will be.
> 
> > The more we discuss this the more it seems the prefilling
> > the minbuffer was something we never wanted users to have.
> 
> 
> Indeed, it was never intended for regular use. It's only meant to be
> used in specific cases like `read-file-name` where we do have something
> to prefill that is almost always useful.
> 
> [ You used the term "dogma" earlier, but I like to think of Emacs
> design as being rather undogmatic in that we go through a lot of
> trouble to allow people to do even those things that we don't want
> them to do. ]
> 
> > > > > So, IIUC, you have a `completing-read` call asking them which template
> > > > > to insert, and you want to order the set of completions based on
> > > > > knowledge of the stage at which they are?
> > > > > No ordering actually happens, a particular element in collection is used
> > > > > to prefill the minibuffer entry and consecutive elements in simple
> > > > > cycling continue through the next stages.
> > > 
> > > Yes, the ordering I'm talking about is the order in the operational
> > > flow refined such that if the likely next stage is "Composition and
> > > Provenance", then you'll want to use:
> > > 
> > > "Composition and Provenance" "Isotope Analysis" "Physical Analysis" "Chronological Dating"
> > > 
> > > That's what you want, right?
> > 
> > Correct, but the actual collection might still be
> > 
> > "Physical_Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"
> > 
> > Rather than applying modifications to it, I can just specify the start index.
> 
> 
> That's an implementation detail.
> 
> E.g. you can use
> 
> (defconst my-phases
> '("Physical_Analysis" "Chronological Dating" "Composition and Provenance" "Isotope Analysis"))
> 
> (defun my-rotate (collection first)
> (let ((x (member first collection)))
> (if (not x) collection
> (let ((idx (- (length collection) (length x))))
> (append x (seq-take collection idx))))))
> 
> [...]
> (let ((next-phase (my-guess-next-phase)))
> (completing-read (format-prompt "Phase" next-phase)
> (my-rotate my-phases next-phase)
> nil t nil nil next-phase))
> [...]
> 
> > > You mean, if they use, say, `icomplete-mode` or `vertico-mode`, you'd
> > > prefer that those UIs use an alphabetical ordering rather than the one
> > > based on operational flow?
> > 
> > I would think that if they use vertico, there is a reason that is convenient
> > to them to use cempletion, if they are not employing simple cycling (through
> > repeated use of <down>).
> 
> 
> Both `icomplete-mode` and `vertico-mode` offer/encourage the use of
> cycling (but their cycling code is completely different from the one
> you're using so it doesn't obey `next-history-element`), so I expect
> their users would also appreciate if the ordering of the completions is
> chosen with the same care as what you do for the
> `next-line-or-history-element` case.
> 
> > Quite right, and help programmers from the need to go as low level as
> > calling 'minibuffer-with-setup-hook' hacks. The unfortunate thing is that
> > the inclusion of INITIAL makes people want to use it, as I did in certain
> > circumstances. Having to go through 'minibuffer-with-setup-hook' hacks
> > is not something one looks forward to do in the interactive clause of a
> > function.
> 
> 
> +1
> 
> I guess we really should work on a replacement for `completing-read`, eh? - Stefan

If we want to surpass the present conundrum, the path in clear.  completing-read has evolved
through incremental changes without considering the long-term implications.  And we continue
to accumulate additional parameters, flags, and branches to accommodate new functionalities. 
This has led to a serious lack of cohesion and a convoluted structure.  Making it difficult
to discern its primary purpose.  I have concluded that its evolution today requires it acquire
dependencies on other parts of its codebase.  Instead, we have started changing the documentation
to discourage this usage, that usage, and so on.  For short-term one can accept it, but robust design
is needed for long-term usage.  

FREDDY - Ahl. Would you mind telling me whose brain I put in?
IGOR - And you won't be angry?
FREDDY - I won't be angry.
IGOR - Abbey someone.
FREDDY - Abbey?? Abbey who?
IGOR - Abbey normal.
FREDDY - ABBEYNORMAL???
IGOR - I'm almost sure that was the name.

FREDDY grabbing Igor's throat

I put -- an abnormal brain -- into a seven-and-a-half
foot long, fourty-four inch wide GORILLA?

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Juri Linkov]

>> Heime, can you try that and tell us if it solves your problem?
>
> The behaviour is as it should be.  Although we should keep 'start' to correspond to
> collection index which customarily starts from value 0.
>
> Although I can use the function provided, I rather have all this incorporated in
> 'completing-read'.  Because 'completing-read' would benefit from it.  Particularly
> now that its use is being extended.  This taking into account that my use case
> is more aligned with its original operation, except that certain aspects which I have
> shown to be important were not recognised at the time.  After some focused attention,
> Stefan concurred how I have been making sense, particularly after seeing how `completing-read`
> actually behaves during the form of cycling I discussed at some length.

Why couldn't you explicitly provide a list of default values
without the initial value?  This solves your problem:

  (let ((cseq '("alpha" "beta" "gamma" "delta" "epsilon" "zeta" "eta")))
    (completing-read "Grapheme: " cseq nil t "alpha" nil (cdr cseq)))

PS: Currently this can't be done automatically because 'completing-read-default'
doesn't set a buffer-local 'minibuffer-initial-input' from its argument
'initial-input'.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Gregory Heytings]

>
> The issue is not "don't use `(n)advice.el`", but "don't modify functions 
> on the sly".  And by "functions" this refers to those things stored in 
> the `symbol-function` slot of symbols in the global obarray.  This is 
> because code that does `(my-foo ...)` usually expects to execute the 
> code found in `(defun my-foo ...)` and not something else, and readers 
> of that code often make the same assumption, so it can make debugging 
> really nasty.
>
> Changing a function with `fset` (or `cl-letf` or `defalias`) is worse 
> than using `advice-add`, for that reason: at least `advice-add` sets up 
> the help system such that `C-h f` will (hopefully) warn you about the 
> presence of an advice.
>
> For variables containing functions (such as 
> `minibuffer-default-add-function`), there is no such expectation that 
> calling this function will run some known piece of code, on the 
> contrary: the whole point of the variable is to run difference pieces of 
> code in different contexts, so modifying the function is perfectly OK, 
> regardless whether you do it with `setq` or `add-function`.  Here I'd 
> use `add-function` because it's simpler.
>

Thanks, that clarifies that question!  Perhaps a paragraph along those 
lines could be added to "(elisp)Advising Functions"?

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Gregory Heytings]

>
> I guess we really should work on a replacement for `completing-read`, 
> eh?
>

Why should we replace it?  Can't we just add another function, alongside 
completing-read-default and completing-read-multiple, using the short POC 
code I sent as a starting point?

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> I guess we really should work on a replacement for `completing-read`, eh?
> Why should we replace it?  Can't we just add another function, alongside
> completing-read-default and completing-read-multiple, using the short POC
> code I sent as a starting point?

That's what I meant by "replacement" (I'd expect new code to use that new
function instead of the other, so one "replaces" the other).


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Sunday, August 27th, 2023 at 2:27 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > > I guess we really should work on a replacement for `completing-read`, eh?
> > > Why should we replace it? Can't we just add another function, alongside
> > > completing-read-default and completing-read-multiple, using the short POC
> > > code I sent as a starting point?
> 
> 
> That's what I meant by "replacement" (I'd expect new code to use that new
> function instead of the other, so one "replaces" the other). Stefan

Additionally, let's have minibuffer prefilling as part of the design without
any discouragement or encouragement regarding its use.  I like it, others 
don't like it.   

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Additionally, let's have minibuffer prefilling as part of the design
> without any discouragement or encouragement regarding its use.
> I like it, others don't like it.

This last sentence is key: it should not be a choice of the caller of the
function, but should depend on the configuration of the UI instead.

Using DEFault makes this customization possible.  Using INITIAL-INPUT
does not.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Monday, August 28th, 2023 at 2:40 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > Additionally, let's have minibuffer prefilling as part of the design
> > without any discouragement or encouragement regarding its use.
> > I like it, others don't like it.
> 
> 
> This last sentence is key: it should not be a choice of the caller of the
> function, but should depend on the configuration of the UI instead.
> 
> Using DEFault makes this customization possible. Using INITIAL-INPUT
> does not. - Stefan

Suppose I want to prefill the buffer with something, I should be allowed 
to do that.  The default only shows up when instructed, so not really
an automatic prefill of the minibuffer.

Not a choice of the caller of the function.  It is the programmer who is 
calling the function and who wants it to behave a certain.  I find it a
big problem that when I mention minibuffer prefilling, the maintainers
reject such possibility.  It basically means that I get the UI behind 
by back trying to hamstring me on what I can do and what I cannot do.

Suppose I want minibuffer prefilling, give me the procedure that allows me
to do that, one that is not complicated in a way that dissuades me from 
implementing such a thing.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Suppose I want to prefill the buffer with something, I should be allowed 
> to do that.

Of course, and we do allow that.
But you said:

    I like it, others don't like it.

so what happens when "others" use your code?
And what happens when you use others' code?

Neither is happy if the choice is made by the coder.  OTOH if the choice
is made by UI customization, then the coder doesn't need to worry about
it and the both you and others get the behavior they like.

> The default only shows up when instructed, so not really
> an automatic prefill of the minibuffer.

Whether the default only shows up when instructed is a UI issue, which
can/should be under control of a UI customization.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Monday, August 28th, 2023 at 4:26 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > Suppose I want to prefill the buffer with something, I should be allowed
> > to do that.
> 
> 
> Of course, and we do allow that.
> But you said:
> 
> I like it, others don't like it.
> 
> so what happens when "others" use your code?
> And what happens when you use others' code?
> 
> Neither is happy if the choice is made by the coder. OTOH if the choice
> is made by UI customization, then the coder doesn't need to worry about
> it and the both you and others get the behavior they like.
> 
> > The default only shows up when instructed, so not really
> > an automatic prefill of the minibuffer.
> 
> 
> Whether the default only shows up when instructed is a UI issue, which
> can/should be under control of a UI customization. - Stefan

I would need an example.  Suppose one changes the UI customization, would that
apply to all calls of that function ?  I would want prefilling occasionally.
Would that be possible ?

All these UI accept any entries by the user.  I am of the school of thought that
doing so automatically is also a valid operation.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Monday, August 28th, 2023 at 4:26 AM, Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> wrote:


> > Suppose I want to prefill the buffer with something, I should be allowed
> > to do that.
> 
> 
> Of course, and we do allow that.
> But you said:
> 
> I like it, others don't like it.
> 
> so what happens when "others" use your code?
> And what happens when you use others' code?
> 
> Neither is happy if the choice is made by the coder. OTOH if the choice
> is made by UI customization, then the coder doesn't need to worry about
> it and the both you and others get the behavior they like.
> 
> > The default only shows up when instructed, so not really
> > an automatic prefill of the minibuffer.
> 
> 
> Whether the default only shows up when instructed is a UI issue, which
> can/should be under control of a UI customization. > Stefan
 
Could one control the UI customization programmatically as well ?

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Whether the default only shows up when instructed is a UI issue, which
>> can/should be under control of a UI customization. - Stefan
> I would need an example.  Suppose one changes the UI customization, would that
> apply to all calls of that function ?  I would want prefilling occasionally.
> Would that be possible ?

There are some cases, such as `read-file-name` where we do want some
prefilling, so of course it's possible, but it has to be linked to some
specific property of the thing we're requesting from the user and how it
relates to the string with which we want to prefill the buffer.

So it goes back to the question of why you want this specific call to
`completing-read` to be prefilled.  The answer shouldn't be "because
I prefer it this way" (that answer should be satisfied instead by
a customization setting since it may not apply to other users of that
code).

E.g. in the case of `read-file-name` it's because in 99% of the cases
the end result wants to include all or most of that prefilled string.

> All these UI accept any entries by the user.

I don't know what you mean by that.

> I am of the school of thought that doing so automatically is also
> a valid operation.

And I didn't understand this part either.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Whether the default only shows up when instructed is a UI issue,
>> which can/should be under control of a UI customization.
> Could one control the UI customization programmatically as well ?

Don't know what you mean by that.
The init file is executed and hence the way the users control their
customization is programmatic, yes, but I suspect that's not what
you mean.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Monday, August 28th, 2023 at 6:02 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > > Whether the default only shows up when instructed is a UI issue,
> > > which can/should be under control of a UI customization.
> > > Could one control the UI customization programmatically as well ?
> 
> 
> Don't know what you mean by that.
> The init file is executed and hence the way the users control their
> customization is programmatic, yes, but I suspect that's not what
> you mean. > Stefan

Users might want to use minibuffer reading with possibility of prefilling.
Not only from the Emacs Customisation Menu as usually recommended.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Monday, August 28th, 2023 at 6:01 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > > Whether the default only shows up when instructed is a UI issue, which
> > > can/should be under control of a UI customization. - Stefan

> > > I would need an example. Suppose one changes the UI customization, would that
> > > apply to all calls of that function ? I would want prefilling occasionally.
> > > Would that be possible ?
 
> There are some cases, such as `read-file-name` where we do want some
> prefilling, so of course it's possible, but it has to be linked to some
> specific property of the thing we're requesting from the user and how it
> relates to the string with which we want to prefill the buffer.

I am proposing a broader use.  I meant an example of how a programmer can
include minibuffer prefilling.   It seems that the functionality exists
but not well known or understood.  If minibuffer prefilling has been possible
for us in `read-file-name`, then the possibility should also be provided to 
programmers and users devising their own functionality.   
 
 
> So it goes back to the question of why you want this specific call to
> `completing-read` to be prefilled. The answer shouldn't be "because
> I prefer it this way" (that answer should be satisfied instead by
> a customization setting since it may not apply to other users of that
> code).
> 
> E.g. in the case of `read-file-name` it's because in 99% of the cases
> the end result wants to include all or most of that prefilled string.
> 
> > All these UI accept any entries by the user.

Customarily all minibuffer input from the user is allowed to be entered.
I claim automatic prefilling can be considered a form of user input, if 
they code their own function with prefilling in mind.
 
> I don't know what you mean by that.
> 
> > I am of the school of thought that doing so automatically is also
> > a valid operation.

> And I didn't understand this part either. > Stefan

There are two audiences in all this.  Programmers who write libraries
and users who write functions for their own use.  The thought is that
if `read-file-name` has capability of minibuffer prefilling, then minibuffer
prefilling capability should be available for them to use without having
them convince anybody why they want it to happen.  

I consider the statement "I prefer it this way" legitimate, because in emacs 
users have the capability to add functionality for their personal use, and their
decisions would not affect other users of that code.

Did some more thinking about all this and can see the possibility of INITIAL
to become much broader.  For maintainers of libraries for public use, the code 
could need some elaboration, but not much for personal use cases.
be

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Monday, August 28th, 2023 at 6:02 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > > Whether the default only shows up when instructed is a UI issue,
> > > which can/should be under control of a UI customization.
> > > Could one control the UI customization programmatically as well ?
> 
> 
> Don't know what you mean by that.

> The init file is executed and hence the way the users control their
> customization is programmatic, yes, but I suspect that's not what
> you mean. > Stefan

It might be what I mean.  The users should have the ability to enable the
minibuffer prefilling for a function they write, but setting some enabling
feature using elisp code. 

After much discussion, I would need to test your ideas out in practice
and provide you with feedback.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> There are two audiences in all this.  Programmers who write libraries
> and users who write functions for their own use.  The thought is that
> if `read-file-name` has capability of minibuffer prefilling, then minibuffer
> prefilling capability should be available for them to use without having
> them convince anybody why they want it to happen.

As I said, as a matter of principle and general design, Emacs goes to
great lengths to make sure users can do even what we *don't* want them
to do (except make our code proprietary).  So of course it is and will
remain possible for users to write code that prefills the minibuffer.

The question is how to make an API that encourages coders to write code in
such a way that the behavior is fairly homogeneous between packages and
such that users can customize the behavior to be the way they like it.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Monday, August 28th, 2023 at 9:48 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > There are two audiences in all this. Programmers who write libraries
> > and users who write functions for their own use. The thought is that
> > if `read-file-name` has capability of minibuffer prefilling, then minibuffer
> > prefilling capability should be available for them to use without having
> > them convince anybody why they want it to happen.
> 
> 
> As I said, as a matter of principle and general design, Emacs goes to
> great lengths to make sure users can do even what we don't want them
> to do (except make our code proprietary). So of course it is and will
> remain possible for users to write code that prefills the minibuffer.
> 
> The question is how to make an API that encourages coders to write code in
> such a way that the behavior is fairly homogeneous between packages and
> such that users can customize the behavior to be the way they like it. > Stefan

Yes, the problematic part is how to make the API without making life too difficult
for those who want to digress from our point of view.  It might happen they find a 
good case for digressing or figure out whether our encouragement makes sense to them.

For using 'completing-read' without INITIAL as argument, do you have a plan of how 
minibuffer prefilling could be activated and set for use within some specific user
function? 

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> For using 'completing-read' without INITIAL as argument, do you have a plan of how 
> minibuffer prefilling could be activated and set for use within some specific user
> function? 

I suggested to accept a `setup-function` argument, which would replace the
`minibuffer-with-setup-hook` hack, so instead of

    (completing-read MYPROMPT ... MYINIT-INPUT ...)

with the new API you could do

    (mynew-completing-read MYPROMPT ...
                           (lambda () (insert "MYINIT-INPUT"))
                           ...)

-- Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

------- Original Message -------
On Monday, August 28th, 2023 at 3:12 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > For using 'completing-read' without INITIAL as argument, do you have a plan of how
> > minibuffer prefilling could be activated and set for use within some specific user
> > function?
> 
> 
> I suggested to accept a `setup-function` argument, which would replace the
> `minibuffer-with-setup-hook` hack, so instead of
> 
> (completing-read MYPROMPT ... MYINIT-INPUT ...)
> 
> with the new API you could do
> 
> (mynew-completing-read MYPROMPT ...
> (lambda () (insert "MYINIT-INPUT"))
> ...)
> 
> -- Stefan

The function "mynew-completing-read", is that something users have to write and use
or would it be provided by emacs release ?  Why the preference for a function, would
its use enhance what to do with it.  Perhaps the different UI would call a different
function?

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> The function "mynew-completing-read", is that something users have to write and use
> or would it be provided by emacs release ?

"mynew-completing-read" stands for the name of the replacement of `completing-read`.

> Why the preference for a function,

What would you use instead?

> would its use enhance what to do with it.

Sorry, I don't know what you mean by that.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

------- Original Message -------
On Tuesday, August 29th, 2023 at 12:44 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > The function "mynew-completing-read", is that something users have to write and use
> > or would it be provided by emacs release ?
> 
> "mynew-completing-read" stands for the name of the replacement of `completing-read`.
> 
> > Why the preference for a function,
> 
> What would you use instead?
> 
> > would its use enhance what to do with it.
> 
> Sorry, I don't know what you mean by that. > Stefan

Could the function body be used for more capability than (insert "MYINIT-INPUT") ?

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > would its use enhance what to do with it.
>> Sorry, I don't know what you mean by that. > Stefan
> Could the function body be used for more capability than (insert "MYINIT-INPUT") ?

Of course, it can move point (just like INITIAL-INPUT lets you),
and it can be used for whatever `minibuffer-with-setup-hook` is used.


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Heime via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Sent with Proton Mail secure email.

------- Original Message -------
On Tuesday, August 29th, 2023 at 1:04 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:


> > > > would its use enhance what to do with it.
> > > > Sorry, I don't know what you mean by that. > Stefan
> > > > Could the function body be used for more capability than (insert "MYINIT-INPUT") ?
> 
> 
> Of course, it can move point (just like INITIAL-INPUT lets you),
> and it can be used for whatever `minibuffer-with-setup-hook` is used. > Stefan

Then I will check the new implementations out, and play with them a little bit.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Eli Zaretskii]

> Date: Sat, 26 Aug 2023 08:06:39 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: Heime <heimeborgia@protonmail.com>, Eli Zaretskii <eliz@gnu.org>, 
>     65459@debbugs.gnu.org
> 
> 
> >
> > The issue is not "don't use `(n)advice.el`", but "don't modify functions 
> > on the sly".  And by "functions" this refers to those things stored in 
> > the `symbol-function` slot of symbols in the global obarray.  This is 
> > because code that does `(my-foo ...)` usually expects to execute the 
> > code found in `(defun my-foo ...)` and not something else, and readers 
> > of that code often make the same assumption, so it can make debugging 
> > really nasty.
> >
> > Changing a function with `fset` (or `cl-letf` or `defalias`) is worse 
> > than using `advice-add`, for that reason: at least `advice-add` sets up 
> > the help system such that `C-h f` will (hopefully) warn you about the 
> > presence of an advice.
> >
> > For variables containing functions (such as 
> > `minibuffer-default-add-function`), there is no such expectation that 
> > calling this function will run some known piece of code, on the 
> > contrary: the whole point of the variable is to run difference pieces of 
> > code in different contexts, so modifying the function is perfectly OK, 
> > regardless whether you do it with `setq` or `add-function`.  Here I'd 
> > use `add-function` because it's simpler.
> >
> 
> Thanks, that clarifies that question!  Perhaps a paragraph along those 
> lines could be added to "(elisp)Advising Functions"?

Feel free to suggest a clarification for the manual, and thanks in
advance.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Thanks, that clarifies that question!  Perhaps a paragraph along those lines
> could be added to "(elisp)Advising Functions"?

What about the following?


        Stefan


diff --git a/doc/lispref/functions.texi b/doc/lispref/functions.texi
index 53525e6b386..236b823e7e6 100644
--- a/doc/lispref/functions.texi
+++ b/doc/lispref/functions.texi
@@ -1994,9 +1994,16 @@ Advising Named Functions
 debugging does not notice or remember that the function has been modified
 by advice.
 
-  For these reasons, advice should be reserved for the cases where you
-cannot modify a function's behavior in any other way.  If it is
-possible to do the same thing via a hook, that is preferable
+  Note that the problems are not due to advice per se, but to the act
+of modifying a named function.  It is even more problematic to modify
+a named function via lower-level primitives like @code{fset},
+@code{defalias}, or @code{cl-letf}.  From that point of view, advice
+is the better way to modify a named function because it keeps track of
+the modifications, so they can be listed and undone.
+
+  Modifying a named function should be reserved for
+the cases where you cannot modify Emacs' behavior in any other way.
+If it is possible to do the same thing via a hook, that is preferable
 (@pxref{Hooks}).  If you simply want to change what a particular key
 does, it may be better to write a new command, and remap the old
 command's key bindings to the new one (@pxref{Remapping Commands}).

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Kangas]

Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

>> Thanks, that clarifies that question!  Perhaps a paragraph along those lines
>> could be added to "(elisp)Advising Functions"?
>
> What about the following?

I think it's a good addition.

> diff --git a/doc/lispref/functions.texi b/doc/lispref/functions.texi
> index 53525e6b386..236b823e7e6 100644
> --- a/doc/lispref/functions.texi
> +++ b/doc/lispref/functions.texi
> @@ -1994,9 +1994,16 @@ Advising Named Functions
>  debugging does not notice or remember that the function has been modified
>  by advice.
>
> -  For these reasons, advice should be reserved for the cases where you
> -cannot modify a function's behavior in any other way.  If it is
> -possible to do the same thing via a hook, that is preferable
> +  Note that the problems are not due to advice per se, but to the act
> +of modifying a named function.  It is even more problematic to modify
> +a named function via lower-level primitives like @code{fset},
> +@code{defalias}, or @code{cl-letf}.  From that point of view, advice
> +is the better way to modify a named function because it keeps track of
> +the modifications, so they can be listed and undone.
> +
> +  Modifying a named function should be reserved for
> +the cases where you cannot modify Emacs' behavior in any other way.
> +If it is possible to do the same thing via a hook, that is preferable
>  (@pxref{Hooks}).  If you simply want to change what a particular key
>  does, it may be better to write a new command, and remap the old
>  command's key bindings to the new one (@pxref{Remapping Commands}).

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Heime <heimeborgia@protonmail.com>,  Eli Zaretskii <eliz@gnu.org>,
>   65459@debbugs.gnu.org
> Date: Mon, 04 Sep 2023 17:35:43 -0400
> 
> > Thanks, that clarifies that question!  Perhaps a paragraph along those lines
> > could be added to "(elisp)Advising Functions"?
> 
> What about the following?

LGTM, thanks.

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > Thanks, that clarifies that question!  Perhaps a paragraph along those lines
>> > could be added to "(elisp)Advising Functions"?
>> What about the following?
> LGTM, thanks.

[ I think it's the first time I send a manual change and you don't find
  anything to improve in it.
  Maybe you were just busy/tired, but I prefer to think that
  I'm improving.  ]

Thanks, pushed,


        Stefan

bug#65459: completing-read INITIAL-VALUE unaware of COLLECTION and REQUIRE-MATCH

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: gregory@heytings.org,  heimeborgia@protonmail.com,  65459@debbugs.gnu.org
> Date: Tue, 05 Sep 2023 08:59:16 -0400
> 
> [ I think it's the first time I send a manual change and you don't find
>   anything to improve in it.
>   Maybe you were just busy/tired, but I prefer to think that
>   I'm improving.  ]

I wasn't tired, FWIW.

> Thanks, pushed,

Thanks.