bug#43572: Feature request: make it possible to choose whether the first lines of the minibuffer should be displayed instead of the last ones

[Gregory Heytings via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

>> In Emacs 27.1, the mini-window displays the last lines of the 
>> minibuffer.
>>
>> This is, in general, the desired behavior, but in some cases it is not.
>>
>> One case in which this behavior is not desirable is when completion 
>> candidates are displayed with an overlay at the end of the buffer. 
>> When this overlay is taller than max-mini-window-height, the prompt and 
>> the user input so far disappear.  A simple example: M-: (setq 
>> max-mini-window-height 1), M-x icomplete-mode, M-x a.
>
> Actually, on the current master this example does show the "M-x a" part.
>

Yes, as I explain just below.  It's an improvement that improves most, but 
not all, cases.

>> This feature request follows the discussion in bug#43519.  The change 
>> proposed there by Eli Zaretskii improves the behavior w.r.t. Emacs 
>> 27.1, but it is still suboptimal to display completion candidates in a 
>> user-friendly way.  For example:
>>
>> Find file: <user input>|
>> <completion candidates>
>>
>> (where | represents the cursor) will become:
>>
>> <user input>|
>> <completion candidates>
>>
>> when the user input becomes larger than a line.  That is, the "Find 
>> file:" prompt and the user input on the first line will disappear.
>
> I suggest to show a recipe for this, because the few I tried failed to 
> produce the described effect (with the current master).  Maybe I'm 
> missing something.
>

I just tested this again with current master, and actually the result is 
slighly worse than what I though, when the prompt and user input becomes 
larger than a line you don't see:

<user input>|
<completion candidates>

but:

|<completion candidates>

(IOW the characters at the beginning of the Nth line, with N > 1, 
disappear.)

Again it's difficult to give a simple recipe, because it depends on the 
width of your Emacs frame and of the size of the font used in the 
mini-window.  The simplest recipe I can think of is:

1. create a "long enough" directory name, with say ten subdirectories
2. emacs -Q
3. make the frame width "small enough"
4. M-: (setq max-mini-window-height 5)
5. M-x icomplete-mode
6. M-: (setq icomplete-separator "\n")
7. C-x C-f and enter the "long enough" directory name

What you will see at this point is:

|{<completion candidate 1>
<completion candidate 2>
...
<completion candidate 5>

>
> How will Lisp programs decide when to set this flag and when not to set 
> it?  What would be the criteria?
>

The criteria is simply: should the prompt and user input be displayed? 
IOW, is what Stefan called the "real content" (prompt and user input so 
far) more important that the overlay (which displays completion candidates 
but is merely an unnecessary help for the user)?

Programs such as icomplete and ido, for example, would most likely want to 
set this flag.

(A more precise way to formulate that criteria would be: should the prompt 
and user input be displayed, unless it is impossible to display them?)

>
> If you are saying that any Lisp program that reads from the minibuffer 
> will want that, then (assuming that others agree), it would be better to 
> do this automatically in the display code.
>

This is not what I'm saying, and I would not dare to make such a general 
judgment.  I only claim that it is better to make this possible.

There is at least one case where I think it is better not to do this 
automatically.  As Stefan indicated in bug#43519, with M-:, when you input 
data, the current behavior is to always have point on the last line of the 
minibuffer.  Doing this automatically (that is, unconditionally) would 
have the consequence that when point reaches the last line of the 
minibuffer (that is, the max-mini-window-height's line), the mini-buffer 
would be recentered, and the topmost lines would be hidden.  This happens 
because the default value of scroll-conservatively is 0; when it is set to 
101 it does not happen anymore.

This is just one case, there are possibly many other cases.  But IMO, the 
mini-buffer is so central to Emacs, and the current behavior is so old 
(twenty years), that I believe changing it requires a lot of care.  This 
could be done in small steps:

1. first with this patch (or if you want with the opposite patch: a 
variable start-display-at-end-of-minibuffer reset to t whenever the 
mini-buffer is entered), which would make it possible to everyone to try 
to set that variable to its non-default value to see if undesirable 
behaviors arise,

2. then by changing the default value to its opposite, say in Emacs 29, if 
it became clear enough that the new behavior does not give rise to any 
undesiable consequences,

3. and finally, in Emacs 3X, by removing that variable.

>
> Binding the variable inside the minibuffer-setup-hook will affect all 
> the subsequent calls to resize_mini_window, until the next call to 
> read-from-minibuffer resets it, which may not be what the Lisp program 
> wants, and could have unintended consequences.
>

I can't think of such unintended consequences.  In the use case of 
displaying completion candidates, this (the fact that it affects all 
successive calls to resize_mini_window) is indeed what is wanted.