BIKESHED: completion faces

BIKESHED: completion faces

[Stefan Monnier]

In Emacs-26, completion faces looked like the following:

- the "common" part shown like the default face.
- the "first-difference" shown in bold.
- everything else uses the default face.

The "common" part is the part applied to the characters that are also
found in the current minibuffer.  E.g. when completing

   M-x ne-li ?

the "ne" and "-li" chars in the *Completions* are the "common" part.

With the basic, prefix completion, the "common" part is not very
important and is separated from the rest at the "first-difference", so
the default faces make a fair bit of sense there.

But for other completion styles such as `substring`,
`partial-completion`, and even more so for `flex`, it's not always
immediately obvious how the minibuffer contents relate to the possible
completions displayed in *Completions*.

For this reason, it is common in other completion systems to
highlight the "common" part somehow (e.g. underline, bold, ...).

I think Emacs's defaults should be changed so that the "common" part
(which uses the `completions-common-part` face) is highlighted somehow.

So, what should we go for:
- bold (but then we need to change the `completions-first-difference`
  face to keep it different, e.g. underlined)?
- underlined (tho I don't like underlined text very much)?
- some foreground color?
- some background color?
- something else?


        Stefan

Re: BIKESHED: completion faces

[João Távora]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> For this reason, it is common in other completion systems to
> highlight the "common" part somehow (e.g. underline, bold, ...).
>
> So, what should we go for:
> - bold (but then we need to change the `completions-first-difference`
>   face to keep it different, e.g. underlined)?

+1 for this option.  Tho any option that uses a prominent face for the
"common" part is good.

João

Re: BIKESHED: completion faces

[Juri Linkov]

> So, what should we go for:
> - bold (but then we need to change the `completions-first-difference`
>   face to keep it different, e.g. underlined)?

bold in completions-first-difference helps to immediately see the
next character to type to narrow completions further.

> - underlined (tho I don't like underlined text very much)?
> - some foreground color?

For several years I've been using completions-common-part
inherited from the 'shadow' face, and it works pretty well
for this purpose.

> - some background color?
> - something else?

Re: BIKESHED: completion faces

[João Távora]

Juri Linkov <juri@linkov.net> writes:

> bold in completions-first-difference helps to immediately see the
> next character to type to narrow completions further.

<mild bikeshedding starts>

Yes, but some questions, Juri:

* wouldn't any other face, say "underline", serve the same purpose?

* would you not be equally and efficiently informed of such facts if
  completions-common-part were _more_ prominent and
  completions-first-difference was _less_ prominent?

* In completion styles other than "basic", there are many other
  characters, besides the one marked with completions-first-difference,
  that you type to narrow completion further, right?

>> - underlined (tho I don't like underlined text very much)?
>> - some foreground color?
> For several years I've been using completions-common-part
> inherited from the 'shadow' face, and it works pretty well
> for this purpose.

Do you use mainly the "basic" completion style? I.e. do you customize
"completion-styles"?

Thanks,
João

Re: BIKESHED: completion faces

[Juri Linkov]

>> bold in completions-first-difference helps to immediately see the
>> next character to type to narrow completions further.
>
> <mild bikeshedding starts>
>
> Yes, but some questions, Juri:
>
> * wouldn't any other face, say "underline", serve the same purpose?

underline is less noticeable than bold, when used on a single character
with completions-first-difference in the "basic" completion style.

> * would you not be equally and efficiently informed of such facts if
>   completions-common-part were _more_ prominent and
>   completions-first-difference was _less_ prominent?

In the "basic" completion style completions-first-difference
needs to be more prominent since it's more important to indicate
the next character to type.

> * In completion styles other than "basic", there are many other
>   characters, besides the one marked with completions-first-difference,
>   that you type to narrow completion further, right?

Other completion styles don't highlight completions-first-difference
at all.  I'm not sure if only the "basic" completion style highlights
completions-first-difference.

Is it possible to use bold for completions-first-difference only
in the "basic" completion style, but for other completion styles
to use bold for completions-common-part?

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 2050 bytes --]

On Mon, Oct 28, 2019 at 11:01 PM Juri Linkov <juri@linkov.net> wrote:
>
> >> bold in completions-first-difference helps to immediately see the
> >> next character to type to narrow completions further.
> >
> > <mild bikeshedding starts>
> >
> > Yes, but some questions, Juri:
> >
> > * wouldn't any other face, say "underline", serve the same purpose?
>
> underline is less noticeable than bold, when used on a single character
> with completions-first-difference in the "basic" completion style.

> > * would you not be equally and efficiently informed of such facts if
> >   completions-common-part were _more_ prominent and
> >   completions-first-difference was _less_ prominent?
>
> In the "basic" completion style completions-first-difference
> needs to be more prominent since it's more important to indicate
> the next character to type.

I understand. But if completion-common-part is super-prominent, you get
more or less the same, right?  It's also very easy to see the "next"
character
to type.

> Other completion styles don't highlight completions-first-difference
> at all.  I'm not sure if only the "basic" completion style highlights
> completions-first-difference.

Eh.  I removed it recently, without asking anyone, waiting for
someone to complain. :-)

Are you complaining? :-)  If you are, I'll revert that bit. If you aren't
I probably should revert it too...

Anyway, I don't think it makes a lot of sense in those styles,
unless you are editing inside the completion string,
which is relatively rare.

> Is it possible to use bold for completions-first-difference only
> in the "basic" completion style, but for other completion styles
> to use bold for completions-common-part?

That would make some sense, yes.  Perhaps we just need better names
for the faces. Perhaps a new alias "completion-important-bits-for-style"
for the current completion-first-difference would do fine.  We probably
need a better name, tho... "completion-style-hint"? "completion-emphasis"?

João

[-- Attachment #2: Type: text/html, Size: 2730 bytes --]

Re: BIKESHED: completion faces

[Stefan Monnier]

>>> bold in completions-first-difference helps to immediately see the
>>> next character to type to narrow completions further.
>> <mild bikeshedding starts>
>> Yes, but some questions, Juri:
>> * wouldn't any other face, say "underline", serve the same purpose?
> underline is less noticeable than bold, when used on a single character
> with completions-first-difference in the "basic" completion style.

I kind of like the underline for first-difference because it is similar
to the underline used in some toolkit's menus to indicate which
key-shortcut to use to select this entry.

>> * In completion styles other than "basic", there are many other
>>   characters, besides the one marked with completions-first-difference,
>>   that you type to narrow completion further, right?
> Other completion styles don't highlight completions-first-difference
> at all.

Apparently it's now the case on `master` but it's a (hopefully
temporary) regression: it definitely worked for `substring` and
`partial-completion` (and I see no reason why it can't work for
`flex` as well) and it makes just as much sense there as for
`basic` completion, if you ask me.

I think it's the `completion-common-part` highlighting which is
influenced by the completion style: its highlighting is much less
important/useful for `basic` completion and than for more complex
completion styles.

> Is it possible to use bold for completions-first-difference only
> in the "basic" completion style, but for other completion styles
> to use bold for completions-common-part?

I don't think we could/should make the faces's aspect depend on the
completion style used.  But the `basic` completion style could use
different faces than `substring`, `flex`, `initials`, and
`partial-completion`, yes.

This said, since the default completion uses sometimes `basic` and
sometimes `partial-completion`, I think it would be odd to have the
*Completions* use different highlighting on a case-by-case basis.


        Stefan

Re: BIKESHED: completion faces

[Juri Linkov]

>> Other completion styles don't highlight completions-first-difference
>> at all.  I'm not sure if only the "basic" completion style highlights
>> completions-first-difference.
>
> Eh.  I removed it recently, without asking anyone, waiting for
> someone to complain. :-)
>
> Are you complaining? :-)  If you are, I'll revert that bit. If you aren't
> I probably should revert it too...

No, I'm not complaining.  Wait for real complaints from someone
who will really get angry at your changes.  Or you could revert
preemptively to avoid such angry complaints. :-)

Or implement a solution that satisfies everyone: since in completion
styles other than "basic", there are many characters that you can type
to narrow completion further, is it possible to mark all them with
completions-first-difference?

Re: BIKESHED: completion faces

[Juri Linkov]

>>>> bold in completions-first-difference helps to immediately see the
>>>> next character to type to narrow completions further.
>>> <mild bikeshedding starts>
>>> Yes, but some questions, Juri:
>>> * wouldn't any other face, say "underline", serve the same purpose?
>> underline is less noticeable than bold, when used on a single character
>> with completions-first-difference in the "basic" completion style.
>
> I kind of like the underline for first-difference because it is similar
> to the underline used in some toolkit's menus to indicate which
> key-shortcut to use to select this entry.

Key-shortcuts in menus is a good analogy.  So underline for
completions-first-difference, and bold for completions-common-part
would look really nice.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 29.10.2019 23:53, Juri Linkov wrote:
> Or implement a solution that satisfies everyone: since in completion
> styles other than "basic", there are many characters that you can type
> to narrow completion further, is it possible to mark all them with
> completions-first-difference?

I don't think it's a great idea. The -first-difference face serves a 
good purpose: showing the most optimal character to type, to narrow down 
the results as much as possible while retaining the given completion.

Re: BIKESHED: completion faces

[João Távora]

Juri Linkov <juri@linkov.net> writes:

>>> Other completion styles don't highlight completions-first-difference
>>> at all.  I'm not sure if only the "basic" completion style highlights
>>> completions-first-difference.
>>
>> Eh.  I removed it recently, without asking anyone, waiting for
>> someone to complain. :-)
>>
>> Are you complaining? :-)  If you are, I'll revert that bit. If you aren't
>> I probably should revert it too...
>
> No, I'm not complaining.  Wait for real complaints from someone
> who will really get angry at your changes.  Or you could revert
> preemptively to avoid such angry complaints. :-)

Yes, probably a good idea.  

> Or implement a solution that satisfies everyone: since in completion
> styles other than "basic", there are many characters that you can type
> to narrow completion further, is it possible to mark all them with
> completions-first-difference?

Yes, but no.  In the flex style, every character besides the one you've
already entered will potentially narrow the completion further.  So
you'd be left with a negative image than what we want.  Do I explain
myself.

And "what we want" is for it to work more like Company, Helm, or other
editors and UI's out there in the wild: show prominently the places in
the candidate where the pattern matches.

João

Re: BIKESHED: completion faces

[João Távora]

Juri Linkov <juri@linkov.net> writes:

>>>>> bold in completions-first-difference helps to immediately see the
>>>>> next character to type to narrow completions further.
>>>> <mild bikeshedding starts>
>>>> Yes, but some questions, Juri:
>>>> * wouldn't any other face, say "underline", serve the same purpose?
>>> underline is less noticeable than bold, when used on a single character
>>> with completions-first-difference in the "basic" completion style.
>>
>> I kind of like the underline for first-difference because it is similar
>> to the underline used in some toolkit's menus to indicate which
>> key-shortcut to use to select this entry.
>
> Key-shortcuts in menus is a good analogy.  So underline for
> completions-first-difference, and bold for completions-common-part
> would look really nice.

Well that's exactly what Stefan is proposing.  OMG two bikesheds of the
same color: the Arthur Jackson award at last!

João

PS: Three sheds actually, I'm +1 on this too.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 27.10.2019 3:58, Stefan Monnier wrote:
> In Emacs-26, completion faces looked like the following:
> 
> - the "common" part shown like the default face.
> - the "first-difference" shown in bold.
> - everything else uses the default face.
> 
> The "common" part is the part applied to the characters that are also
> found in the current minibuffer.  E.g. when completing
> 
>     M-x ne-li ?
> 
> the "ne" and "-li" chars in the *Completions* are the "common" part.
> 
> With the basic, prefix completion, the "common" part is not very
> important and is separated from the rest at the "first-difference", so
> the default faces make a fair bit of sense there.
> 
> But for other completion styles such as `substring`,
> `partial-completion`, and even more so for `flex`, it's not always
> immediately obvious how the minibuffer contents relate to the possible
> completions displayed in *Completions*.

I wonder if it's *that* important, to be able to see the matching chars.

The -first-difference face shows actionable info: the next character to 
type, to narrow down the completions list as much as possible while 
ensuring that a given completion is still in. You could type other 
chars, but the likelihood of ending up with several options (and no more 
characters to type) will be higher.

So IMHO we could do nothing, and it will be a valid choice. As long as 
all completion styles highlight completion-first-difference properly.

Furthermore, the position of -first-difference also hints at what 
characters were matched (definitely ones before it).

> For this reason, it is common in other completion systems to
> highlight the "common" part somehow (e.g. underline, bold, ...).

Personally, I'm not a big fan of underline, italic and even bold (unless 
it's used sparingly, like it is now). With that said...

> I think Emacs's defaults should be changed so that the "common" part
> (which uses the `completions-common-part` face) is highlighted somehow.
> 
> So, what should we go for:
> - bold (but then we need to change the `completions-first-difference`
>    face to keep it different, e.g. underlined)?

If we end up doing this, we might as well change 
completions-first-difference to be indistinguishable from 'default'. 
Because the common part, when highlighted, duplicates the information. 
The "first difference" character is simply the one after the last 
segment of the common part, isn't it?

> - underlined (tho I don't like underlined text very much)?
> - some foreground color?
> - some background color?
> - something else?

If we do end up highlighting completions-common-part somehow, I vote for 
background color, some very subtle one. Because using bold is going to 
be annoying with prefix-based styles, and I don't think we can/should 
use different methods for different styles.

Using "white smoke" seems okay with the default theme, as well as with 
the light-background theme I'm using. No idea what color would be good 
with dark-background themes.

Re: BIKESHED: completion faces

[Stefan Monnier]

> I wonder if it's *that* important, to be able to see the matching chars.

It has an explanatory value which makes the completion-style more
understandable (e.g. I'm pretty sure there are still users who don't
know about the `partial-completion` style that's enabled by default:
they just occasionally see completion lists which they don't understand
and shrug it off).

But no, I don't know how important it is (I do find it useful when
implementing/debugging the code to make sure it worked right ;-)

This said, it seems that highlighting the common part is the "standard"
behavior is all other completion systems whereas highlighting the "first
difference" seems to be specific to Emacs.

>> For this reason, it is common in other completion systems to
>> highlight the "common" part somehow (e.g. underline, bold, ...).
> Personally, I'm not a big fan of underline, italic and even bold (unless
> it's used sparingly, like it is now). With that said...

FWIW, I'm no big fan of underline (maybe it's the LaTeX influence speaking).
But I found it tolerable for first-difference after trying it out.


        Stefan


> The -first-difference face shows actionable info: the next character
> to type, to narrow down the completions list as much as possible while
> ensuring that a given completion is still in.
[...]
> Furthermore, the position of -first-difference also hints at what
> characters were matched (definitely ones before it).
[...]
> The "first difference" character is simply the one after the last
> segment of the common part, isn't it?

Agreed.

This is largely irrelevant for the choice of bikeshed color, but
I figure it's as a good a time as any to clarify that the above, while
largely true is not quite right either:

It's a bit more subtle than that (and the "first-difference" name is
a misnomer, due to history): when I revamped the default completion
system for Emacs-24, adding partial-completion (as well as `basic` which
is a bit more sophisticated than just "prefix completion", in reality),
I had to decide what to do with "first-difference" since it doesn't have
as clear a meaning as for the plain old prefix completion.  So the way
I adapted it to the new reality is that "first difference" now really
means "first char after point".  If you do

    M-x dovi C-b C-b ?

you'll see that `completions-first-difference` is applied to the "c"
of the "doc-view..." commands because that's the first character after
the position in the candidate that corresponds to the position of
point in the pattern.

So strictly speaking:
- No, there can also be common-part characters after first-difference
- Also, it's not always the case that the "first-difference" character
  is "the next character to type, to narrow down the completions list
  as much as possible while ensuring that a given completion is still
  in".  Two reasons for that:
  1- the "first-difference" character may already be part of the
     "common part".  E.g. `M-x revert C-b ?` will show the `t` with
     both `completions-first-difference` and `completion-common-part`
     and typing `t` will actually end up with a pattern that doesn't
     match anything.
  2- depending on the completion style, there might be further
     character that still keep the desired completion while
     eliminating more of the others.
     For example let's say you want to run
     `doc-view-fit-height-to-window`; after you do `M-x doc-view- ?`
     you'll see that the `f` is highlighted as first-difference
     but if hit `f` you still have 5 remaining choices, whereas if you
     hit `h` your choice is the only remaining one.

But by and large, these are exceptions.  Point (2) above is a bit more
true for `flex`, but I don't think it matters very much anyway.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 30.10.2019 3:49, Stefan Monnier wrote:

> It has an explanatory value which makes the completion-style more
> understandable (e.g. I'm pretty sure there are still users who don't
> know about the `partial-completion` style that's enabled by default:
> they just occasionally see completion lists which they don't understand
> and shrug it off).

And just use it, probably. Even if they don't exactly see what matches what.

> But no, I don't know how important it is (I do find it useful when
> implementing/debugging the code to make sure it worked right ;-)

Indeed, but that's covered by your ability to customize the face anyway.

> This said, it seems that highlighting the common part is the "standard"
> behavior is all other completion systems whereas highlighting the "first
> difference" seems to be specific to Emacs.

Probably, but "all other completion systems" use a popup, which is a 
significantly different UI. They usually have some different background 
for that rectangle, and a different foreground as well. So there's some 
pre-existing theming/styling work there.

Further, none of the other systems AFAIK use a fallback list of 
completion styles. So normally the matching is fuzzy to begin with, and 
in that case, highlighting the matches is moderately useful.

They won't see a situation with a long list of simple prefix matches 
where the first column is highlighted uniformly.

>>> For this reason, it is common in other completion systems to
>>> highlight the "common" part somehow (e.g. underline, bold, ...).
>> Personally, I'm not a big fan of underline, italic and even bold (unless
>> it's used sparingly, like it is now). With that said...
> 
> FWIW, I'm no big fan of underline (maybe it's the LaTeX influence speaking).
> But I found it tolerable for first-difference after trying it out.

I'm more against using bold for the common part. Underline is less 
aggravating.

> It's a bit more subtle than that (and the "first-difference" name is
> a misnomer, due to history): when I revamped the default completion
> system for Emacs-24, adding partial-completion (as well as `basic` which
> is a bit more sophisticated than just "prefix completion", in reality),
> I had to decide what to do with "first-difference" since it doesn't have
> as clear a meaning as for the plain old prefix completion.  So the way
> I adapted it to the new reality is that "first difference" now really
> means "first char after point".  If you do
> 
>      M-x dovi C-b C-b ?
> 
> you'll see that `completions-first-difference` is applied to the "c"
> of the "doc-view..." commands because that's the first character after
> the position in the candidate that corresponds to the position of
> point in the pattern.

I'm having trouble reproducing this. Should I be using icomplete-mode? 
Anyway, it bold-s the whole current selected completion.

And if I type "(dovi" in an Elisp buffer, C-M-i moves point to after the 
common part.

So I get what you're hinting at, in theory, but these cases should 
indeed be rare. And there might be other things we could do about them.

>    2- depending on the completion style, there might be further
>       character that still keep the desired completion while
>       eliminating more of the others.
>       For example let's say you want to run
>       `doc-view-fit-height-to-window`; after you do `M-x doc-view- ?`
>       you'll see that the `f` is highlighted as first-difference
>       but if hit `f` you still have 5 remaining choices, whereas if you
>       hit `h` your choice is the only remaining one.

This one in interesting. Some pre-computation could give a better hint 
in this regard, but it could be disorienting as well, if one is used for 
first-difference to be the char after point.

Re: BIKESHED: completion faces

[Stefan Monnier]

> Further, none of the other systems AFAIK use a fallback list of completion
> styles. So normally the matching is fuzzy to begin with, and in that case,
> highlighting the matches is moderately useful.
>
> They won't see a situation with a long list of simple prefix matches where
> the first column is highlighted uniformly.

Indeed, part of the issue is the difference between something like
`basic` and something like `flex`: in `basic` the
completions-common-part is not terribly interesting, whereas in `flex`
it's much more so.

We could use different faces in those two cases, but I feel like it
would be better if we can find a default that works well in both cases.

> I'm more against using bold for the common part. Underline is
> less aggravating.

Hmm... sadly, I find underlying the completions-common-part to be pretty
awful.  So it looks like both bold and underline are out?
Should we look for colors (background? foreground?)?

>>      M-x dovi C-b C-b ?
>> you'll see that `completions-first-difference` is applied to the "c"
>> of the "doc-view..." commands because that's the first character after
>> the position in the candidate that corresponds to the position of
>> point in the pattern.
>
> I'm having trouble reproducing this. Should I be using icomplete-mode?

No, just `emacs -Q` and then `M-x dovi C-b C-b ?`

> So I get what you're hinting at, in theory, but these cases should indeed be
> rare. And there might be other things we could do about them.

Agreed, I was just pointing out some corner-case behavior.  It's of no
consequence for the current discussion.


        Stefan

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 04.11.2019 1:30, Stefan Monnier wrote:

> Indeed, part of the issue is the difference between something like
> `basic` and something like `flex`: in `basic` the
> completions-common-part is not terribly interesting, whereas in `flex`
> it's much more so.
> 
> We could use different faces in those two cases, but I feel like it
> would be better if we can find a default that works well in both cases.

I concur.

>> I'm more against using bold for the common part. Underline is
>> less aggravating.
> 
> Hmm... sadly, I find underlying the completions-common-part to be pretty
> awful.  So it looks like both bold and underline are out?
> Should we look for colors (background? foreground?)?

As suggested previously, a light-grey background might work. Or a dark 
foreground.

IOW, either

   (set-face-foreground 'completions-common-part "blue3")

or

   (set-face-background 'completions-common-part "white smoke")

But neither seems optimal for prefix completion still.

Some other ideas:

- No nothing, on the reasoning that 'flex' is not in 'completion-styles' 
by default. And many users who would add it there would probably use it 
exclusively, and could thus customize their completions-common-part face 
to be more informative, to their satisfaction. This suggestion could be 
in the docs somewhere, maybe in the NEWS section that introduces 'flex'.

- Define a new face that would be applied by callers of 
completion-all-completions to those completions where the equation 
"common part == the whole part of the string before the char highlighted 
by first-difference" doesn't hold true. So all uses of this face would 
be informative. Although some absences of it might stand out (where a 
flex match is a strict-prefix one). That face would have a distinct 
background or foreground.

>>>       M-x dovi C-b C-b ?
>>> you'll see that `completions-first-difference` is applied to the "c"
>>> of the "doc-view..." commands because that's the first character after
>>> the position in the candidate that corresponds to the position of
>>> point in the pattern.
>>
>> I'm having trouble reproducing this. Should I be using icomplete-mode?
> 
> No, just `emacs -Q` and then `M-x dovi C-b C-b ?`

OK, thank you. I see it now. Normally one would press TAB, not '?', though.

Re: BIKESHED: completion faces

[Stefan Monnier]

>>> I'm having trouble reproducing this. Should I be using icomplete-mode?
>> No, just `emacs -Q` and then `M-x dovi C-b C-b ?`
> OK, thank you. I see it now. Normally one would press TAB, not '?', though.

Yes, but ? gives more control over the pattern (with TAB the pattern
has to be "already fully expanded" in order to get a list of
completions), so it's more work to find a good example ;-)


        Stefan

Re: BIKESHED: completion faces

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Some other ideas:
>
> - No nothing, on the reasoning that 'flex' is not in
>   'completion-styles' by default. And many users who would add it
>  there would probably use it exclusively, and could thus customize
> their completions-common-part face to be more informative, to their
> satisfaction. This suggestion could be in the docs somewhere, maybe in
> the NEWS section that introduces 'flex'.

I don't agree with this reasoning at all.  First I think we agree Emacs
would profit (in influx from users) from defaults that make it behave
less like 90's Emacs and more like other younger editors.  However we
also know that we can't just change the defaults without bothering
users already relying on those defaults.

So the next best thing is to add new functionality with a minimum of
customization necessary.  So, IMO, it's nonsensical to ask users to
customize the matching style to 'flex' AND also ask them customize a
face so that that more-than-familiar matching strategy reveals its
familiarity.

> - Define a new face that would be applied by callers of
>   completion-all-completions to those completions where the equation 
> "common part == the whole part of the string before the char
> highlighted by first-difference" doesn't hold true. So all uses of
> this face would be informative. Although some absences of it might
> stand out (where a flex match is a strict-prefix one). That face would
> have a distinct background or foreground.

This is better: I think this would lead you to my idea of a new
`completion-relevant` face, which we would set to bold or something very
prominent.  For the 'prefix' the relevant part is the "next character"
and for the 'flex' style is whatever has matched.
completions-first-difference could then be made an alias of that new
face.

So my idea is:

1.  New 'completion-relevant' face, defaulting to the current
completions-first-difference value, used for emphasizing the relevant
parts of a completion.  Also alias the exiting
'completions-first-difference' to this new face.

2.  New 'completion-shadow' face, defaulting to the current
'completions-common-part' values, used for marking "seconday" parts of
completions .  Also alias the existing 'completions-common-part' face to
this new face.

Instead of 'completion-shadow', we could also call this
'completion-auxiliary' or something like that.

3. Completion styles decide which parts which faces to apply and when.
Prefix uses both like it does now.  Flex should probably only use
'completion-relevant'.

João

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 05.11.2019 0:52, João Távora wrote:

> I don't agree with this reasoning at all.  First I think we agree Emacs
> would profit (in influx from users) from defaults that make it behave
> less like 90's Emacs and more like other younger editors.  However we
> also know that we can't just change the defaults without bothering
> users already relying on those defaults.

We agree, yes. Although I'm more in favor of changing the defaults, as 
soon as the kinks are worked out (e.g. flex is a bit slow when prefix is 
1-2 chars).

> So the next best thing is to add new functionality with a minimum of
> customization necessary.  So, IMO, it's nonsensical to ask users to
> customize the matching style to 'flex' AND also ask them customize a
> face so that that more-than-familiar matching strategy reveals its
> familiarity.

I disagree that it's a significant problem, though. Enabling 'flex' is 
one line. Customizing the face is just another line.

> This is better: I think this would lead you to my idea of a new
> `completion-relevant` face, which we would set to bold or something very
> prominent.  For the 'prefix' the relevant part is the "next character"
> and for the 'flex' style is whatever has matched.
> completions-first-difference could then be made an alias of that new
> face.

Hmm, I don't like this changing of terms, sorry. This way, the same 
highlighting would be applied to two fairly different things.

> 3. Completion styles decide which parts which faces to apply and when.
> Prefix uses both like it does now.  Flex should probably only use
> 'completion-relevant'.

With my proposal, the positions of the new face can be unambiguously 
determined by the current markings. So its application doesn't need to 
be implemented in completion styles. It can be done in one place: the 
code rendering the *Completions* buffer.

Re: BIKESHED: completion faces

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:
> On 05.11.2019 0:52, João Távora wrote:

> We agree, yes. Although I'm more in favor of changing the defaults,

So are we all, but that's a non-starter.  I'm sure everyone here thinks
"their" settings should be the defaults.

So what you are proposing with the "do-nothing" approach amounts to a
lose-lose.

> as soon as the kinks are worked out (e.g. flex is a bit slow when
> prefix is 1-2 chars).

You have to think about this "slowness".  A part of it might be
discovering if the 1-2 chars are in the potential match.  Another
different part of it might be due to the fact that the set of matches is
much greater when the pattern in short.

The former part can be improved in flex, the latter can't: it's
intrinsic to the technique.  But in matching systems like icomplete-mode
it isn't a problem (in terms of responsiveness) because icomplete-mode
has a while-no-input trick in it.  Perhaps so should company (presuming
that's what you are using).

>> So the next best thing is to add new functionality with a minimum of
>> customization necessary.  So, IMO, it's nonsensical to ask users to
>> customize the matching style to 'flex' AND also ask them customize a
>> face so that that more-than-familiar matching strategy reveals its
>> familiarity.
>
> I disagree that it's a significant problem, though. Enabling 'flex' is
> one line. Customizing the face is just another line.

Isn't true for custom.el users.  And it just doesn't make sense that to
enable "good" flex matching you have to go touch two places.  We're
discussing usability, after all.

>> This is better: I think this would lead you to my idea of a new
>> `completion-relevant` face, which we would set to bold or something very
>> prominent.  For the 'prefix' the relevant part is the "next character"
>> and for the 'flex' style is whatever has matched.
>> completions-first-difference could then be made an alias of that new
>> face.
>
> Hmm, I don't like this changing of terms, sorry. This way, the same
> highlighting would be applied to two fairly different things.

So is 'shadow' and 'bold' and many more.  It all has to do with how you
design the semantics, something that is our prerrogative.  The current
face semantics were designed for 'prefix', they just don't scale well
for other pattern-matching strategies.

What I'm proposing is no different from say, mode-line-emphasis, which
lisp/man.el and lisp/vc/vc-dispatcher.el use in "two wildly different
things".

Of course, another way out, at the expense of yet another face, is to
add completions-flex-emphasis or something like that.

João

Re: BIKESHED: completion faces

[Dmitry Gutov]

[-- Attachment #1: Type: text/plain, Size: 3386 bytes --]

On 05.11.2019 13:10, João Távora wrote:

>> We agree, yes. Although I'm more in favor of changing the defaults,
> 
> So are we all, but that's a non-starter.  I'm sure everyone here thinks
> "their" settings should be the defaults.

The implication would be that trying to improve the defaults is useless 
or hopeless. I disagree.

> So what you are proposing with the "do-nothing" approach amounts to a
> lose-lose.

Have you read my other idea, then?

>> as soon as the kinks are worked out (e.g. flex is a bit slow when
>> prefix is 1-2 chars).
> 
> You have to think about this "slowness".  A part of it might be
> discovering if the 1-2 chars are in the potential match.  Another
> different part of it might be due to the fact that the set of matches is
> much greater when the pattern in short.
> 
> The former part can be improved in flex, the latter can't: it's
> intrinsic to the technique.

All can be improved, just with varying degrees of difficulty. But there 
are other techniques, like limiting the number of matches shown at a 
time. One just has to make sure not to cache the result improperly.

> But in matching systems like icomplete-mode
> it isn't a problem (in terms of responsiveness) because icomplete-mode
> has a while-no-input trick in it.  Perhaps so should company (presuming
> that's what you are using).

I'm trying it out with 'M-x completion-at-point'.

>> I disagree that it's a significant problem, though. Enabling 'flex' is
>> one line. Customizing the face is just another line.
> 
> Isn't true for custom.el users.  And it just doesn't make sense that to
> enable "good" flex matching you have to go touch two places.  We're
> discussing usability, after all.

As you can imagine, IMHO this part "making sense" is less important than 
the consistency in highlighting.

>>> This is better: I think this would lead you to my idea of a new
>>> `completion-relevant` face, which we would set to bold or something very
>>> prominent.  For the 'prefix' the relevant part is the "next character"
>>> and for the 'flex' style is whatever has matched.
>>> completions-first-difference could then be made an alias of that new
>>> face.
>>
>> Hmm, I don't like this changing of terms, sorry. This way, the same
>> highlighting would be applied to two fairly different things.
> 
> So is 'shadow' and 'bold' and many more.  It all has to do with how you
> design the semantics, something that is our prerrogative.

I wouldn't use 'shadow' or 'bold' for the new face either. Nothing that 
matches completions-common-part of completions-first-difference exactly.

> The current
> face semantics were designed for 'prefix', they just don't scale well
> for other pattern-matching strategies.
> 
> What I'm proposing is no different from say, mode-line-emphasis, which
> lisp/man.el and lisp/vc/vc-dispatcher.el use in "two wildly different
> things".

Here's an example. When the input is one char, how will you figure out 
what the highlighting in the *Completions* buffer means? Just by whether 
the character under the face is the same one? It's not reliable.

> Of course, another way out, at the expense of yet another face, is to
> add completions-flex-emphasis or something like that.

I've tried to implement my idea, but somehow the added highlighting gets 
eaten before the buffer is displayed. But the attached patch should 
illustrate it anyway.

[-- Attachment #2: completions-nontrivial-common-part.diff --]
[-- Type: text/x-patch, Size: 4034 bytes --]

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 43dd277a2e..e272364b49 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -1649,10 +1649,8 @@ completion--insert-strings
 		  nil))))
             (setq first nil)
             (if (not (consp str))
-                (put-text-property (point) (progn (insert str) (point))
-                                   'mouse-face 'highlight)
-              (put-text-property (point) (progn (insert (car str)) (point))
-                                 'mouse-face 'highlight)
+                (completion--insert-string str)
+              (completion--insert-string (car str))
               (let ((beg (point))
                     (end (progn (insert (cadr str)) (point))))
                 (put-text-property beg end 'mouse-face nil)
@@ -1672,6 +1670,53 @@ completion--insert-strings
 			      ;; Round up to a whole number of columns.
 			      (* colwidth (ceiling length colwidth))))))))))))
 
+(defun completion--insert-string (s)
+  (require 'text-property-search)
+  (let* ((beg (point))
+         (end (progn (insert s)
+                     (point)))
+         (search-pred (lambda (expected value)
+                        (or (eq expected value)
+                            (and (listp value)
+                                 (memq expected value)))))
+         fdiff-match)
+    (put-text-property beg end 'mouse-face 'highlight)
+    (save-excursion
+      (save-restriction
+        (narrow-to-region beg end)
+        (goto-char beg)
+        (setq fdiff-match
+              (text-property-search-forward 'face 'completions-first-difference
+                                            search-pred))
+        (when fdiff-match
+          (let ((fdiff-pos (prop-match-beginning fdiff-match))
+                (prop-pos (point)))
+            (goto-char fdiff-pos)
+            (when (or
+                   ;; There is a place before fdiff-pos without common part.
+                   (catch 'found
+                     (while (setq prop-pos
+                                  (previous-single-property-change prop-pos 'face
+                                                                   nil beg))
+                       (when (= prop-pos beg)
+                         (throw 'found nil))
+                       (let ((value (get-text-property (1- prop-pos) 'face)))
+                         (if (or (eq value 'completions-common-part)
+                                 (and (listp value)
+                                      (memq 'completions-common-part value)))
+                             (setq prop-pos (1- prop-pos))
+                           (throw 'found t)))))
+                   ;; There is a place after fdiff-pos with common part.
+                   (text-property-search-forward 'face 'completions-common-part
+                                                 search-pred))
+              ;; FIXME: For some reason, this highlighting gets eaten
+              ;; before the buffer is displayed, somewhere.
+              ;; TODO: Highlight only the parts with -common-part.
+              (font-lock-prepend-text-property beg fdiff-pos
+                                               'font-lock-face
+                                               'completions-nontrivial-common-part)
+              )))))))
+
 (defvar completion-common-substring nil)
 (make-obsolete-variable 'completion-common-substring nil "23.1")
 
@@ -1691,6 +1736,10 @@ completions-common-part
   "Face for the parts of completions which matched the pattern.
 See also the face `completions-first-difference'.")
 
+(defface completions-nontrivial-common-part '((t (:background "white smoke")))
+  "Face for the parts of completions which matches the pattern nontrivially.
+Meaning that the match is a non-prefix one.")
+
 (defun completion-hilit-commonality (completions prefix-len &optional base-size)
   "Apply font-lock highlighting to a list of completions, COMPLETIONS.
 PREFIX-LEN is an integer.  BASE-SIZE is an integer or nil (meaning zero).

Re: BIKESHED: completion faces

[Stefan Monnier]

> I've tried to implement my idea, but somehow the added highlighting gets
> eaten before the buffer is displayed. But the attached patch should 
> illustrate it anyway.

Regardless of how we implement this, the question remains: do we want to
highlight the common-part of the prefix/basic completion differently
from the other cases?


        Stefan


PS: I think the approach used in your patch will lead to undesirable
behavior when some matches (but not all) have all chars before
first-diff highlighted with common-part.  E.g.

    src/emacs -Q --eval '(setq completion-styles (quote (partial-completion)))'
    M-x re-b ?

where `re-builder` would be highlighted with a "trivial common part"
whereas `rename-buffer` would be highlighted differently.

You might consider it as a feature, of course.  But if not, then we'd
have to first look at all the completions to make sure they all have
a "trivial common part".

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1115 bytes --]

> > I've tried to implement my idea, but somehow the added highlighting gets
> > eaten before the buffer is displayed. But the attached patch should
> > illustrate it anyway.
>
> Regardless of how we implement this, the question remains: do we want to
> highlight the common-part of the prefix/basic completion differently
> from the other cases?

I, for one, think we shouldn't.  But I won't mind it, if that means
having a good, familiar, promiment, those-really-are-the-mathing-parts,
default for the 'flex' and 'substring' styles.

My personal bikeshedding proposal goes in this direction.  We can
later enhance it by convincing 'prefix' users that it's just as
explanatory to have the common part prominent and the first difference
un-prominent.  Then there won't be much of a dissonance.

But I liked any of your initial proposals, too, Stefan.  They are just
as trivial and also scratch my back--shed.  Also Juri seemed to
suggest something in consonance with it at some point.

And if Dmitry wants to push new defaults, I'm behind him, too.

João  "nearing bikeshed stack overflow"

[-- Attachment #2: Type: text/html, Size: 1280 bytes --]

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 5154 bytes --]

[sorry for the duplicate email Dmitry, I forgot to cc emacs-devel]

On Tue, Nov 5, 2019 at 2:57 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 05.11.2019 13:10, João Távora wrote:
>
> >> We agree, yes. Although I'm more in favor of changing the defaults,
> >
> > So are we all, but that's a non-starter.  I'm sure everyone here thinks
> > "their" settings should be the defaults.
>
> The implication would be that trying to improve the defaults is useless
> or hopeless. I disagree.

I would rather say that setting a change to Emacs defaults as a bar to
clear is counter-productive.  And frequently demotivational.

Best solutions arise when we can understand exactly what it is that
people like about current defaults, and can accomodate that specific
preference in a new system, so that they barely notice it.

> > So what you are proposing with the "do-nothing" approach amounts to a
> > lose-lose.
> Have you read my other idea, then?

Yes.  A new face, I presume.  And a generic attempt to figure a good
place to apply it for every possible style.  The patch is a bit too
large for reviewing right now.

By the way, every other proposal in this thread, is trivial.  Only yours
adds significant new logic.  Not saying that's bad, but it's a drawback
the others don't have.

> > The former part can be improved in flex, the latter can't: it's
> > intrinsic to the technique.
> All can be improved, just with varying degrees of difficulty.

Sure, a pig and a large enough rocket...

> But there
> are other techniques, like limiting the number of matches shown at a
> time.

Sure, I was talking about parts of 'flex' itself.  There are of course
improvements outside it (even besides the large enough rocket).  Say,
generators and lazy evaluation in completions could indeed be useful.
But that's a possibly big big change in minibuffer.el and, much worse,
all its users.

> One just has to make sure not to cache the result improperly.

Cache invalidation, one of the "hard" problems in computing.  Buying
trouble, I say.

> > But in matching systems like icomplete-mode
> > it isn't a problem (in terms of responsiveness) because icomplete-mode
> > has a while-no-input trick in it.  Perhaps so should company (presuming
> > that's what you are using).
> I'm trying it out with 'M-x completion-at-point'.

Then the slowness you're experiencing stems from the fact that that is
blocking, eager, interface, i.e. it's not a "kink" in flex.

> >> I disagree that it's a significant problem, though. Enabling 'flex' is
> >> one line. Customizing the face is just another line.
> > Isn't true for custom.el users.  And it just doesn't make sense that to
> > enable "good" flex matching you have to go touch two places.  We're
> > discussing usability, after all.
> As you can imagine, IMHO this part "making sense" is less important than
> the consistency in highlighting.

It's only "inconsistent" if you you refuse to accept that concepts such
as "relevance" or "emphasis" are more important the specifics of the
matching technique implemented.

> >> highlighting would be applied to two fairly different things.
> >
> > So is 'shadow' and 'bold' and many more.  It all has to do with how you
> > design the semantics, something that is our prerrogative.
>
> I wouldn't use 'shadow' or 'bold' for the new face either. Nothing that
> matches completions-common-part of completions-first-difference exactly.
>
> > The current
> > face semantics were designed for 'prefix', they just don't scale well
> > for other pattern-matching strategies.
> >
> > What I'm proposing is no different from say, mode-line-emphasis, which
> > lisp/man.el and lisp/vc/vc-dispatcher.el use in "two wildly different
> > things".
>
> Here's an example. When the input is one char, how will you figure out
> what the highlighting in the *Completions* buffer means?

OK, let's take that example.  It depends on whether I'm using a "prefix"
or "flex" style.  But even if I'm using them both together, or I'm such
a doofus that I don't remember anymore:

1. If I see that "one char" being matched in the middle of a lot of
completions, I know what it means: I'm using 'flex' and/or 'substring'
and that is the character that's matched.

2. If I see that every completion has its second character highlighted,
I know I'm using prefix.

This is discounting the fact that I probably want to know and control
whether I'm using flex or prefix anyway.  Using both is strange.

It's also discounting the fact that in 2. prefix completion should, IMO,
use my proposed "emphasis" face to highlight the first -- not the second
-- character. I've argued before that no loss of explanatory power would
materialize from this.

Finally it also dicounts the fact that yours is an extremely contorted
example.  I almost never want completions for stuff that contains or
starts with a specific single character.  Flex/substring becomes really
useful when you type 2-4 characters, and you are made visually aware of
the subsequence they matched within the candidate.

João

[-- Attachment #2: Type: text/html, Size: 6112 bytes --]

Re: BIKESHED: completion faces

[Stefan Monnier]

Hmm...

After trying it out for a while, I think I can't root for "common=bold,
firstdiff=underline" any more because in the basic/prefix-completion
case the firstdiff chars end up drowned by the previous bold common
prefix and the underlines: it's noticeable less legible IMO.

I tried the "white smoke" background for "common" but didn't like it
because it's too discrete, barely noticeable on my screen.

I'm no big fan of "common=blue3-background firstdiff=bold" but it might
be the least bad option for now: for the basic/prefix-completion it's
sufficiently discrete that it doesn't get in the way; for
flex-completion I find it to be a bit too discrete, but at least the
highlighting is visible enough that some users may be satisfied and that
those who aren't can clearly see that all it takes is to change the face
(whereas currently they can reasonably be expected to presume that the
common part does not have any face applied to it since its face is
indistinguishable from `default`).

BTW, what do "typical" other systems use as highlight for the
common part?  Is there enough agreement there that it's worth trying to
pay attention to it (kinda like the "blue" for hyperlink convention)?


        Stefan

Re: BIKESHED: completion faces

[Eli Zaretskii]

I think this thread lacks focus.  I think you should take a step back
and first decide on the _purpose_ of highlighting parts of the
completion candidates.  I don't think I saw the purpose discussed and
agreed upon anywhere; apologies if I missed something.

The purpose of completion-first-difference is to help the user decide
what to type next.  What is the purpose of highlighting other parts of
the candidates?  (If there is no useful purpose, then we might as well
do nothing and leave completion-first-difference the only important
highlighting in all kinds of completion.)

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1225 bytes --]

> After trying it out for a while, I think I can't root for "common=bold,
> firstdiff=underline" any more because in the basic/prefix-completion
> case the firstdiff chars end up drowned by the previous bold common
> prefix and the underlines: it's noticeable less legible IMO.

Just nullify the firstdiff face and it becomes better! And you still
"see" the first diff via very advanced visual differentiation
techniques available today in most mammal's brains.

> I'm no big fan of "common=blue3-background firstdiff=bold" but it might
> be the least bad option for now: for the basic/prefix-completion it's

Surely you jest, or you inverse-video the "common" font or you put
something in the foreground, because black on blue3 is illegible.

> BTW, what do "typical" other systems use as highlight for the
> common part?  Is there enough agreement there that it's worth trying to
> pay attention to it (kinda like the "blue" for hyperlink convention)?

Company uses the same face for flex and common, a drab red
that is barely visible against the default yellow.  But it's consistent.

Textmate used to use bold.  I bet Sublime text and other still use
it, but probably I'll lose that bet.

João

[-- Attachment #2: Type: text/html, Size: 1583 bytes --]

Re: BIKESHED: completion faces

[Stefan Monnier]

> Just nullify the firstdiff face and it becomes better! And you still
> "see" the first diff via very advanced visual differentiation
> techniques available today in most mammal's brains.

I don't think this has much of a chance of getting wide approval.

>> I'm no big fan of "common=blue3-background firstdiff=bold" but it might
                                   ^^^^
                                   fore
Duh!  Sorry!

> Company uses the same face for flex and common, a drab red
> that is barely visible against the default yellow.

It's a different colorscheme, but it's comparable to the
"blue3-foreground" in terms of visibility/discretion (maybe the
red-on-yellow is a bit more visible).

> Textmate used to use bold.  I bet Sublime text and other still use
> it, but probably I'll lose that bet.

I don't have any such editors around for testing, so if anybody else has
datapoints to contribute, that would be helpful.


        Stefan

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1.1: Type: text/plain, Size: 1131 bytes --]

On Tue, Nov 5, 2019 at 6:51 PM Eli Zaretskii <eliz@gnu.org> wrote:
> The purpose of completion-first-difference is to help the user decide
> what to type next.

Indeed it does that. But there are equally efficient other ways to do the
same, in my opinion.  One of them is to highlight the preceding
character.  You can experiment by switching the values of the "common"
and "first-difference" faces. Do you think you've lost that help?

> What is the purpose of highlighting other parts of
> the candidates?

For a matching style such as flex or substring (as you would find in
many other editors) it's important to visually explain users to users
why certain strings that don't start with the pattern they entered
are being shown on the screen.  I attach an image.

>  (If there is no useful purpose, then we might as well
> do nothing and leave completion-first-difference the only important
> highlighting in all kinds of completion.)

completion-first-difference is at the very least a misnomer for
other types of completion, because with flex there can be infinitely
many "first" differences.

João

[-- Attachment #1.2: Type: text/html, Size: 1526 bytes --]

[-- Attachment #2: index.jpg --]
[-- Type: image/jpeg, Size: 9754 bytes --]

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Tue, 5 Nov 2019 19:16:00 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>, 
> 	emacs-devel <emacs-devel@gnu.org>
> 
> > The purpose of completion-first-difference is to help the user decide
> > what to type next.  
> 
> Indeed it does that. But there are equally efficient other ways to do the
> same, in my opinion.

If we have one efficient way, why do we need to consider others?

> One of them is to highlight the preceding character.

??? How does this help me to select what to type next?

> > What is the purpose of highlighting other parts of
> > the candidates? 
> 
> For a matching style such as flex or substring (as you would find in 
> many other editors) it's important to visually explain users to users
> why certain strings that don't start with the pattern they entered 
> are being shown on the screen.  I attach an image.

I don't see why it's important to explain how did the completion
algorithm arrive at a particular candidate.  The completion algorithm
is there to intuit what we mean in the most efficient way, but the
details of how it does that are immaterial.  The only ones who may be
interested are those who study completion algorithms ;-)

> completion-first-difference is at the very least a misnomer for
> other types of completion, because with flex there can be infinitely 
> many "first" differences. 

No, "first difference" is always the character to be typed after
point.  At least for the vastly important case that point is at the
end of what you typed, i.e. you don't move point back after typing
something.

Re: BIKESHED: completion faces

[Juri Linkov]

>> I've tried to implement my idea, but somehow the added highlighting gets
>> eaten before the buffer is displayed. But the attached patch should 
>> illustrate it anyway.
>
> Regardless of how we implement this, the question remains: do we want to
> highlight the common-part of the prefix/basic completion differently
> from the other cases?

It seems inevitable to have a different set of faces for different
completion styles.  So for the prefix/basic completion the default
could be underline for completions-first-difference (like in menus)
and inheriting completions-common-part from the 'shadow' face
to make the trivial part less prominent.

More complex completion styles could use bold for a new face
to make non-trivial parts more prominent.

>     src/emacs -Q --eval '(setq completion-styles (quote (partial-completion)))'
>     M-x re-b ?
>
> where `re-builder` would be highlighted with a "trivial common part"
> whereas `rename-buffer` would be highlighted differently.
>
> You might consider it as a feature, of course.  But if not, then we'd
> have to first look at all the completions to make sure they all have
> a "trivial common part".

This really looks like a feature - it could help the users
to see the prefix in the completions list.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 05.11.2019 20:54, João Távora wrote:
>  > After trying it out for a while, I think I can't root for "common=bold,
>  > firstdiff=underline" any more because in the basic/prefix-completion
>  > case the firstdiff chars end up drowned by the previous bold common
>  > prefix and the underlines: it's noticeable less legible IMO.
> 
> Just nullify the firstdiff face and it becomes better! And you still
> "see" the first diff via very advanced visual differentiation
> techniques available today in most mammal's brains.

I do believe first-difference continues to play a useful role. 
Especially in situations where input's suffix is non-empty.

>  > BTW, what do "typical" other systems use as highlight for the
>  > common part?  Is there enough agreement there that it's worth trying to
>  > pay attention to it (kinda like the "blue" for hyperlink convention)?
> 
> Company uses the same face for flex and common, a drab red
> that is barely visible against the default yellow.  But it's consistent.

I think it's quite visible. If you have problems with its contrast, 
please file an issue. I'm surprised, though, since I generally prefer 
higher-contrast themes.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 05.11.2019 21:16, João Távora wrote:
> completion-first-difference is at the very least a misnomer for
> other types of completion, because with flex there can be infinitely
> many "first" differences.

Since we've established that its name isn't accurate anymore, it could 
use a rename, yes.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Date: Tue, 5 Nov 2019 19:16:00 +0000
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>, 
>> 	emacs-devel <emacs-devel@gnu.org>
>> 
>> > The purpose of completion-first-difference is to help the user decide
>> > what to type next.  
>> 
>> Indeed it does that. But there are equally efficient other ways to do the
>> same, in my opinion.
>
> If we have one efficient way, why do we need to consider others?

Indeed we don't, you're totally right, not for the 'prefix/basic'
completion.  However some people have made a point that there should be
some kind of consistency at this level between completion styles, that
the thing emphasized in 'prefix/basic' should have some semantic
relation to the thing emphasized thing for 'flex' and 'substring' too.

So, to "please greeks and trojans" [1], an equally efficient AND
cross-style-consistent style could be found, we could keep the "common"
and "first-difference" face names, and 'flex' would ship without a
default "crippling" (see below)..

>> One of them is to highlight the preceding character.
>
> ??? How does this help me to select what to type next?

Did you try the experiment I proposed (swap the two faces)?  I have no
problem recognizing the character "to type next" when I do it.  See the
attached gif another-way-to-see-the-first-difference.gif


[-- Attachment #2: another-way-to-see-the-first-difference.gif --]
[-- Type: image/gif, Size: 45739 bytes --]

[-- Attachment #3: Type: text/plain, Size: 1221 bytes --]


(BTW I'm sorry I made a .gif and not a .png.  Animation was not strictly
required here, but I don't know the keybindings to make simple
screenshots on this computer.)

>> > What is the purpose of highlighting other parts of
>> > the candidates? 
>> 
>> For a matching style such as flex or substring (as you would find in 
>> many other editors) it's important to visually explain users to users
>> why certain strings that don't start with the pattern they entered 
>> are being shown on the screen.  I attach an image.
>
> I don't see why it's important to explain how did the completion
> algorithm arrive at a particular candidate.  The completion algorithm
> is there to intuit what we mean in the most efficient way, but the
> details of how it does that are immaterial.  The only ones who may be
> interested are those who study completion algorithms ;-)

I may sound like a completion scholar to you, but you also sound like
you haven't experimented with 'flex' for more than 1 second, otherwise
you wouldn't be asking that question ;-) .  So I attach another Emacs -Q
gif, crippled-flex.gif, where you see the current problem, and yet
another gif, useful-flex.gif, where the matching part is highlighted
bold.


[-- Attachment #4: crippled-flex.gif --]
[-- Type: image/gif, Size: 104261 bytes --]

[-- Attachment #5: Type: text/plain, Size: 551 bytes --]


I think you will agree it's not a detail.  The reason we want to
highlight matching parts in flex is the same reason grep and every
search tool and search engine I know decides to highlight matching
parts: to call attention to the part that matched.

Of course, fixing that crippled default state of 'flex' is a couple of
customizations away (Put the common face to 'bold' and the
first-difference to nothing).  But, IMHO, it would be a shame if we were
to release Emacs 27 with this familar matching method and no good
default faces to go with it.


[-- Attachment #6: useful-flex.gif --]
[-- Type: image/gif, Size: 68378 bytes --]

[-- Attachment #7: Type: text/plain, Size: 1165 bytes --]


>> completion-first-difference is at the very least a misnomer for
>> other types of completion, because with flex there can be infinitely 
>> many "first" differences. 
> No, "first difference" is always the character to be typed after
> point.  At least for the vastly important case that point is at the
> end of what you typed, i.e. you don't move point back after typing
> something.

But for the 'flex' case (among others more obscure) that first character
"to be typed" -- presumably to narrow down the set further -- might be
any character following 'point', for each completion, not just the one
in the 'point+1' position.  If you use "flex" for a little while, you'll
soon see that this face (which, by default in Emacs, is the only "bold"
one of the two) is quite useless for 'flex'.

Finally, I believe other UI's that have this kind of flex search (I
think they are not rare at all nowadays, not just in editors) also use a
prominent (often bold) face, to mark the common part.  But, like Stefan,
I would like casual users of those other UI's to present the specifics.

João

[1]: Apparently an exclusively portuguese saying.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 05.11.2019 20:36, Stefan Monnier wrote:

> I tried the "white smoke" background for "common" but didn't like it
> because it's too discrete, barely noticeable on my screen.

You could experiment with other subtle colors, it was just my 
suggestion. But a foreground color might be a better choice, yes.

> BTW, what do "typical" other systems use as highlight for the
> common part?  Is there enough agreement there that it's worth trying to
> pay attention to it (kinda like the "blue" for hyperlink convention)?

IMHO we shouldn't give too much weight to those since they use a 
different UI anyway, but:

- VS Code uses a blue foreground (where the default is white). And the 
background is darkish grey. An odd combination, IMO.

- IntelliJ IDEA uses pink foreground instead of black: 
https://www.jetbrains.com/help/img/idea/2019.2/codeCompletionSecondClass1.png

- Atom uses bold.

- Vim doesn't highlight matching characters at all.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1426 bytes --]

On Tue, Nov 5, 2019 at 9:39 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> I do believe first-difference continues to play a useful role.
> Especially in situations where input's suffix is non-empty.

Those are relatively rare.
But yes, it plays an important secondary role there.  It should just
be named differently after that secondary importance, maybe
even using the word "secondary".

> >  > BTW, what do "typical" other systems use as highlight for the
> >  > common part?  Is there enough agreement there that it's worth trying
to
> >  > pay attention to it (kinda like the "blue" for hyperlink convention)?
> >
> > Company uses the same face for flex and common, a drab red
> > that is barely visible against the default yellow.  But it's consistent.
> I think it's quite visible. If you have problems with its contrast,
> please file an issue. I'm surprised, though, since I generally prefer
> higher-contrast themes.

I could use a better contrast yes.  But by "barely" visible, I was trying
to express that I _can_ still recognize it.  But it's really the threshold:
Stefan's blue3 foreground is beyond it, it's no good. I'll open an issue
if I find some better default.

I don't understand why we don't take cues from systems where
actual professional UI designers, veritable color-theme scholars,
are probably employed.  Don't we have some of those in our
ranks?

--
João Távora

[-- Attachment #2: Type: text/html, Size: 1876 bytes --]

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 05.11.2019 23:54, João Távora wrote:
> I don't understand why we don't take cues from systems where
> actual professional UI designers, veritable color-theme scholars,
> are probably employed.  Don't we have some of those in our
> ranks?

If we do, I wish they spoke out more.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 529 bytes --]

On Tue, Nov 5, 2019 at 9:54 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> - VS Code uses a blue foreground (where the default is white). And the
> background is darkish grey. An odd combination, IMO.
>

Do you have a screenshot of that?


> - IntelliJ IDEA uses pink foreground instead of black:
>
> https://www.jetbrains.com/help/img/idea/2019.2/codeCompletionSecondClass1.png


What was the pattern here? Just the letter n?

- Atom uses bold.
>

A screeshot would be great too, if you can spare it.

João

[-- Attachment #2: Type: text/html, Size: 1275 bytes --]

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 05.11.2019 17:44, Stefan Monnier wrote:
>> I've tried to implement my idea, but somehow the added highlighting gets
>> eaten before the buffer is displayed. But the attached patch should
>> illustrate it anyway.
> 
> Regardless of how we implement this, the question remains: do we want to
> highlight the common-part of the prefix/basic completion differently
> from the other cases?

We might. But I'd rather do that per-completion rather per completion 
style, hence the proposed patch.

> 
>          Stefan
> 
> 
> PS: I think the approach used in your patch will lead to undesirable
> behavior when some matches (but not all) have all chars before
> first-diff highlighted with common-part.  E.g.
> 
>      src/emacs -Q --eval '(setq completion-styles (quote (partial-completion)))'
>      M-x re-b ?
> 
> where `re-builder` would be highlighted with a "trivial common part"
                      ^ won't be                   ^nontrivial

> whereas `rename-buffer` would be highlighted differently.

Indeed. That's the whole idea. "re" and "-b" in "rename-buffer" would 
indeed be highlighted since the match is nontrivial.

> You might consider it as a feature, of course.

Yes. That's the idea.

As for how it would work in practice, we'll have to see once the patch 
is functioning.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 05.11.2019 23:58, João Távora wrote:
> On Tue, Nov 5, 2019 at 9:54 PM Dmitry Gutov <dgutov@yandex.ru 
> <mailto:dgutov@yandex.ru>> wrote:
> 
>     - VS Code uses a blue foreground (where the default is white). And the
>     background is darkish grey. An odd combination, IMO.
> 
> 
> Do you have a screenshot of that?

https://code.visualstudio.com/assets/docs/editor/intellisense/intellisense.gif

>     - IntelliJ IDEA uses pink foreground instead of black:
>     https://www.jetbrains.com/help/img/idea/2019.2/codeCompletionSecondClass1.png
> 
> 
> What was the pattern here? Just the letter n?

Probably.

>     - Atom uses bold.
> 
> 
> A screeshot would be great too, if you can spare it.

https://flight-manual.atom.io/using-atom/images/autocomplete.png

In case you were wondering, I don't have any of these editors installed.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

On Tue, Nov 5, 2019 at 10:03 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 05.11.2019 23:58, João Távora wrote:
> > On Tue, Nov 5, 2019 at 9:54 PM Dmitry Gutov <dgutov@yandex.ru
> > <mailto:dgutov@yandex.ru>> wrote:
> >
> >     - VS Code uses a blue foreground (where the default is white). And
> the
> >     background is darkish grey. An odd combination, IMO.
> > Do you have a screenshot of that?
>
>
> https://code.visualstudio.com/assets/docs/editor/intellisense/intellisense.gif


Thanks. Looks like its bold blue, not just blue. It's quite visible.

> A screeshot would be great too, if you can spare it.
>
> https://flight-manual.atom.io/using-atom/images/autocomplete.png
>
> In case you were wondering, I don't have any of these editors installed.
>

I don't know if I was but doesn't matter. :-)

-- 
João Távora

[-- Attachment #2: Type: text/html, Size: 1765 bytes --]

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 05.11.2019 18:31, João Távora wrote:

> I would rather say that setting a change to Emacs defaults as a bar to
> clear is counter-productive.  And frequently demotivational.

Emacs has a problem of trying to accommodate too many workflows in the 
same UI. The result sometimes turns out to not be optimal for any of 
them. I'm trying to avoid this problem here.

> Best solutions arise when we can understand exactly what it is that
> people like about current defaults, and can accomodate that specific
> preference in a new system, so that they barely notice it.

That sounds like a change in defaults, though.

>  > > So what you are proposing with the "do-nothing" approach amounts to a
>  > > lose-lose.
>  > Have you read my other idea, then?
> 
> Yes.  A new face, I presume.  And a generic attempt to figure a good
> place to apply it for every possible style.

Yup. Okay.

> By the way, every other proposal in this thread, is trivial.  Only yours
> adds significant new logic.  Not saying that's bad, but it's a drawback
> the others don't have.

Your proposal would add a different styling for different completion 
styles. That would require some code as well, likely a similar amount.

>  > > The former part can be improved in flex, the latter can't: it's
>  > > intrinsic to the technique.
>  > All can be improved, just with varying degrees of difficulty.
> 
> Sure, a pig and a large enough rocket...

Is that because the current completion system is not optimal for flex?

>  > But there
>  > are other techniques, like limiting the number of matches shown at a
>  > time.
> 
> Sure, I was talking about parts of 'flex' itself.  There are of course
> improvements outside it (even besides the large enough rocket).  Say,
> generators and lazy evaluation in completions could indeed be useful.
> But that's a possibly big big change in minibuffer.el and, much worse,
> all its users.

minibuffer.el will likely need changes, yes.

But as for "outside"... I was thinking of a data structure where you 
would accumulate completions in a sorted tree, of sorts. And, when a new 
one is added, if the maximum allowed number of completions has been 
reached, the least-scored one is evicted. Or we do the eviction at the 
end, whatever.

Point is, we end up with a limited-size list containing only the best 
matches. That wouldn't avoid performance problems with the flex 
algorithm itself, but any subsequent bottlenecks would not be triggered.

This approach really relies on good and fast scoring, though.

>  > One just has to make sure not to cache the result improperly.
> 
> Cache invalidation, one of the "hard" problems in computing.  Buying
> trouble, I say.

Let us not forget that we're competing with other editors who routinely 
show off flex matching and somehow deal with accompanying performance 
problems.

>  > As you can imagine, IMHO this part "making sense" is less important than
>  > the consistency in highlighting.
> 
> It's only "inconsistent" if you you refuse to accept that concepts such
> as "relevance" or "emphasis" are more important the specifics of the
> matching technique implemented.

I'm more interested in highlighting being consistent and relevant to 
whatever the next action the user should perform.

>  > > What I'm proposing is no different from say, mode-line-emphasis, which
>  > > lisp/man.el and lisp/vc/vc-dispatcher.el use in "two wildly different
>  > > things".
>  >
>  > Here's an example. When the input is one char, how will you figure out
>  > what the highlighting in the *Completions* buffer means?
> 
> OK, let's take that example.  It depends on whether I'm using a "prefix"
> or "flex" style.  But even if I'm using them both together, or I'm such
> a doofus that I don't remember anymore:
> 
> 1. If I see that "one char" being matched in the middle of a lot of
> completions, I know what it means: I'm using 'flex' and/or 'substring'
> and that is the character that's matched.

That implies an extra thinking step. Whatever synapses take part it 
that, my brain could've used for something else.

> 2. If I see that every completion has its second character highlighted,
> I know I'm using prefix.

_Probably_, but you won't know for sure until you also read those 
characters. There could be just 2-3 completions anyway, so you wouldn't 
get the picture at a glance.

> This is discounting the fact that I probably want to know and control
> whether I'm using flex or prefix anyway.

But you probably won't. Or other users won't. That's the idea behind 
completion-styles anyway. If you are the "controlling" type anyway, you 
would probably customize faces anyway.

And here's also the argument for waiting until the default changes: 
because if the new default is (setq completion-styles '(flex)), then the 
dilemma we have before us now is easily dismissed.

> It's also discounting the fact that in 2. prefix completion should, IMO,
> use my proposed "emphasis" face to highlight the first -- not the second
> -- character. I've argued before that no loss of explanatory power would
> materialize from this.

No loss of explanatory power indeed. The only argument we've made is 
that it'll probably be annoying.

> Finally it also dicounts the fact that yours is an extremely contorted
> example.  I almost never want completions for stuff that contains or
> starts with a specific single character.  Flex/substring becomes really
> useful when you type 2-4 characters, and you are made visually aware of
> the subsequence they matched within the candidate.

But okay, should those really be emphasized more than "first 
difference"? The latter tells the user what they might want to type 
next. The former is mostly useful to see "why am I getting these 
completions at all" or they're bored and want to study the completion 
logic or something. In other words, less frequent situations than simply 
typing code.

Re: BIKESHED: completion faces

[Stefan Monnier]

>> completion-first-difference is at the very least a misnomer for
>> other types of completion, because with flex there can be infinitely
>> many "first" differences.
> Since we've established that its name isn't accurate anymore, it could use
> a rename, yes.

FWIW, I changed its docstring to better reflect what it does.
I'm not interested in renaming it (it might require changes
in codes), but if someone else is, maybe the new docstring can give them
an idea how to name it (or how not to, as the case may be ;-0)


        Stefan

Re: BIKESHED: completion faces

[Stefan Monnier]

> It seems inevitable to have a different set of faces for different
> completion styles.

But since different styles can be used at different times, because of
a slightly different pattern, I think we should try to avoid it, despite
its being "inevitable".  Also I'm playing with a "multi" completion
style which internally combines the output of several other
completion styles, so it could lead to really odd result if they don't
use the same face-scheme.

>> where `re-builder` would be highlighted with a "trivial common part"
>> whereas `rename-buffer` would be highlighted differently.
>>
>> You might consider it as a feature, of course.  But if not, then we'd
>> have to first look at all the completions to make sure they all have
>> a "trivial common part".
>
> This really looks like a feature - it could help the users
> to see the prefix in the completions list.

Not sure what you mean here.


        Stefan

Re: BIKESHED: completion faces

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:
> On 05.11.2019 18:31, João Távora wrote:
>
>> I would rather say that setting a change to Emacs defaults as a bar to
>> clear is counter-productive.  And frequently demotivational.
>
> Emacs has a problem of trying to accommodate too many workflows in the
> same UI. The result sometimes turns out to not be optimal for any of 
> them. I'm trying to avoid this problem here.
>
>> Best solutions arise when we can understand exactly what it is that
>> people like about current defaults, and can accomodate that specific
>> preference in a new system, so that they barely notice it.
>
> That sounds like a change in defaults, though.

Yes, but much more controlled.  This particular change in defaults so is
done so that the explanatory part that people (presumably) like about
the current 'basic' match highlighting is still preserved.

> Your proposal would add a different styling for different completion
> styles. That would require some code as well, likely a similar amount.

No, we are not talking about the same thing.  In my latest proposal, two
faces are renamed, obsolete alias are created, and
completion-pcm--hilit-commonality is trivially changed to use
'completions-emphasis' instead of 'completions-common'.

>>  > > The former part can be improved in flex, the latter can't: it's
>>  > > intrinsic to the technique.
>>  > All can be improved, just with varying degrees of difficulty.
>> Sure, a pig and a large enough rocket...
>
> Is that because the current completion system is not optimal for flex?

No, no. I just lightheartedly commented that in response to your "all
can be improved".  Like "all animals can fly, even pigs, provided you
attach a large enough rocket".

> algorithm itself, but any subsequent bottlenecks would not be triggered.
>
> This approach really relies on good and fast scoring, though.

If we're going to do extensive changes in the name of performance, isn't
it better to use Daniel's generator.el library?  It sounds like just the
thing.

>>  > One just has to make sure not to cache the result improperly.
>> Cache invalidation, one of the "hard" problems in computing.  Buying
>> trouble, I say.
> Let us not forget that we're competing with other editors who
> routinely show off flex matching and somehow deal with accompanying
> performance problems.

Possibly/probably by using delayed evaluation techniques.

>>  > As you can imagine, IMHO this part "making sense" is less important than
>>  > the consistency in highlighting.
>> It's only "inconsistent" if you you refuse to accept that concepts
>> such
>> as "relevance" or "emphasis" are more important the specifics of the
>> matching technique implemented.
> I'm more interested in highlighting being consistent and relevant to
> whatever the next action the user should perform.

And that's cool, I maintain that this isn't broken at all by my
proposal.  Can you explain how it would be?

>>  > > What I'm proposing is no different from say, mode-line-emphasis, which
>>  > > lisp/man.el and lisp/vc/vc-dispatcher.el use in "two wildly different
>>  > > things".
>>  > Here's an example. When the input is one char, how will you figure out
>>  > what the highlighting in the *Completions* buffer means?
>> OK, let's take that example.  It depends on whether I'm using a
>> "prefix"
>> or "flex" style.  But even if I'm using them both together, or I'm such
>> a doofus that I don't remember anymore:
>> 1. If I see that "one char" being matched in the middle of a lot of
>> completions, I know what it means: I'm using 'flex' and/or 'substring'
>> and that is the character that's matched.
>
> That implies an extra thinking step. Whatever synapses take part it
> that, my brain could've used for something else.
>
>> 2. If I see that every completion has its second character highlighted,
>> I know I'm using prefix.
>
> _Probably_, but you won't know for sure until you also read those
> characters. There could be just 2-3 completions anyway, so you
> wouldn't get the picture at a glance.

I don't aggree with your quantification of useful/wasful sinapses,
obviously.  And it seems we are confusing two things here (and I may be
at fault, too).  One thing is "discovering what style Emacs is using"
the other thing is "why is this particular style showing me this visual
feedback".  The first can be solved by other means (by hinting to
temporarily in the mode-line for example).

>> This is discounting the fact that I probably want to know and control
>> whether I'm using flex or prefix anyway.
>
> But you probably won't. Or other users won't. That's the idea behind
> completion-styles anyway. If you are the "controlling" type anyway,
> you would probably customize faces anyway.

No.  I think it's unreasonable to ask that.

> And here's also the argument for waiting until the default changes:
> because if the new default is (setq completion-styles '(flex)), then
> the dilemma we have before us now is easily dismissed.

We agree, totally, but that's not going to happen anytime soon.  So we
should make sure that whoever does that 'setq' or customizes that
variable doesn't get a strange "alien" experience.

>> It's also discounting the fact that in 2. prefix completion should, IMO,
>> use my proposed "emphasis" face to highlight the first -- not the second
>> -- character. I've argued before that no loss of explanatory power would
>> materialize from this.
> No loss of explanatory power indeed. The only argument we've made is
> that it'll probably be annoying.

Why annoying?

>> Finally it also dicounts the fact that yours is an extremely contorted
>> example.  I almost never want completions for stuff that contains or
>> starts with a specific single character.  Flex/substring becomes really
>> useful when you type 2-4 characters, and you are made visually aware of
>> the subsequence they matched within the candidate.
>
> But okay, should those really be emphasized more than "first
> difference"? The latter tells the user what they might want to type 
> next. The former is mostly useful to see "why am I getting these
> completions at all" or they're bored and want to study the completion 
> logic or something. In other words, less frequent situations than
> simply typing code.

Every 'grep' search also brings that visual explanation of the matches.
So does every search engine I've tried.  Why shouldn't 'flex' or 'p-c-m'
or 'string'?  We've seen how all other editors, except Vim (apparently),
highlight them by default.

João

Re: BIKESHED: completion faces

[Stefan Monnier]

>> where `re-builder` would be highlighted with a "trivial common part"
>                      ^ won't be                   ^nontrivial

[ same difference ;-) ]

>> whereas `rename-buffer` would be highlighted differently.
> Indeed. That's the whole idea. "re" and "-b" in "rename-buffer" would indeed
> be highlighted since the match is nontrivial.
>> You might consider it as a feature, of course.
> Yes. That's the idea.

I'm wondering why you think it's a feature.  Could you explain?

The way I see it, it in 99% of the cases it would just work identically
on all displayed completions (sometimes using trivial for all and
sometimes using nontrivial for all), and in the remaining cases it would
use a mix and my natural reaction would likely be "M-x report-emacs-bug".


        Stefan

Re: BIKESHED: completion faces

[Stefan Monnier]

> But as for "outside"... I was thinking of a data structure where you would
> accumulate completions in a sorted tree, of sorts. And, when a new one is
> added, if the maximum allowed number of completions has been reached, the
> least-scored one is evicted. Or we do the eviction at the end, whatever.

Sounds like a good idea, but can we move the discussion of flex's
performance to some other thread?


        Stefan

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 06.11.2019 1:11, João Távora wrote:

>>> Best solutions arise when we can understand exactly what it is that
>>> people like about current defaults, and can accomodate that specific
>>> preference in a new system, so that they barely notice it.
>>
>> That sounds like a change in defaults, though.
> 
> Yes, but much more controlled.  This particular change in defaults so is
> done so that the explanatory part that people (presumably) like about
> the current 'basic' match highlighting is still preserved.

I might have missed what you are referring to here exactly.

>> Your proposal would add a different styling for different completion
>> styles. That would require some code as well, likely a similar amount.
> 
> No, we are not talking about the same thing.  In my latest proposal, two
> faces are renamed, obsolete alias are created, and
> completion-pcm--hilit-commonality is trivially changed to use
> 'completions-emphasis' instead of 'completions-common'.

OK.

Renames aside, you'll have completions-common use bold, right? And hide 
first-difference.

The first one will result in long annoying columns with prefix-only 
completion (this won't happen in other editors because a) they use flex, 
b) popup is limited in height), the second one will remove a bit of 
extra information.

>>>   > > The former part can be improved in flex, the latter can't: it's
>>>   > > intrinsic to the technique.
>>>   > All can be improved, just with varying degrees of difficulty.
>>> Sure, a pig and a large enough rocket...
>>
>> Is that because the current completion system is not optimal for flex?
> 
> No, no. I just lightheartedly commented that in response to your "all
> can be improved".  Like "all animals can fly, even pigs, provided you
> attach a large enough rocket".

That kind of discounts the valid and useful avenues for improvement we 
still have.

>> algorithm itself, but any subsequent bottlenecks would not be triggered.
>>
>> This approach really relies on good and fast scoring, though.
> 
> If we're going to do extensive changes in the name of performance, isn't
> it better to use Daniel's generator.el library?  It sounds like just the
> thing.

Last I checked, it's not very relevant. Or if it is, it'll just be a 
minor implementation detail.

>>>   > One just has to make sure not to cache the result improperly.
>>> Cache invalidation, one of the "hard" problems in computing.  Buying
>>> trouble, I say.
>> Let us not forget that we're competing with other editors who
>> routinely show off flex matching and somehow deal with accompanying
>> performance problems.
> 
> Possibly/probably by using delayed evaluation techniques.

My limiting the number of completions, most likely. And/or doing it all 
in C++/Java.

>>>   > As you can imagine, IMHO this part "making sense" is less important than
>>>   > the consistency in highlighting.
>>> It's only "inconsistent" if you you refuse to accept that concepts
>>> such
>>> as "relevance" or "emphasis" are more important the specifics of the
>>> matching technique implemented.
>> I'm more interested in highlighting being consistent and relevant to
>> whatever the next action the user should perform.
> 
> And that's cool, I maintain that this isn't broken at all by my
> proposal.  Can you explain how it would be?

Hiding first-difference will lose some info when suffix is non-empty.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 06.11.2019 2:18, Stefan Monnier wrote:

>>> whereas `rename-buffer` would be highlighted differently.
>> Indeed. That's the whole idea. "re" and "-b" in "rename-buffer" would indeed
>> be highlighted since the match is nontrivial.
>>> You might consider it as a feature, of course.
>> Yes. That's the idea.
> 
> I'm wondering why you think it's a feature.  Could you explain?

It only adds extra info to completions that can really use it. Thus no 
extra highlighting for prefix-only completions.

> The way I see it, it in 99% of the cases it would just work identically
> on all displayed completions (sometimes using trivial for all and
> sometimes using nontrivial for all), and in the remaining cases it would
> use a mix and my natural reaction would likely be "M-x report-emacs-bug".

Sometimes seeing a mix might be surprising, I guess, but I'll have to 
try using it in practice to form some practical opinion on the subject.

I don't have a better suggestion, though. Different highlightings for 
different completions styles doesn't sound user-friendly to me, and will 
suffer from a similar problem anyway when completion-styles has more 
than 1 entry.

We can still fall back to subtle blue foreground for common-part. *shrug*

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 3535 bytes --]

On Wed, Nov 6, 2019 at 8:18 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
> OK.
>
> Renames aside, you'll have completions-common use bold, right? And hide
> first-difference.

No. Completions-first-difference is renamed completion-emphasis and
completions-common is renamed completion-shadow or
completions-secondary-emphasis. Then aliases are made. Then
completion-pcm-hilit-commonality starts using completion-emphasis
for the matching parts.

> The first one will result in long annoying columns with prefix-only
> completion (this won't happen in other editors because a) they use flex,
> b) popup is limited in height), the second one will remove a bit of
> extra information.

I don't understand this part but I think this doesn't apply
since you misunderstood.

> >>>   > > The former part can be improved in flex, the latter can't: it's
> >>>   > > intrinsic to the technique.
> >>>   > All can be improved, just with varying degrees of difficulty.
> >>> Sure, a pig and a large enough rocket...
> >>
> >> Is that because the current completion system is not optimal for flex?
> >
> > No, no. I just lightheartedly commented that in response to your "all
> > can be improved".  Like "all animals can fly, even pigs, provided you
> > attach a large enough rocket".
>
> That kind of discounts the valid and useful avenues for improvement we
> still have.

Sure, it was lighthearted.  I was just making a point before that making
flex match "less" isn't possible.  But feel free to make faster lists
and such to improve it from the "outside".

> > If we're going to do extensive changes in the name of performance, isn't
> > it better to use Daniel's generator.el library?  It sounds like just the
> > thing.
> Last I checked, it's not very relevant. Or if it is, it'll just be a
> minor implementation detail.

It's a good approach at delayed evaluation.  You could hand-code it, but
the patterns and the accessors you need are already solidly there.
It doesn't magically code the generator for you, if that's what you were
expeecting. :-)

> > Possibly/probably by using delayed evaluation techniques.
>
> My limiting the number of completions, most likely.

Really? Then they suck. But hey, that's easy to do in Emacs, too :-)

> And/or doing it all in C++/Java.

We've seen elsewhere those speedups are relevant, but not mind-blowing.
Even if the speedup is 10x it's easy to switch to a project
that has 100x the symbols.

And "doing it all" is a a poor approach at optimization.  We should
optimize the hotspots.  And we can do that in Emacs, too.

BTW Java has generators too. And so does recent C++.

> >>>   > As you can imagine, IMHO this part "making sense" is less
important than
> >>>   > the consistency in highlighting.
> >>> It's only "inconsistent" if you you refuse to accept that concepts
> >>> such
> >>> as "relevance" or "emphasis" are more important the specifics of the
> >>> matching technique implemented.
> >> I'm more interested in highlighting being consistent and relevant to
> >> whatever the next action the user should perform.
> >
> > And that's cool, I maintain that this isn't broken at all by my
> > proposal.  Can you explain how it would be?
>
> Hiding first-difference will lose some info when suffix is non-empty.

That's only if you don't (also trivially) change 'basic/prefix' to use
'completion-emphasis' for the common part and my
completion-secondary-emphasis for those
situations.

--
João Távora

[-- Attachment #2: Type: text/html, Size: 4713 bytes --]

Re: BIKESHED: completion faces

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> Yes, but much more controlled.  This particular change in defaults
>> so is
>> done so that the explanatory part that people (presumably) like about
>> the current 'basic' match highlighting is still preserved.
> I might have missed what you are referring to here exactly.

And I missed this comment in my last reply.  Let me summarize, if I
can.  My proposal has two parts, the second one is optional.  I propose:

Part 1: Face renaming so that each completion style is relatively free
to emphasize the things that it decides need emphasizing.
completions-first-difference is renamed completions-emphasis and
completions-common is renamed completions-secondary-emphasis.  Default
values absolutely unchanged.  completion-pcm--hilit-commonality starts
using completions-emphasis.

Current users of 'basic' would see no difference whatsoever.  Current
users of pcm-based styles would see a defaults change, but there are not
that many, I think.  If I'm wrong, we restrict this change just to
'flex' by changing just completion-flex-all-completions.

Part 2: A "controlled" defaults change whereby the 'basic' style starts using
completions-emphasis where it used to use completions-common.  And
completions-emphasis-secondary where it used to use
completions-first-difference.

This second steps yields a moderate gain in "consistency" to other
editors that also use prefix-matching somehow.

>> Possibly/probably by using delayed evaluation techniques.
> My limiting the number of completions, most likely.

I also want to review my previous comment to this.  I said those editors
"suck", but that is totally gratuitous since I have no idea exactly what
they do.  If limiting yields performance increaase, then of course we
should do it.  In fact I use this in SLY, when I have to send
completions through the wire.  But I don't in any way limit the set of
symbols from which completions can be collected.  This set has to be
explored fully, because the "best" match might lurk at the very
boundaries or the set.

João

VOTE: Changing completions-common-part face's default

[Stefan Monnier]

> We can still fall back to subtle blue foreground for common-part. *shrug*

I'm more and more leaning this way, yes.

I don't think we'll find an agreement on what is the best default
setting, so I'll settle for a default setting that everyone can agree is
"not objectionably worse" than the status quo.

So, here's my suggestion: we change completions-common-part by making it
use a `blue3` foreground when the background is light.

Does anyone object to this change?


        Stefan

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 06.11.2019 12:25, João Távora wrote:

 > My proposal has two parts, the second one is optional

OK, thank you. I'm sorry to say, it's too convoluted for my liking. But 
others' opinions are welcome.

> This second steps yields a moderate gain in "consistency" to other
> editors that also use prefix-matching somehow.

Speaking of other editors, the usual way to pick a completion there is 
either to use arrow keys then press RET, or to press TAB several times.

In completion-at-point, the user needs to type a letter to disambiguate. 
So completions-first-difference, or whatever it should be called, is 
more important here.

> I also want to review my previous comment to this.  I said those editors
> "suck", but that is totally gratuitous since I have no idea exactly what
> they do.  If limiting yields performance increaase, then of course we
> should do it.  In fact I use this in SLY, when I have to send
> completions through the wire.  But I don't in any way limit the set of
> symbols from which completions can be collected.  This set has to be
> explored fully, because the "best" match might lurk at the very
> boundaries or the set.

Right. I think I've made that distinction when describing the 
"sorted/scored tree of completions" idea.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 06.11.2019 10:53, João Távora wrote:

>  > The first one will result in long annoying columns with prefix-only
>  > completion (this won't happen in other editors because a) they use flex,
>  > b) popup is limited in height), the second one will remove a bit of
>  > extra information.
> 
> I don't understand this part but I think this doesn't apply
> since you misunderstood.

I think it applies after you second step. And after just the first one, 
we'll have inconsistency between the styles.

>  > > If we're going to do extensive changes in the name of performance, 
> isn't
>  > > it better to use Daniel's generator.el library?  It sounds like 
> just the
>  > > thing.
>  > Last I checked, it's not very relevant. Or if it is, it'll just be a
>  > minor implementation detail.
> 
> It's a good approach at delayed evaluation.  You could hand-code it, but
> the patterns and the accessors you need are already solidly there.
> It doesn't magically code the generator for you, if that's what you were
> expeecting. :-)

I'm saying writing a generator is largely irrelevant (especially if 
completions come over the network). And all completion frontends do some 
sort of sorting, which usually means they will try to realize the whole 
lazy sequence up front.

>  > > Possibly/probably by using delayed evaluation techniques.
>  >
>  > My limiting the number of completions, most likely.
> 
> Really? Then they suck. But hey, that's easy to do in Emacs, too :-)

Easy in a way that would minimally hinder the user?

>  > And/or doing it all in C++/Java.
> 
> We've seen elsewhere those speedups are relevant, but not mind-blowing.
> Even if the speedup is 10x it's easy to switch to a project
> that has 100x the symbols.

I don't think this puts the right emphasis on things.

Every approach has a limit. But with a 10x more performant/compact 
system, one could handle projects 10x the size.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 3335 bytes --]

On Wed, Nov 6, 2019 at 3:11 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> OK, thank you. I'm sorry to say, it's too convoluted for my liking. But
> others' opinions are welcome.

Really, you think the patch below is "convoluted"?

João?

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 43dd277a2e..c202863ee0 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -1682,14 +1682,19 @@ completion-setup-hook
 The completion list buffer is available as the value of `standard-output'.
 See also `display-completion-list'.")

-(defface completions-first-difference
+(defface completions-emphasis
   '((t (:inherit bold)))
-  "Face for the first character after point in completions.
-See also the face `completions-common-part'.")
+  "Primary face for highlighting important parts of completions.
+See also the face `completions-secondary-emphasis'.")

-(defface completions-common-part '((t nil))
-  "Face for the parts of completions which matched the pattern.
-See also the face `completions-first-difference'.")
+(defface completions-secondary-emphasis '((t nil))
+  "Secondary face for highlighting important parts of completions.
+See also the face `completions-emphasis'.")
+
+(define-obsolete-face-alias
+  'completions-first-difference 'completion-emphasis "27.1")
+(define-obsolete-face-alias
+  'completions-common-part 'completion-secondary-emphasis "27.1")

 (defun completion-hilit-commonality (completions prefix-len &optional
base-size)
   "Apply font-lock highlighting to a list of completions, COMPLETIONS.
@@ -1721,11 +1726,11 @@ completion-hilit-commonality
              ;; values, all-completions may return strings
              ;; that don't contain the prefix.
              (min com-str-len (length str))
-             'face 'completions-common-part str)
+             'face 'completions-secondary-emphasis str)
             (if (> (length str) com-str-len)
                 (font-lock-prepend-text-property com-str-len (1+
com-str-len)
                                                  'face
-
'completions-first-difference
+                                                 'completions-emphasis
                                                  str)))
           elem)
         completions)
@@ -3145,16 +3150,16 @@ completion-pcm--hilit-commonality
            (while md
              (funcall update-score start (car md))
              (put-text-property start (pop md)
-                                'font-lock-face 'completions-common-part
+                                'font-lock-face 'completions-emphasis
                                 str)
              (setq start (pop md)))
            (funcall update-score len len)
            (put-text-property start end
-                              'font-lock-face 'completions-common-part
+                              'font-lock-face 'completions-emphasis
                               str)
            (if (> (length str) pos)
                (put-text-property pos (1+ pos)
-                                  'font-lock-face
'completions-first-difference
+                                  'font-lock-face
'completions-secondary-emphasis
                                   str))
            (unless (zerop (length str))
              (put-text-property

[-- Attachment #2: Type: text/html, Size: 4275 bytes --]

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 06.11.2019 17:31, João Távora wrote:
> Really, you think the patch below is "convoluted"?

Yes.

Not the number of lines, but the essence: renaming faces, as well as 
swapping them in certain conditions.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 411 bytes --]

On Wed, Nov 6, 2019 at 3:43 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 06.11.2019 17:31, João Távora wrote:
> > Really, you think the patch below is "convoluted"?
>
> Yes.
>
> Not the number of lines, but the essence: renaming faces, as well as
> swapping them in certain conditions.
>

It's not "in certain conditions". There is not a single "if" in my patch.
It's totally linear.

João

[-- Attachment #2: Type: text/html, Size: 803 bytes --]

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1634 bytes --]

On Wed, Nov 6, 2019 at 3:16 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 06.11.2019 10:53, João Távora wrote:
>
> >  > The first one will result in long annoying columns with prefix-only
> >  > completion (this won't happen in other editors because a) they use
flex,
> >  > b) popup is limited in height), the second one will remove a bit of
> >  > extra information.
> >
> > I don't understand this part but I think this doesn't apply
> > since you misunderstood.
>
> I think it applies after you second step. And after just the first one,
> we'll have inconsistency between the styles.

Depends on what you call consistent.  It's up to us to decide.  The current
state is horribly inconsistent, to me at least.  All other editors except
Vim
(I've now checked Atom, VsCode, Sublime, Textmate, and even these
super-fancy new things like https://playcode.io/) highlight flex patterns
bold.  So does helm and ivy and whatnot.  Only emacs seems intent
on this first-difference business that doesn't make a lot of sense for flex.
But I'm not even proposing taking that out, just giving it another name.

> I'm saying writing a generator is largely irrelevant (especially if
> completions come over the network).

It's irrelevant when calculating completions, yes, but not when
say, displaying them chunks at a time, (I thought you told me
the slowness you were experiencing was somehow related to
*Completions* and completion-at-point). Nor when applying a
bunch of properties, say, faces to a very big list. I tried to
explain this when I told you "we have to think about this slowness".

João

[-- Attachment #2: Type: text/html, Size: 2117 bytes --]

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 06.11.2019 17:48, João Távora wrote:
> On Wed, Nov 6, 2019 at 3:43 PM Dmitry Gutov <dgutov@yandex.ru 
> <mailto:dgutov@yandex.ru>> wrote:
> 
>     On 06.11.2019 17:31, João Távora wrote:
>      > Really, you think the patch below is "convoluted"?
> 
>     Yes.
> 
>     Not the number of lines, but the essence: renaming faces, as well as
>     swapping them in certain conditions.
> 
> 
> It's not "in certain conditions". There is not a single "if" in my patch.
> It's totally linear.

You don't have to mention "if" in a diff to introduce different behaviors.

In one function you replace completions-common-part with 
completions-secondary-emphasis (its new alias). In another function, you 
replace it with completions-emphasis.

And a similar thing with the other face.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: monnier@iro.umontreal.ca,  dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Tue, 05 Nov 2019 21:43:42 +0000
> 
> > If we have one efficient way, why do we need to consider others?
> 
> Indeed we don't, you're totally right, not for the 'prefix/basic'
> completion.  However some people have made a point that there should be
> some kind of consistency at this level between completion styles, that
> the thing emphasized in 'prefix/basic' should have some semantic
> relation to the thing emphasized thing for 'flex' and 'substring' too.
> 
> So, to "please greeks and trojans" [1], an equally efficient AND
> cross-style-consistent style could be found, we could keep the "common"
> and "first-difference" face names, and 'flex' would ship without a
> default "crippling" (see below)..

Sorry, but I see no reason for any kind of "consistency" here.  We
need to highlight to help the user specify the right completion
candidate, all the rest is secondary IMO.

> >> One of them is to highlight the preceding character.
> >
> > ??? How does this help me to select what to type next?
> 
> Did you try the experiment I proposed (swap the two faces)?  I have no
> problem recognizing the character "to type next" when I do it.  See the
> attached gif another-way-to-see-the-first-difference.gif

I did try that, and it makes the task of finding the next character to
type much harder for me.

> > I don't see why it's important to explain how did the completion
> > algorithm arrive at a particular candidate.  The completion algorithm
> > is there to intuit what we mean in the most efficient way, but the
> > details of how it does that are immaterial.  The only ones who may be
> > interested are those who study completion algorithms ;-)
> 
> I may sound like a completion scholar to you, but you also sound like
> you haven't experimented with 'flex' for more than 1 second, otherwise
> you wouldn't be asking that question ;-)

There's no need to make such assumptions just because I happen to
disagree with something you consider self-evident.  I did play with
that long enough to make up my mind.

> So I attach another Emacs -Q gif, crippled-flex.gif, where you see
> the current problem, and yet another gif, useful-flex.gif, where the
> matching part is highlighted bold.

Thanks, but it doesn't change my mind.

> I think you will agree it's not a detail.  The reason we want to
> highlight matching parts in flex is the same reason grep and every
> search tool and search engine I know decides to highlight matching
> parts: to call attention to the part that matched.

Grep's goal is to find the match, so it makes sense to highlight that
match.  The goal here is quite different, so highlighting the match
makes much less sense in this context.

> Of course, fixing that crippled default state of 'flex' is a couple of
> customizations away (Put the common face to 'bold' and the
> first-difference to nothing).  But, IMHO, it would be a shame if we were
> to release Emacs 27 with this familar matching method and no good
> default faces to go with it.

If you are saying that the default face needs to be changed, it is a
completely different issue.  This thread AFAIU is mainly about
defining new faces that are put on other parts of the candidate
strings.  It is not about highlighting the same parts, but with
different colors or other attributes.

> > No, "first difference" is always the character to be typed after
> > point.  At least for the vastly important case that point is at the
> > end of what you typed, i.e. you don't move point back after typing
> > something.
> 
> But for the 'flex' case (among others more obscure) that first character
> "to be typed" -- presumably to narrow down the set further -- might be
> any character following 'point', for each completion, not just the one
> in the 'point+1' position.

So show all of them ,with the same face.  That's not what this thread
started with, AFAIU, it started with an assertion that
completion-first-difference is flawed as a concept, and should be
replaced with something new and different.

> Finally, I believe other UI's that have this kind of flex search (I
> think they are not rare at all nowadays, not just in editors) also use a
> prominent (often bold) face, to mark the common part.

That's a different issue.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 6 Nov 2019 10:18:42 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>  emacs-devel <emacs-devel@gnu.org>
> 
> Renames aside, you'll have completions-common use bold, right? And hide 
> first-difference.

FWIW, I don't think I'd like this change, it makes the completion much
less efficient for me.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 621 bytes --]

On Wed, Nov 6, 2019 at 4:12 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Dmitry Gutov <dgutov@yandex.ru>
> > Date: Wed, 6 Nov 2019 10:18:42 +0200
> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
> >  emacs-devel <emacs-devel@gnu.org>
> >
> > Renames aside, you'll have completions-common use bold, right? And hide
> > first-difference.
>
> FWIW, I don't think I'd like this change, it makes the completion much
> less efficient for me.
>

Dmitry assumed I wanted to do that in the "renames" proposal. I don't. The
"renames" proposal (the one I added a patch for) avoids this.

-- 
João Távora

[-- Attachment #2: Type: text/html, Size: 1254 bytes --]

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 6 Nov 2019 17:11:01 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>  emacs-devel <emacs-devel@gnu.org>
> 
> In completion-at-point, the user needs to type a letter to disambiguate. 
> So completions-first-difference, or whatever it should be called, is 
> more important here.

Full agreement.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

On Wed, Nov 6, 2019 at 4:03 PM Eli Zaretskii <eliz@gnu.org> wrote:

> So show all of them ,with the same face.  That's not what this thread
> started with, AFAIU, it started with an assertion that
> completion-first-difference is flawed as a concept, and should be
> replaced with something new and different.

Threads are allowed to change scope, right? And this is a pretty minimal
scope change, ff one at all.  Let me ask you simply: Do you agree that
the new 'flex' style should -- by default -- mark the matched pattern
strongly
and  prominently (at least as prominently as completion-first-difference),
so that the user gets help in choosing the completion, in much the
same way as he gets such help in VScode, Sublime, Atom, Helm, Ivy,
etc.?

Some of your statements make me think you agree with this, and some
make me think that you don't.

If you are opposed to this, and you can't be convinced, we agree
to disagree and I'm going to go do something else. If you are for it,
I think we've collected some alternatives that do that, each with
advantages and drawbacks.

João

[-- Attachment #2: Type: text/html, Size: 1461 bytes --]

Re: VOTE: Changing completions-common-part face's default

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 514 bytes --]

> > We can still fall back to subtle blue foreground for common-part.
*shrug*
> I'm more and more leaning this way, yes.
> I don't think we'll find an agreement on what is the best default
> setting, so I'll settle for a default setting that everyone can agree is
> "not objectionably worse" than the status quo.

Still pretty bad, tho. Count me as "abstain".

> Does anyone object to this change?

Can't it be cyan or cyan 3 or something that? Dark blue against black
has terrible contrast.

João

[-- Attachment #2: Type: text/html, Size: 739 bytes --]

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 06.11.2019 19:16, João Távora wrote:
>  > > We can still fall back to subtle blue foreground for common-part. 
> *shrug*
>  > I'm more and more leaning this way, yes.
>  > I don't think we'll find an agreement on what is the best default
>  > setting, so I'll settle for a default setting that everyone can agree is
>  > "not objectionably worse" than the status quo.
> 
> Still pretty bad, tho. Count me as "abstain".

Honestly, I'd rather use blue4 even. blue3 is very noticeable.

>  > Does anyone object to this change?
> 
> Can't it be cyan or cyan 3 or something that? Dark blue against black
> has terrible contrast.

It needs to contrast against the background as the first priority.

Are you using a dark theme? Cyan looks entirely inappropriate over here.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Wed, 6 Nov 2019 16:26:45 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>, 
> 	emacs-devel <emacs-devel@gnu.org>
> 
> Do you agree that the new 'flex' style should -- by default -- mark
> the matched pattern strongly and prominently (at least as
> prominently as completion-first-difference), so that the user gets
> help in choosing the completion, in much the same way as he gets
> such help in VScode, Sublime, Atom, Helm, Ivy, etc.?

No, I don't agree.  Highlighting the matched pattern in the context of
completion makes very little sense to me.  It is immaterial, at least
when I review the candidates to find the one(s) I'm after.

> If you are opposed to this, and you can't be convinced, we agree
> to disagree and I'm going to go do something else. If you are for it, 
> I think we've collected some alternatives that do that, each with 
> advantages and drawbacks.

If enough people want this, we could make it an optional behavior.  I
don't want to dictate to others what are their preferences.

Re: VOTE: Changing completions-common-part face's default

[João Távora]

[-- Attachment #1.1: Type: text/plain, Size: 1152 bytes --]

On Wed, Nov 6, 2019 at 5:26 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

>
> >  > Does anyone object to this change?
> >
> > Can't it be cyan or cyan 3 or something that? Dark blue against black
> > has terrible contrast.
>
> It needs to contrast against the background as the first priority.
>

I agree, and cyan 3 does that, IMO. Or cyan4 if you prefer.  Or cyan.

But curiously, until very recently, the recommendation for this face, as
stated in
this docstring, is that you set it to shadow, i.e. that you set it to
something that
_doesn't_ contrast  against the background.   That is presumably for
'prefix'
users so that they can focus on 'first-difference' even more.

This recommendation has been removed recently (a good change, imo).

Are you using a dark theme? Cyan looks entirely inappropriate over here.
>

I'm not. I'm using the default. We're talking about defaults, right?

In a dark MinTTY emulator with "xterm" defaults and Emacs -Q defaults,
cyan is pretty readable.  It's the same as minibuffer prompt and many,
many other faces.

I attach a screenshot, substring-completing "comp".

João Távora

[-- Attachment #1.2: Type: text/html, Size: 1938 bytes --]

[-- Attachment #2: cyan.png --]
[-- Type: image/png, Size: 11145 bytes --]

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

On Wed, Nov 6, 2019 at 6:07 PM Eli Zaretskii <eliz@gnu.org> wrote:

> If enough people want this, we could make it an optional behavior.  I
>

No need. It is already optional behavior. This thread discusses
a (minimal) change to the defaults from the beginning, so that
new users coming from any of those other editors are less confused
by this particularity of Emacs when choosing 'flex' completion.

João Távora

[-- Attachment #2: Type: text/html, Size: 809 bytes --]

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Wed, 6 Nov 2019 18:14:42 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>, 
> 	emacs-devel <emacs-devel@gnu.org>
> 
>  If enough people want this, we could make it an optional behavior.  I
> 
> No need. It is already optional behavior. This thread discusses 
> a (minimal) change to the defaults from the beginning, so that 
> new users coming from any of those other editors are less confused
> by this particularity of Emacs when choosing 'flex' completion.

If you want to make the matched parts prominent by default (and I
guess make the first-different character less prominent), then you
will have to try much harder in order to convince me.  Just saying
that newbies who come from other editors will want that is not enough.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1296 bytes --]

On Wed, Nov 6, 2019 at 6:29 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> If you want to make the matched parts prominent by default

I do. As I believe do some other people.

> (and I guess make the first-different character less prominent),

No, I had already given up that part for 'basic'. As you can read
in my latest "renames" patch which affects just 'flex'.

> then you will have to try much harder in order to convince me.  Just
saying
> that newbies who come from other editors will want that is not enough.

In the end, this is about UI usability and at least I don't have
the scientific credentials to prove to you that this is more
useful. And I doubt these credentials exist. It's very hard
to prove to you specifically that it is useful intrinsically, even
more so because you are not a regular user of this feature.

So I simply point to the vast ecosystem outside Emacs
where a particular choice has been made for some time
and it has stuck. I do that as an argument of authority,
inherently weaker than other arguments types, but it's the
best one, I think.  If you decide not to consider it, that's fine.

Also, dismissing emacs newcomers expectations is a bad
strategy for this project. But that's an entirely different
discussion.

João Távora

[-- Attachment #2: Type: text/html, Size: 1500 bytes --]

RE: BIKESHED: completion faces

[Drew Adams]

[-- Attachment #1: Type: text/plain, Size: 1904 bytes --]

> > mark the matched pattern strongly and prominently ... so
> > that the user gets help in choosing the completion, in
> > much the same way as he gets such help in VScode, Sublime,
> > Atom, Helm, Ivy, etc.?
> 
> No, I don't agree.  Highlighting the matched pattern in the context of
> completion makes very little sense to me.  It is immaterial, at least
> when I review the candidates to find the one(s) I'm after.

FWIW -

I believe Icicles was the first to do this in Emacs
(in 2006).  Can't speak for VScode, Sublime, or
Atom, though.

Showing the matches for your current minibuffer
input is not only about "finding the one" you're
after.  (By "you" I mean "one", here.)

Completion is not just about choosing a completion
candidate - at least it _need not_ be only about
that. It's also about exploring a set - the set of
things that match your current input.

And when there are different ways to match (via
`completion-styles' or otherwise), showing just
how each candidate matches can help.

Among other things, knowing what & how your input
matches can help you adjust it to match different
sets of candidates. Think of regexp matching, for
example - it can be complex, and a slight change
in the pattern can have considerable effect.

Icicles highlights matches for your input with
different faces.  See attached screenshot.

 * Direct match ("mini" in screenshot)
 * Expanded common match - common to all matches
   ("minibuffer" in screenshot)

And Icicles doesn't bother with the vanilla faces
`completions-first-difference' and
`completions-common-part'.

(So do I care about this thread, as far as my
development is concerned?  No, except that changes
to vanilla Emacs might mean I need to do more work,
e.g. to avoid even more vanilla highlighting.  Just
speaking to the question of whether showing input
matches can be helpful.)

 

[-- Attachment #2: drew-emacs-icicles.PNG --]
[-- Type: image/png, Size: 49699 bytes --]

Re: BIKESHED: completion faces

[Eli Zaretskii]

> Date: Wed, 6 Nov 2019 12:34:26 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: emacs-devel@gnu.org, monnier@iro.umontreal.ca, dgutov@yandex.ru
> 
> Completion is not just about choosing a completion
> candidate - at least it _need not_ be only about
> that. It's also about exploring a set - the set of
> things that match your current input.

Yes, but I don't see how highlighting what exactly matched is of any
help in that case.  Especially with the weirdo matches some aggressive
completion styles produce.  I care about the candidates and about how
to narrow the list, not about how and why the completion picked up
these particular candidates.

> Among other things, knowing what & how your input
> matches can help you adjust it to match different
> sets of candidates.

That's what the "first difference" highlighting solves.

> Icicles highlights matches for your input with
> different faces.  See attached screenshot.

Sorry, I'm unconvinced.  I cannot think of a situation where such
highlighting would help me.

Re: BIKESHED: completion faces

[Ergus]

On Wed, Nov 06, 2019 at 08:29:05PM +0200, Eli Zaretskii wrote:
>> From: Jo?o T?vora <joaotavora@gmail.com>
>> Date: Wed, 6 Nov 2019 18:14:42 +0000
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>,
>> 	emacs-devel <emacs-devel@gnu.org>
>>
>>  If enough people want this, we could make it an optional behavior.  I
>>
>> No need. It is already optional behavior. This thread discusses
>> a (minimal) change to the defaults from the beginning, so that
>> new users coming from any of those other editors are less confused
>> by this particularity of Emacs when choosing 'flex' completion.
>
>If you want to make the matched parts prominent by default (and I
>guess make the first-different character less prominent), then you
>will have to try much harder in order to convince me.  Just saying
>that newbies who come from other editors will want that is not enough.
>
Hi:

I don't use to come in this kind of discussions because they always end
with too many opinions and too few agreements.

But.

In this kinds of things I agree with Joao. Sometimes we should agree
with what all the other editors do (except when we provide something
much better) for many reasons

1) It is the behaviour that the users expect, so familiar for them and
that will reduce learning curve (we have enough curve for a newbie).

2) We should be modest and think that if most of the other editors
follow a similar approach there should be a reason for that. (also
private companies invest time and money in social market studies about
what option is better... that's why MS windows is blue and have changes
among the years, or the start button is in the left down corner)

3) Sometimes the implemented approach is a consequence of feature
requests from users and developers and in these days they have orders of
magnitude more users than us... so they are statistically more
representative. But also they have tools to easily collect more opinions
from less expert or casual users (like feature hub, the "issues" window
on github and so on) more familiar for new generations of programmers.

4) New users will start asking for ways to reproduce the behaviour they
had in their previous editors, so in the best case we will have feature
requests and many questions about how to reproduce it, and people
writing in the emacswiki, ergoemacs and other foros about how to do what
we didn't for them. But in the worst case, external configs like
ergoemacs or spacemacs will implement code to do it.

Apart from this; personally:

I would prefer to have highlighted the matching parts with a different
colour only and the first unmatch (optional for me) being just
underlined (to be consistent somehow with the toolbar everywhere else
where the underline usually informs the shortcut.)

We don't need to reinvent the wheel about this...

Re: BIKESHED: completion faces

[Juri Linkov]

> Speaking of other editors, the usual way to pick a completion there is
> either to use arrow keys then press RET, or to press TAB several times.

Speaking of using arrow keys and RET, this reminded how completion
works in web browsers.  In both Firefox and Chromium completions
are highlighted in bold, but Chromium doesn't hesitate to use
different highlighting for different completion styles in the
same completions list.  The first part of so called omnibox
displays completions from history/bookmarks using flex completion
style where matched characters are highlighted in bold.  The bottom
part of the same omnibox displays search results using prefix completion
style where bold is used for non-matched parts (i.e. the inverse of
Emacs completions-common-part, i.e. substrings starting from
completions-first-difference are highlighted in bold).

This screenshot demonstrates different completion styles:
https://www.chromestory.com/2018/05/omnibox-redesign/
where "time" in the link from bookmark/history
is highlighted in bold for flex completion style,
but "time" in search results for prefix completion style
is not highlighted.  Instead it highlights the remaining part
to help the user to continue typing the highlighted characters.

RE: BIKESHED: completion faces

[Drew Adams]

> > Completion is not just about choosing a completion
> > candidate - at least it _need not_ be only about
> > that. It's also about exploring a set - the set of
> > things that match your current input.
> 
> Yes, but I don't see how highlighting what exactly matched is of any
> help in that case.  Especially with the weirdo matches some aggressive
> completion styles produce.  I care about the candidates and about how
> to narrow the list, not about how and why the completion picked up
> these particular candidates.

If you care about the set of candidates as a
whole, and are not just trying to "find the
one", or if you're interested in redefining
that set (including possibly narrowing it),
then "what exactly matched" your input can
be very helpful info.

Vanilla Emacs doesn't let you operate on the
set of matching candidates.  And it doesn't
update that set incrementally as you change
your input.  So your not appreciating the
value of what-parts-currently-match feedback
is understandable.

Maybe think about the fact that other UIs
have added similar such behavior, and it's
come up independently.  This wheel has been
discovered multiple times.

At least it was independent in the case of
Icicles, and I'm guessing the same was true
for at least one of VScode, Sublime, and Atom.

It's not (at least in the case of Icicles)
as if such highlighting was present at the
outset.  It came naturally out of using the
different kinds of matching - i.e., out of
practice.

With basic prefix matching, sure, the only
thing you need to look for, to understand
why you get the candidates you do, is the
matching prefix (hence "first difference").

With other kinds of matching it's not always
obvious why this and that are candidates.
Highlighting the parts that match helps.

But I don't imagine I can convince you.

> > Among other things, knowing what & how your input
> > matches can help you adjust it to match different
> > sets of candidates.
> 
> That's what the "first difference" highlighting solves.

No, sorry, it doesn't.  And there's really no
special significance for "first difference".

> > Icicles highlights matches for your input with
> > different faces.  See attached screenshot.
> 
> Sorry, I'm unconvinced.  I cannot think of a
> situation where such highlighting would help me.

Yes, I imagine you can't.  Sorry, but I don't
have the energy, time, or will to try to help
you with that.  Maybe someone else does (or not).

I think one key to understanding is to practice
with different ways of matching.  Another is to
think of operating on the set of matches.

Perhaps, with the latter for example, you can
guess how it can help to know whether all
candidates have a common match part and what
that part is.  Or not.

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 06.11.2019 20:10, João Távora wrote:
> I attach a screenshot, substring-completing "comp"

Thanks. Just as I thought.

You are using Emacs in a terminal, on a dark background. Which are 
special conditions.

Emacs, by default, looks like this: 
https://www.gnu.org/software/emacs/images/teaser.png

Re: VOTE: Changing completions-common-part face's default

[João Távora]

[-- Attachment #1.1: Type: text/plain, Size: 1266 bytes --]

On Wed, Nov 6, 2019 at 10:20 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 06.11.2019 20:10, João Távora wrote:
> > I attach a screenshot, substring-completing "comp"
>
> Thanks. Just as I thought.
>
> You are using Emacs in a terminal, on a dark background. Which are
> special conditions.
>

No Dmitry I'm not. How could I be if I was objecting earlier
about the difference to black text??! You asked if I was
using a dark theme, and I opened one in a terminal to
show you what it looked like.  I thought _you_ were using one.

Do you mean cyan3 or cyan4 are aren't good enough
contrast for the default GUI white background? I think
it is. Here's a screenshot of cyan4, which downgrades
to cyan in those special conditions.

Do you think it's not sufficient contrast to the background?

Anyway, defface has ways to change what the face
looks like depending on the background you have,
the amount of colours you have, etc.

Again, and this is pretty objective, I'm saying blue3 or
blue4 are bad because, while they indeed do fill
the requirement of distinguishing themselves from the
background, they do not distinguish themselves sufficiently
from the default black text.  Which is the whole point of this
discussion.

João

[-- Attachment #1.2: Type: text/html, Size: 1859 bytes --]

[-- Attachment #2: Peek 2019-11-06 22-33.gif --]
[-- Type: image/gif, Size: 53308 bytes --]

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 07.11.2019 0:42, João Távora wrote:

>     You are using Emacs in a terminal, on a dark background. Which are
>     special conditions.
> 
> 
> No Dmitry I'm not. How could I be if I was objecting earlier
> about the difference to black text??!

Ah. Okay.

> You asked if I was
> using a dark theme, and I opened one in a terminal to
> show you what it looked like.  I thought _you_ were using one.

blue3 was proposed specifically for light-background themes.

> Do you mean cyan3 or cyan4 are aren't good enough
> contrast for the default GUI white background?

Yes, that's my impression. Text goes from dark-on-white to light-on-white.

> I think
> it is. Here's a screenshot of cyan4, which downgrades
> to cyan in those special conditions.
> 
> Do you think it's not sufficient contrast to the background?

Yup.

> Anyway, defface has ways to change what the face
> looks like depending on the background you have,
> the amount of colours you have, etc.

Yup.

> Again, and this is pretty objective, I'm saying blue3 or
> blue4 are bad because, while they indeed do fill
> the requirement of distinguishing themselves from the
> background, they do not distinguish themselves sufficiently
> from the default black text.  Which is the whole point of this
> discussion.

It is indeed subjective. I'd like to just reiterate my impression: that 
blue3 distinguishes itself from the black text _too much_.

blue4, on the other hand, somewhat blends in, but in doing so it avoids 
the problem of being too in-your-face with prefix completion. And lets 
the users know anyway that if they really want to see the common part 
more prominently, there is a face they can customize.

Re: VOTE: Changing completions-common-part face's default

[Stefan Monnier]

>> > We can still fall back to subtle blue foreground for common-part. *shrug*
>> I'm more and more leaning this way, yes.
>> I don't think we'll find an agreement on what is the best default
>> setting, so I'll settle for a default setting that everyone can agree is
>> "not objectionably worse" than the status quo.
> Still pretty bad, tho.

Remember it's a step.  We can later consider something else.

The purpose is to get *some* highlighting without bothering those who
don't want any.  It's also to advertise the fact that there's a face
there that can be customized and it will also get more users to think
about "what this face should look like".

>> Does anyone object to this change?
> Count me as "abstain".

Good.


        Stefan


> Can't it be cyan or cyan 3 or something that? Dark blue against black
> has terrible contrast.

I find `cyan on white` to be barely legible.
I could live with `cyan3 on white` but I suspect more people will be
annoyed by this change than the change to `blue3`.

Re: VOTE: Changing completions-common-part face's default

[Stefan Monnier]

> Honestly, I'd rather [...]

But that's not the question: the question is whether you'd object to
the change because you think it's worse than what we currently have.


        Stefan

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 07.11.2019 1:14, Stefan Monnier wrote:
>> Honestly, I'd rather [...]
> 
> But that's not the question: the question is whether you'd object to
> the change because you think it's worse than what we currently have.

Yes, sure. But that's an easy bar to clean because I'm fine with what we 
have now (no highlighting for common-part). blue3 is *very* noticeable, 
which brings back the main objection to using it for prefix completion.

I recognize others feel differently, though.

So if blue4 also works for your stated goal, which is

   ...to get *some* highlighting without bothering those who
   don't want any.  It's also to advertise the fact that there's a face
   there that can be customized and it will also get more users to think
   about "what this face should look like"

then I would really suggest we use it instead. But if it's not legible 
for you, e.g. my monitor is unusually high-contrast or whatever, oh well.

But on the balance, I might consider blue4 an actual improvement over 
the status quo.

Re: VOTE: Changing completions-common-part face's default

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07.11.2019 1:14, Stefan Monnier wrote:
>>> Honestly, I'd rather [...]
> Yes, sure. But that's an easy bar to clean because I'm fine with what
> we have now (no highlighting for common-part). blue3 is *very*
> noticeable, which brings back the main objection to using it for
> prefix completion.
>
> I recognize others feel differently, though.

It seems to be more than "feeling".  Have you asked yourself why the
whole industry seems to does it differently (i.e. with bold or some
other contrasting color)?  I think it's because not all people have your
spectacular eyesight and not all of them have such great monitors (or
use them at a more conservative power setting).

> then I would really suggest we use it instead. But if it's not legible
> for you, e.g. my monitor is unusually high-contrast or whatever, oh
> well.

If the proposal is blue4 than my votes changes to "against".  The
reasoning is that it is counter-productive.  I imagine a new user
enabling, say, M-x fido-mode in Emacs 27, and asking himself "If they
were going to choose face for this, why choose this
near-indistinguishable, vision-straining thing?".  There's no space
there to answer "oh, you see, some people didn't like highlighting at
all, mostly in other matching styles you're not going to use, so we
chose this crappy middle-of-road compromise.  Also we don't care much
for your old-man's eyesight."

João

PS: If only there was a way to make flex use another highlighting...

Re: VOTE: Changing completions-common-part face's default

[Yuri Khan]

On Thu, 7 Nov 2019 at 17:28, João Távora <joaotavora@gmail.com> wrote:
> > blue3 is *very*
> > noticeable, which brings back the main objection to using it for
> > prefix completion.
> >
> > I recognize others feel differently, though.
>
> It seems to be more than "feeling".  Have you asked yourself why the
> whole industry seems to does it differently (i.e. with bold or some
> other contrasting color)?  I think it's because not all people have your
> spectacular eyesight and not all of them have such great monitors (or
> use them at a more conservative power setting).


I thought I’d chime in with some actual color science to back the
feelings in this thread.

The Web Content Accessibility Guidelines by W3C define a set of
criteria that text on the Web should satisfy in order to be considered
accessible. The complete document is at
<https://www.w3.org/TR/WCAG21>. Among other things, it defines color
luminance and contrast ratio, and specifies that text at normal size
should have a contrast ratio of at least 4.5:1 (and 3:1 for large
size, where large means 14pt bold or 18pt non-bold).

    https://www.w3.org/TR/WCAG21/#contrast-minimum

Practically, there are online color contrast calculators such as these:

    https://contrast-ratio.com/
    https://contrastchecker.com/
    https://contrastchecker.online/

(but I didn’t check which of them work well with LibreJS).

I also happen to have an implementation of the WCAG formulae in Elisp at:

    https://github.com/yurikhan/yk-color

Here are some values for pairs mentioned here:

black on white, default text in GUI Emacs: 21, passes
blue4 (#00008b) on white: 15.29, passes
blue3 (#0000cd) on white: 11.16, passes
cyan  (#00ffff) on white: 1.25, fails
cyan4 (#008b8b) on white: 4.14, fails by a small margin


There is no formula for calculating how distinguishable two different
text colors on the same background are. Because of color blindness,
there will always be users who cannot distinguish almost any two
colors that you pick. I think the relevant guideline is 1.4.1 Use of
Color:

    Color is not used as the only visual means of conveying information,
    indicating an action, prompting a response,
    or distinguishing a visual element.

Re: VOTE: Changing completions-common-part face's default

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1172 bytes --]

On Thu, Nov 7, 2019 at 1:27 PM Yuri Khan <yuri.v.khan@gmail.com> wrote:

> I thought I’d chime in with some actual color science to back the
> feelings in this thread.

Very nice.  Thanks.

> There is no formula for calculating how distinguishable two different
> text colors on the same background are.

I would say that the next best thing is to calculate the contrast
difference between two colors period.

blue4 vs black is 1.27
blue3 vs black is 1.87

So very good for reading the text, very bad for distinguishing
the colors in two adjacent letters.

This points to a rule for finding the perfect good compromise:
find the color equidistant to both black and white that passes
both tests.  Of all the proposed faces so far, cyan4 seems to
come closest to passing both these bars. But I found DodgerBlue3
passes the "vs white" (4.7) bar and almost passes the "vs black"
bar (4.1).

So I'd propose that color for the GUI default.

On text terminals we could use whatever minibuffer-prompt uses.

João

(of course we could just let styles choose how it highlights completions
from a given face pool, but that's sadly off the table)

[-- Attachment #2: Type: text/html, Size: 1650 bytes --]

Re: BIKESHED: completion faces

[Eli Zaretskii]

> Date: Wed, 6 Nov 2019 21:51:33 +0100
> From: Ergus <spacibba@aol.com>
> Cc: João Távora <joaotavora@gmail.com>,
> 	emacs-devel@gnu.org, monnier@iro.umontreal.ca, dgutov@yandex.ru
> 
> 1) It is the behaviour that the users expect, so familiar for them and
> that will reduce learning curve (we have enough curve for a newbie).

You mean, _some_ users.  With that caveat, I agree.

> 2) We should be modest and think that if most of the other editors
> follow a similar approach there should be a reason for that. (also
> private companies invest time and money in social market studies about
> what option is better... that's why MS windows is blue and have changes
> among the years, or the start button is in the left down corner)

I don't think it's right for us to blindly follow what other editors
do without (a) deciding positively that it's a good idea for Emacs,
and (b) making sure the new feature is adapted to the basic Emacs
operation.  One issue I have with "the other editors" in this regard
is that the way their completion UI works (specifically, how the
candidates are presented) is many times very different from what we
have in Emacs, so it will only work well to copy their faces if that
makes sense in the context of Emacs completion UI.

In any case, I already said I agree to have this as an optional
behavior.  We could have a special minor mode which will turn on
whatever features we think people who come from other editors will
expect.  Many current Emacs defaults started that way, and I think
it's a prudent way of introducing a feature that radically changes
some part of our UI.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> Date: Wed, 6 Nov 2019 13:40:41 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: joaotavora@gmail.com, emacs-devel@gnu.org, monnier@iro.umontreal.ca,
>         dgutov@yandex.ru
> 
> If you care about the set of candidates as a
> whole, and are not just trying to "find the
> one", or if you're interested in redefining
> that set (including possibly narrowing it),
> then "what exactly matched" your input can
> be very helpful info.

These tasks are rare, at least IME, so changing the presentation of
the candidates _by_default_ to cater to them is IMO too radical.
However, we could have a completion command to add/remove indications
of the matches, for when people want to perform these tasks.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Wed, 06 Nov 2019 23:36:23 +0200
> Cc: emacs-devel <emacs-devel@gnu.org>,
>  João Távora <joaotavora@gmail.com>,
>  Stefan Monnier <monnier@iro.umontreal.ca>
> 
> This screenshot demonstrates different completion styles:
> https://www.chromestory.com/2018/05/omnibox-redesign/
> where "time" in the link from bookmark/history
> is highlighted in bold for flex completion style,
> but "time" in search results for prefix completion style
> is not highlighted.  Instead it highlights the remaining part
> to help the user to continue typing the highlighted characters.

IMO and IME, this UI is optimized for clicking on the match you fancy,
not for continuing typing.

Re: BIKESHED: completion faces

[João Távora]

> In any case, I already said I agree to have this as an optional
> behavior.  We could have a special minor mode

FWIW I disagree with this.  It's needless complexity,
even more confusing. And we already have it: themes.

> that radically changes some part of our UI.

My proposal doesn't change the UI radically at all.
In fact it produces 0 changes to the UI that existed
before the introduction of flex.

João

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 07.11.2019 12:27, João Távora wrote:

>> I recognize others feel differently, though.
> 
> It seems to be more than "feeling".  Have you asked yourself why the
> whole industry seems to does it differently (i.e. with bold or some
> other contrasting color)?

I'm assuming that's rhetorical. But I have also pointed out how we 
differ from "the whole industry" in other respects, for good or bad.

But there are two questions here: do we highlight common-part by default 
at all, and which color to use.

> I think it's because not all people have your
> spectacular eyesight and not all of them have such great monitors (or
> use them at a more conservative power setting).

I'm always using mine at 50% brightness at most. But never mind.

>> then I would really suggest we use it instead. But if it's not legible
>> for you, e.g. my monitor is unusually high-contrast or whatever, oh
>> well.
> 
> If the proposal is blue4 than my votes changes to "against".  The
> reasoning is that it is counter-productive.  I imagine a new user
> enabling, say, M-x fido-mode in Emacs 27, and asking himself "If they
> were going to choose face for this, why choose this
> near-indistinguishable, vision-straining thing?".

The idea is that one would really need to find out why a given string is 
among the matches very rarely. Maybe once of twice, and they'll simply 
trust the completion engine after that. And for a one-off task, one 
might as well squint a little.

If the common scenario is different, and we expect that the user will 
look at common-part every time, to figure out what to type next, maybe 
we should discuss why the first-difference face is not enough for that.

> There's no space
> there to answer "oh, you see, some people didn't like highlighting at
> all, mostly in other matching styles you're not going to use, so we
> chose this crappy middle-of-road compromise.  Also we don't care much
> for your old-man's eyesight."

Vice versa, one might ask why we added this in-your-face highlighting 
why simply looks like a solid blue (or aquamarine) column in the 
Completions buffer. And we'll have to answer that, well, there exists a 
recently added non-default completion style where this highlighting is 
really useful.

> João
> 
> PS: If only there was a way to make flex use another highlighting...

BTW, speaking of backward compatibility, some of the changes in face 
placement you proposed would break company-capf's implementation of the 
'match' action.

Re: VOTE: Changing completions-common-part face's default

[Yuri Khan]

On Thu, 7 Nov 2019 at 20:53, João Távora <joaotavora@gmail.com> wrote:

> This points to a rule for finding the perfect good compromise:
> find the color equidistant to both black and white that passes
> both tests.

Because of the way contrast ratio is defined:

R = (L_lighter + 0.05) / (L_darker + 0.05)

if we require R(color, black) ≥ 4.5 and R(color, white) ≥ 4.5, we get:

(L(white) + 0.05) / (L(color) + 0.05) ≥ 4.5
1.05 / (L(color) + 0.05) ≥ 4.5
1.05 / 4.5 ≥ L(color) + 0.05
0.183333 ≥ L(color)

and:

(L(color) + 0.05) / (L(black) + 0.05) ≥ 4.5
(L(color) + 0.05) / 0.05 ≥ 4.5
L(color) + 0.05 ≥ 4.5 * 0.05 = 0.225
L(color) ≥ 0.175

So the set of colors that satisfy the 4.5:1 contrast criterion against
both white and black is a very thin surface of colors with relative
luminance between 0.175 and 0.183. Basically:

greens #008800 to #008a00
cyans #008282 to #008484
blues (somewhat tinted) #5e5eff to #6161ff
magentas #ce00ce to #d100d1
reds #eb0000 to #ee0000
yellows (rather, olives) #797900 to #7a7a00
grays #757575 to #767676

and various intermediates.

If you also want to account for the uncustomized 256-color xterm
palette, then I think it comes down to a single gray and maybe a
handful of cube colors. (Most non-black colors of that palette are
optimized for contrast against black background.)


However, the 4.5:1 threshold is tuned for readability of text on
background, not distinguishability of two text colors. For
distinguishability, maybe 3:1 or even lower would be enough.

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 07.11.2019 15:52, João Távora wrote:
> On Thu, Nov 7, 2019 at 1:27 PM Yuri Khan <yuri.v.khan@gmail.com 
> <mailto:yuri.v.khan@gmail.com>> wrote:
> 
>  > I thought I’d chime in with some actual color science to back the
>  > feelings in this thread.
> 
> Very nice.  Thanks.

I concur. I should find the same links very useful in some future 
theming discussions.

>  > There is no formula for calculating how distinguishable two different
>  > text colors on the same background are.
> 
> I would say that the next best thing is to calculate the contrast
> difference between two colors period.
> 
> blue4 vs black is 1.27
> blue3 vs black is 1.87
> 
> So very good for reading the text, very bad for distinguishing
> the colors in two adjacent letters.

I don't know if it works like that.

The necessary contrast between fg and bg is about being able to *read* 
the letter, and not just being able to notice that the color is 
different. So the standard for distinguishing colors should be much lower.

> This points to a rule for finding the perfect good compromise:
> find the color equidistant to both black and white that passes
> both tests.  Of all the proposed faces so far, cyan4 seems to
> come closest to passing both these bars. But I found DodgerBlue3
> passes the "vs white" (4.7) bar and almost passes the "vs black"
> bar (4.1).

It's a really pretty color, but if the goal is choosing a "not 
in-your-face" option, it fails that condition for the above reason.

> (of course we could just let styles choose how it highlights completions
> from a given face pool, but that's sadly off the table)

To reiterate something I said in another email: if flex stops using 
common-part where it does now, company-capf will become confused.

Re: VOTE: Changing completions-common-part face's default

[João Távora]

On Thu, Nov 7, 2019 at 2:34 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 07.11.2019 12:27, João Távora wrote:

> > It seems to be more than "feeling".  Have you asked yourself why the
> > whole industry seems to does it differently (i.e. with bold or some
> > other contrasting color)?
> I'm assuming that's rhetorical.

It wasn't. Assume you have asked yourself that, what answer did you
arrive at?

> The idea is that one would really need to find out why a given string is
> among the matches very rarely. Maybe once of twice, and they'll simply
> trust the completion engine after that. And for a one-off task, one
> might as well squint a little.

squinting is bad, mkay?

But it's interesting you point that out, because it allows me
to think why I want this:  one of the inherent characteristics of "flex"
is precisely that it can't be "trusted" 100%.  It makes educated guesses,
hopefully very good ones, but sometimes fails, so the user double-checks
via the highlighting.

> Vice versa, one might ask why we added this in-your-face highlighting
> why simply looks like a solid blue (or aquamarine) column in the
> Completions buffer. And we'll have to answer that, well, there exists a
> recently added non-default completion style where this highlighting is
> really useful.

No. It's totally _not_ harmful in the default style. It doesn't make the
first-difference character any harder to spot, nor the preceding text
harder to read

> BTW, speaking of backward compatibility, some of the changes in face
> placement you proposed would break company-capf's implementation of the
> 'match' action.

I can make a patch for that, if you explain what happens.  If it's the
decision where to put company's emphasis, just put it wherever
completions-emphasis is.

João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Nov 2019 14:28:55 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> > that radically changes some part of our UI.
> 
> My proposal doesn't change the UI radically at all.
> In fact it produces 0 changes to the UI that existed
> before the introduction of flex.

I wasn't talking about the changes which introduce zero changes in the
UI.

Re: VOTE: Changing completions-common-part face's default

[João Távora]

On Thu, Nov 7, 2019 at 2:43 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> >  > There is no formula for calculating how distinguishable two different
> >  > text colors on the same background are.
> >
> > I would say that the next best thing is to calculate the contrast
> > difference between two colors period.
> >
> > blue4 vs black is 1.27
> > blue3 vs black is 1.87
> >
> > So very good for reading the text, very bad for distinguishing
> > the colors in two adjacent letters.
>
> I don't know if it works like that.

This measures the contrast between two colors. We agree on
that. You can place the thresholds for each use case (reading on a
background, distinguishing nearby objects) where you like them,
and that's indeed subjective.

Anyhoo, you yourself admitted in the other mail that blue4 vs
black would lead to "squinting".  So I think it really does work
more or less like that.

> It's a really pretty color, but if the goal is choosing a "not
> in-your-face" option, it fails that condition for the above reason.

Thank you for the faith in my asthetic sensibilities :-), but
naturally,  in-your-faceness is not something we can objectively
discuss.

> To reiterate something I said in another email: if flex stops using
> common-part where it does now, company-capf will become confused.

Then teach company-capf to use `completions-emphasis` and
`completions-secondary-emphasis`.  I think I coded the part
where it sniffs for the common-part (didn't I?), I can make it sniff
for other stuff (and write tests and all).

João

Re: VOTE: Changing completions-common-part face's default

[João Távora]

> However, the 4.5:1 threshold is tuned for readability of text on
>  background, not distinguishability of two text colors. For
> distinguishability, maybe 3:1 or even lower would be enough.

Maybe.  Indeed I don't have the science to tell me otherwise.

So let's choose 3:1 as a threshold for that use.  Certainly
blue3 (1.87) and  blue4 (1.27) don't clear that bar, so let's
ditch those.

João

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 07.11.2019 16:44, João Távora wrote:

>>> It seems to be more than "feeling".  Have you asked yourself why the
>>> whole industry seems to does it differently (i.e. with bold or some
>>> other contrasting color)?
>> I'm assuming that's rhetorical.
> 
> It wasn't. Assume you have asked yourself that, what answer did you
> arrive at?

Some guesses regarding a different UI that I posted earlier. Or user 
expectations correlating to the current understanding of easy-to-use.

There are modern UI trends that are objectively bad, you know. Like the 
proliferation of dark-background themes (especially by default), or 
mixing light-on-dark with dark-on-light in the same theme without regard 
for contrast levels.

>> The idea is that one would really need to find out why a given string is
>> among the matches very rarely. Maybe once of twice, and they'll simply
>> trust the completion engine after that. And for a one-off task, one
>> might as well squint a little.
> 
> squinting is bad, mkay?

Maybe.

> But it's interesting you point that out, because it allows me
> to think why I want this:  one of the inherent characteristics of "flex"
> is precisely that it can't be "trusted" 100%.  It makes educated guesses,
> hopefully very good ones, but sometimes fails, so the user double-checks
> via the highlighting.

I don't understand this. flex is deterministic. Which part exactly 
cannot be trusted?

>> BTW, speaking of backward compatibility, some of the changes in face
>> placement you proposed would break company-capf's implementation of the
>> 'match' action.
> 
> I can make a patch for that, if you explain what happens.

You wrote the current code yourself, actually.

> If it's the
> decision where to put company's emphasis, just put it wherever
> completions-emphasis is.

I'm not sure I can agree with that. Company highlights the common part, 
not simply whatever part completion style chooses to highlight.

Re: BIKESHED: completion faces

[João Távora]

On Thu, Nov 7, 2019 at 2:47 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > My proposal doesn't change the UI radically at all.
> > In fact it produces 0 changes to the UI that existed
> > before the introduction of flex.
>
> I wasn't talking about the changes which introduce zero changes in the
> UI.

OK! So will you accept a version of my "renames" patch that affects
just the `flex` completion style? (or maybe the flex and the other
esoteric completion styles (pcm, substring)?

João Távora

Re: VOTE: Changing completions-common-part face's default

[Stefan Monnier]

> Yes, sure. But that's an easy bar to clean because I'm fine with what we
> have now (no highlighting for common-part).

Great, so you represent an important part of the audience.

> blue3 is *very* noticeable,

That's the goal (maybe not the "very" but the "noticeable").

> which brings back the main objection to using it for prefix completion.

To be honest, I'm not completely sure what is the main objection.
I chose `blue3` so far because while noticeable it seems to me that it
doesn't affect the overall legibility nor makes the first-diff
less noticeable.

So, yes, it's "useless" highlighting for basic/prefix-completion, but
that's pretty much a given.  "useless" is compatible with "harmless".

> I recognize others feel differently, though.
> So if blue4 also works for your stated goal, which is

I could live with "blue4" but it's significantly less visible, so
I think the only role it could play is to advertise the existence of the
face, but nothing else: I expect that very few people who want to see
the common-part (e.g. because they're using flex) will find it
sufficient, whereas I suspect that `blue3` would be "good enough" for
a non-negligible fraction of those users.

It is better than what we have, IMO, so I'm willing to "retreat" to
`blue4` for this "first step" if you really think `blue3` harms the
basic/prefix-completion case.


        Stefan

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Nov 2019 15:00:41 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> OK! So will you accept a version of my "renames" patch that affects
> just the `flex` completion style? (or maybe the flex and the other
> esoteric completion styles (pcm, substring)?

I admit that I don't understand the purpose of that renaming.  I
thought we were discussing how to highlight parts of completion
candidates differently from what we do now, not about renaming the
related faces.  Why are face names important enough to justify their
renaming?

Re: VOTE: Changing completions-common-part face's default

[João Távora]

> There are modern UI trends that are objectively bad, you know. Like the
> proliferation of dark-background themes (especially by default), or
> mixing light-on-dark with dark-on-light in the same theme without regard
> for contrast levels.

We mostly agree.  But the big IDE players like Visual Studio and Eclipse
don't follow shallow trends. They look quite ugly compared to Sublime or
Atom. As soon as flex/fuzzy/scatter became accessible and popular
(because of  programming speed) they decided to use ways to
highlight the matches,  very often boldface.  So it's not a "trend" at
all. We should ask Drew who probably has had such system in
place for a much longer time.

> I don't understand this. flex is deterministic. Which part exactly
> cannot be trusted?

Precisely, I thought you wouldn't.  Flex is designed to let you
smash some keys that vaguely fit into your intended completion,
or at least enter the spectrum where you kinda remember the
completion to be.  Some Flex implementations even allow you
to reverse keys so that when my lazy hand types "lyaz" it still
maches "the-lazy-fox".  When scoring, flex makes some mistakes
(depending on how good/sophisticated the scoring algorithm is).
So it's important to understand why the suggestion is at that
place.

You might think this double-checking "wastes your synapses".
It's a bargain indeed. it doesn't work for everyone. But it's
nonsensical to have flex with that crucial part crippled.

So that in the case of "lzy", I can choose between "the-lazy-fox"
and "le-zorg-yorg". Both can be reasonable intentions at different
points in time.

> I'm not sure I can agree with that. Company highlights the common part,
>  not simply whatever part completion style chooses to highlight.

If it's somehow so important for company to highlight always the
common part, we can put a 'this-is-part-of-the-common-part' prop
there, too, for company's benetif.  Sniffing for faces, the way I
implemented it, was really a low-hanging-hack anyway.  The patch
to sniff for a property is trivial.

So I wouldn't worry about that.

By the way, doesn't this requirement for company clash with your
view of the other Emacs frontends?  Why is it important for company
to always highlight the common part, but not for Emacs's icomplete,
or *Completions*?

João

Re: BIKESHED: completion faces

[João Távora]

On Thu, Nov 7, 2019 at 3:40 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Thu, 7 Nov 2019 15:00:41 +0000
> > Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>,
> >       Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> >
> > OK! So will you accept a version of my "renames" patch that affects
> > just the `flex` completion style? (or maybe the flex and the other
> > esoteric completion styles (pcm, substring)?
>
> I admit that I don't understand the purpose of that renaming.  I
> thought we were discussing how to highlight parts of completion
> candidates differently from what we do now, not about renaming the
> related faces.  Why are face names important enough to justify their
> renaming?

We _can_ do it without renaming, but then flex would be applying
"first-difference" to mean "I, flex, want these parts of the completions
I generate emphasized".  Which could lead to confusion when reading
the code.

So, if we rename, keeping backward-compatible face aliases

  "first-difference" to "completion-emphasis" and
  "common-part"  to "secondary-emphasis"

then it's not so confusing, when reading the code or when inspecting
a completion candidate.  It still fits nicely with the traditional 'basic's
view of completion (main emphasis on first difference, secondary
emphasis on common part) and is generically useful for new
completion styles.

Thanks for giving this a second look,
João

Re: VOTE: Changing completions-common-part face's default

[Stefan Monnier]

> So let's choose 3:1 as a threshold for that use.  Certainly
> blue3 (1.87) and  blue4 (1.27) don't clear that bar, so let's
> ditch those.

Having a lot of contrast between `default` and `completions-common-part`
is an explicit non-goal for this first step because we want
`completions-common-part` not to get in your face in the typical
basic/prefix-completion.


        Stefan

Re: VOTE: Changing completions-common-part face's default

[João Távora]

> > So let's choose 3:1 as a threshold for that use.  Certainly
> > blue3 (1.87) and  blue4 (1.27) don't clear that bar, so let's
> > ditch those.
> Having a lot of contrast between `default` and `completions-common-part`
> is an explicit non-goal for this first step because we want
> `completions-common-part` not to get in your face in the typical

As I wrote, in-your-face-ness is subjective.  But I think, like a stray
kitten with huge little eyesies, it would get in your heart.

João Meow

RE: VOTE: Changing completions-common-part face's default

[Drew Adams]

> As soon as flex/fuzzy/scatter became accessible and popular
> (because of  programming speed) they decided to use ways to
> highlight the matches,  very often boldface.  So it's not a
> "trend" at all. We should ask Drew who probably has had such
> system in place for a much longer time.

I can't speak to any trends.  Different kinds of
fuzzy matching have existed outside and inside Emacs
for a long time, in various 3rd-party libraries.

AFAIK, I (in Icicles) was the first to use such
kinds of matching for _completion_ - that's all.
That dates back to 2006 or before.

More generally, beyond various kinds of what is
sometimes called "fuzzy" matching, there are kinds
of matching other than basic, prefix-only matching -
substring matching, for example.  And some of those
were added to Emacs, including for completion, and
including fairly long ago, ultimately as
`completion-styles'.

"Fuzzy" matching can, if you like, include
so-called "flex" matching (Icicles calls this
"scatter matching").

The basic scatter match is a "poor man's fuzzy".
It just uses a simple regexp constructed from the
chars you type, by inserting `.*' between them -
typing `abc' uses regexp `a.*b.*c'.

What Icicles also provides "SPC scatter" matching,
which, IIUC, what is used by Ivy (but someone can
correct me).  It's like scatter matching, but it
matches the parts of your input that are separated
by SPC chars - it matches arbitrary text at the
separations between those parts.

On their own, such scatter matching does not
provide or use any scoring.  But some more-truly
"fuzzy" matches do sometimes provide scoring.

Icicles highlighting of matches is pretty simple.
It doesn't bother to try, for some kinds of fuzzy
matching.  For those that translate to using a
regexp, such as the scatter matches, highlighting
is trivial (`string-match-p').

I already spoke to the utility of highlighting the
matched part (plus, in the case of Icicles, the
part that is common to all matching completions).

I'll just add, since someone brought this up: it's
not about not trusting the matching algorithm.
It's not about checking by eyeball, to see if
there are matches shown that you don't think
really belong there.  Not at all.

It's, yes, partly that "fuzzy" etc. completion is
not always obvious; the results are not always
what one might expect.

But more importantly, it's just to provide you
_feedback on what you're doing_.  You're matching
input against a domain of possible candidates -
you're exploring that domain.  Visually connecting
what you do with what you get really does help.

And, since the contrary has been said several
times now in this thread, it's _not_ just about
figuring out what the next char you type should be.

The "next char you type" is only one way to
interact with the set of possible candidates and
the set of current matches.  Focusing on that's
based on thinking in terms of prefix matching,
choosing a single candidate, and immediately
exiting completion.

Besides the "next char you type", there's what
chars you might delete, and where, what chars
you might insert, and where - and many other
actions, which might have nothing to do with
editing your input.

When you can act in many different ways on
particular, or all, current matches, it really
helps to be able to distinguish them in terms
of which of their parts match the input pattern.

Dunno how to explain this more, briefly.  I'm
sure that those users who've used completion
setups that let you act on multiple candidates
or in multiple ways, or that let you match in
more complex (sometimes unclear) ways, get it.
For those who don't, I say play with it, and
you will.

FWIW, I'm _not_ a big proponent of fuzzy
matching.  I'm a big proponent of substring
and regexp matching.

What I use, myself, the vast majority of the
time, is regexp matching, which includes
substring matching.

But YMMV.  And Icicles does provide several
kinds of fuzzy matching.  This page describes
them:

https://www.emacswiki.org/emacs/Icicles_-_Completion_Methods_and_Styles

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Nov 2019 16:00:26 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> We _can_ do it without renaming, but then flex would be applying
> "first-difference" to mean "I, flex, want these parts of the completions
> I generate emphasized".  Which could lead to confusion when reading
> the code.

Then how about simply using face aliases, without obsoleting the
current ones?

>   "first-difference" to "completion-emphasis" and
>   "common-part"  to "secondary-emphasis"

I think secondary-emphasis is not a good name, it says nothing about
its use.  We should find a better name.

But anyway, this is a very minor issue, the main issue is not how to
call the faces.

Re: BIKESHED: completion faces

[João Távora]

On Thu, Nov 7, 2019 at 5:39 PM Eli Zaretskii <eliz@gnu.org> wrote:
> Then how about simply using face aliases, without obsoleting the
> current ones?

I didn't know that was possible.  It's perfect.

> >   "first-difference" to "completion-emphasis" and
> >   "common-part"  to "secondary-emphasis"
>
> I think secondary-emphasis is not a good name, it says nothing about
> its use.  We should find a better name.

"alternate-emphasis"?  But if faces can be aliased without obsoleting,
then 'basic' can continue the old names with maximum semantics.

And since I don't think flex has much need for "alternate-emphasis"
, it can continue to use the old names.

> But anyway, this is a very minor issue, the main issue is not how to
> call the faces.

The main issue, for me, is letting 'flex' emphasize the part of the
string that matched the pattern, so that users of flex can get
visual feedback (like Drew, albeit a little longwindedly, explained
here: https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00261.html).

This is the core of my proposal: to improve usability of the new
flex style by default.  With your face aliases idea, the patch
becomes even simpler, and can be contained to just 'flex' (or
expanded to substring and p-c-m if their authors/users find
acceptable).

João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Nov 2019 17:53:02 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> > >   "first-difference" to "completion-emphasis" and
> > >   "common-part"  to "secondary-emphasis"
> >
> > I think secondary-emphasis is not a good name, it says nothing about
> > its use.  We should find a better name.
> 
> "alternate-emphasis"?

Same problem.  How about "match-emphasis"?

Re: BIKESHED: completion faces

[João Távora]

On Thu, Nov 7, 2019 at 6:16 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Thu, 7 Nov 2019 17:53:02 +0000
> > Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>,
> >       Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> >
> > > >   "first-difference" to "completion-emphasis" and
> > > >   "common-part"  to "secondary-emphasis"
> > >
> > > I think secondary-emphasis is not a good name, it says nothing about
> > > its use.  We should find a better name.
> >
> > "alternate-emphasis"?
>
> Same problem.  How about "match-emphasis"?

First, I hope we don't confuse ourselves, there is a prefix implicit
here right? It's "completions" (or actually it should be "completion"
singular).  Apologies if I've been sloppy in using and not using
the full name.

So I'm proposing "completion-emphasis", which we seem to agree
on, for _primary_ emphasis, the thing that the matching style wants
to highlight most prominently.

Regarding the second (but not necessarily secondary) face, for other
things the completion style you are proposing "completion-match-emphasis",
right? This is the one aliased to existing "completions-common-part".

It works but it is slightly akward of flex _not_ to use it.  Because
if it uses it, the only reasonable place to put it is in exactly the same
places that it is already applying "completion-emphasis".  But
I don't think there can be two different faces for the same
character in a string, can there?

Nothing a comment in the code can't fix, but users might be
surprised that they customize something with that name and
nothing happens in flex.

I personally don't see it problematic to choose a name that says
less about the use, because we can't predict exactly what the use
is: just that it will be used to highlight parts of completions where
prominence is less important than the one offered by
"completion-emphasis", which defaults to "foreground: bold".

So "alternate", "supporting", "secondary" or anything that expresses
that would be my choices.

João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Nov 2019 18:37:53 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> > > > >   "first-difference" to "completion-emphasis" and
> > > > >   "common-part"  to "secondary-emphasis"
> > > >
> > > > I think secondary-emphasis is not a good name, it says nothing about
> > > > its use.  We should find a better name.
> > >
> > > "alternate-emphasis"?
> >
> > Same problem.  How about "match-emphasis"?
> 
> First, I hope we don't confuse ourselves, there is a prefix implicit
> here right? It's "completions" (or actually it should be "completion"
> singular).  Apologies if I've been sloppy in using and not using
> the full name.
> 
> So I'm proposing "completion-emphasis", which we seem to agree
> on

Not really, no.  It suffers from the same problem: too vague to say
what it's used for.

> for _primary_ emphasis, the thing that the matching style wants
> to highlight most prominently.
> 
> Regarding the second (but not necessarily secondary) face, for other
> things the completion style you are proposing "completion-match-emphasis",
> right? This is the one aliased to existing "completions-common-part".

The names "primary" and "secondary" are artificial and even
subjective.  We should try to find more descriptive terms that tell
more about the uses of the face.

> So "alternate", "supporting", "secondary" or anything that expresses
> that would be my choices.

Not mine, sorry.

Re: BIKESHED: completion faces

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Date: Thu, 7 Nov 2019 18:37:53 +0000
>> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
>> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
>> 
>> > > > >   "first-difference" to "completion-emphasis" and
>> > > > >   "common-part"  to "secondary-emphasis"
>> > > >
>> > > > I think secondary-emphasis is not a good name, it says nothing about
>> > > > its use.  We should find a better name.
>> > >
>> > > "alternate-emphasis"?
>> >
>> > Same problem.  How about "match-emphasis"?
>> 
>> First, I hope we don't confuse ourselves, there is a prefix implicit
>> here right? It's "completions" (or actually it should be "completion"
>> singular).  Apologies if I've been sloppy in using and not using
>> the full name.
>> 
>> So I'm proposing "completion-emphasis", which we seem to agree
>> on
>
> Not really, no.

Why didn't you object to it 1..2..six mails ago?  You just said
"secondary-emphasis" is not a good name...  This makes us go in circles,
Eli.

One more time: my only goal in this entire giant conversation, is for
the 'flex' style to be able to transform this match on the pattern "foo":

  fabrobazor
           ^
           `- this letter is bold by default currently

into this:

  fabrobazor
  ^   ^   ^
  `---`---`----- I want these bold by default in the future

no change to the 'basic/prefix' completion.  If you don't agree to this
we can stop the discussion right here, no hard feelings, really no
problem.

Otherwise, if you do, then the only completion-related face that
currently has "bold" is completions-first-difference, which is a silly
name for this.  Should I alias that to "completions-match-emphasis",
your 3-mails-ago suggestion?  To something else?

Thanks in advance,
João

Re: BIKESHED: completion faces

[Juri Linkov]

>> This screenshot demonstrates different completion styles:
>> https://www.chromestory.com/2018/05/omnibox-redesign/
>> where "time" in the link from bookmark/history
>> is highlighted in bold for flex completion style,
>> but "time" in search results for prefix completion style
>> is not highlighted.  Instead it highlights the remaining part
>> to help the user to continue typing the highlighted characters.
>
> IMO and IME, this UI is optimized for clicking on the match you fancy,
> not for continuing typing.

The matches highlighted in the links from the history/bookmarks
using flex completion style are intended for clicking, indeed.
But the search results with the highlighted first difference
and remaining string are intended for help in continuing typing
to narrow the search results.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: spacibba@aol.com,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,
>   dgutov@yandex.ru
> Date: Thu, 07 Nov 2019 21:43:00 +0000
> 
> >> So I'm proposing "completion-emphasis", which we seem to agree
> >> on
> >
> > Not really, no.
> 
> Why didn't you object to it 1..2..six mails ago?  You just said
> "secondary-emphasis" is not a good name...  This makes us go in circles,
> Eli.

Sorry, I don't always have time to catch up with huge threads, and in
this particular case had difficulties to understand what were the main
issues.

> One more time: my only goal in this entire giant conversation, is for
> the 'flex' style to be able to transform this match on the pattern "foo":
> 
>   fabrobazor
>            ^
>            `- this letter is bold by default currently
> 
> into this:
> 
>   fabrobazor
>   ^   ^   ^
>   `---`---`----- I want these bold by default in the future
> 
> no change to the 'basic/prefix' completion.

I understand that you want to highlight both f, o, o, and r, but the
latter with a different face.  If my understanding is incorrect, then
what will the other face be used for?

> Otherwise, if you do, then the only completion-related face that
> currently has "bold" is completions-first-difference, which is a silly
> name for this.  Should I alias that to "completions-match-emphasis",
> your 3-mails-ago suggestion?  To something else?

I understand that what I called completions-match-emphasis will
highlight f, o, and o -- the characters that were in the string typed
by the user.  These are currently not highlighted at all.  I also
understand that there will be another face, the one we currently call
completions-first-difference (and you want another name for it), which
will highlight r in this example.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org,  joaotavora@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Fri, 08 Nov 2019 00:40:05 +0200
> 
> >> This screenshot demonstrates different completion styles:
> >> https://www.chromestory.com/2018/05/omnibox-redesign/
> >> where "time" in the link from bookmark/history
> >> is highlighted in bold for flex completion style,
> >> but "time" in search results for prefix completion style
> >> is not highlighted.  Instead it highlights the remaining part
> >> to help the user to continue typing the highlighted characters.
> >
> > IMO and IME, this UI is optimized for clicking on the match you fancy,
> > not for continuing typing.
> 
> The matches highlighted in the links from the history/bookmarks
> using flex completion style are intended for clicking, indeed.
> But the search results with the highlighted first difference
> and remaining string are intended for help in continuing typing
> to narrow the search results.

I don't think so.  The second image indeed shows in bold the string
following what the user typed (and we do the same, but only with the
first character), but the presentation clearly invites you to click on
a candidate, it has nothing specific for typing, unlike Emacs, which
specifically highlights the next character to type for narrowing the
list of candidates.

I'm not surprised by these differences, because as I said elsewhere in
this thread, the other applications present a completely different UI
for these completions.  We have discussed this with RMS long ago and
came to this conclusion, and IME it still stands, at least in
browsers.

Re: BIKESHED: completion faces

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> Why didn't you object to it 1..2..six mails ago?  You just said
>> "secondary-emphasis" is not a good name...  This makes us go in circles,
>> Eli.
> Sorry, I don't always have time to catch up with huge threads, and in
> this particular case had difficulties to understand what were the main
> issues.

It's pretty simple.  In that mail I made three statements, you objected
to the second, which lead me to think that you agree with the other
two.

Here are the statements again:

1. Alias 'completions-first-difference' to 'completion-emphasis'
2. Alias 'completions-common-part' to 'completion-secondary-emphasis'
3. Make 'flex' use only completion-emphasis to highlight the
   pattern in the candidates.  

That's it.

>> One more time: my only goal in this entire giant conversation, is for
>> the 'flex' style to be able to transform this match on the pattern "foo":
>> 
>>   fabrobazor
>>            ^
>>            `- this letter is bold by default currently
>> 
>> into this:
>> 
>>   fabrobazor
>>   ^   ^   ^
>>   `---`---`----- I want these bold by default in the future
>> 
>> no change to the 'basic/prefix' completion.
>
> I understand that you want to highlight both f, o, o, and r, but the
> latter with a different face.  If my understanding is incorrect,

It is incorrect indeed: 'r' is should have the same face as 'a' or 'b'.

> then what will the other face be used for?

Nothing.  I don't (yet) see 'flex' as needing secondary kinds of
emphasis.

>> Otherwise, if you do, then the only completion-related face that
>> currently has "bold" is completions-first-difference, which is a silly
>> name for this.  Should I alias that to "completions-match-emphasis1",
>> your 3-mails-ago suggestion?  To something else?
>
> I understand that what I called completions-match-emphasis will
> highlight f, o, and o -- the characters that were in the string typed
> by the user.  These are currently not highlighted at all.  I also
> understand that there will be another face, the one we currently call
> completions-first-difference (and you want another name for it), which
> will highlight r in this example.

No, that is again incorrect.  That "other face" is already the most
important one of the two.  Again, I repeat, the one we currently call
'completions-first-difference', is currently the bold one, by default.
I want to change its name (to 'completion-emphasis' in my proposal) so
that it makes sense for flex to apply it to flex's most important parts,
which, in the above example, are the three characters 'f', 'o' and 'o'
and no other characters.

Thanks,
João

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 07.11.2019 17:08, Stefan Monnier wrote:

>> which brings back the main objection to using it for prefix completion.
> 
> To be honest, I'm not completely sure what is the main objection.
> I chose `blue3` so far because while noticeable it seems to me that it
> doesn't affect the overall legibility nor makes the first-diff
> less noticeable.

Excessive highlighting is something we've been normally avoiding in 
Emacs, for its distracting properties.

> So, yes, it's "useless" highlighting for basic/prefix-completion, but
> that's pretty much a given.  "useless" is compatible with "harmless".

That's also true.

>> I recognize others feel differently, though.
>> So if blue4 also works for your stated goal, which is
> 
> I could live with "blue4" but it's significantly less visible, so
> I think the only role it could play is to advertise the existence of the
> face, but nothing else: I expect that very few people who want to see
> the common-part (e.g. because they're using flex) will find it
> sufficient, whereas I suspect that `blue3` would be "good enough" for
> a non-negligible fraction of those users.

Like Joao said, blue4 will seem like a half-measure, and I generally 
don't support those. So while I'd prefer my first suggestion (wait until 
flex is used universally), I'd like to retract my personal objection here.

blue3 is a good enough color, and it fits the default theme. So unless 
we can agree on a better idea, and unless other people object, let's go 
with blue3 and call it a day.

> It is better than what we have, IMO, so I'm willing to "retreat" to
> `blue4` for this "first step" if you really think `blue3` harms the
> basic/prefix-completion case.

If we have more time to think on this, I'd try to consider how to better 
help the pcmpl/flex cases by using completions-first-difference on more 
characters instead. And/or adding an extra face for the other 
"non-matching" characters.

For "fabrobazor" example and input "foo",

"abr", "b" and "az" would be highlighted with that new face. And "r" 
still with first-difference.

Information-wise, the result should be the same as highlighting the 
common part.

Not quite sure what colors to use, though.

Re: BIKESHED: completion faces

[Stefan Monnier]

>>   fabrobazor
>>   ^   ^   ^
>>   `---`---`----- I want these bold by default in the future
[...]
> I understand that what I called completions-match-emphasis will
> highlight f, o, and o -- the characters that were in the string typed
> by the user.  These are currently not highlighted at all.

They're currently "highlighted" with the `completions-common-part` face,
which by default so far has no visible attribute, so it "looks"
not-highlighted.

I'd like to change that so the face has a "blue3" (or if that's
considered too loud, "blue4") foreground.


        Stefan

Re: BIKESHED: completion faces

[João Távora]

On Fri, Nov 8, 2019 at 12:58 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> >>   fabrobazor
> >>   ^   ^   ^
> >>   `---`---`----- I want these bold by default in the future
> [...]
> > I understand that what I called completions-match-emphasis will
> > highlight f, o, and o -- the characters that were in the string typed
> > by the user.  These are currently not highlighted at all.
>
> They're currently "highlighted" with the `completions-common-part` face,
> which by default so far has no visible attribute, so it "looks"
> not-highlighted.
>
> I'd like to change that so the face has a "blue3" (or if that's
> considered too loud, "blue4") foreground.

If we do that, the situation will improve, but only very very
slightly and in the wrong direction.  Those letters 'f', 'o' and 'o'
must have a more prominent face, indeed just as prominent
or "loud" as the classic 'prefix' style puts in the "first different
letter". So it has to be "loud" to have any meaning.

If Eli allows my proposal, it's better for everybody, I think.  I've
described it fully.  If he doesn't we can go with yours.

João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: spacibba@aol.com,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,
>   dgutov@yandex.ru
> Date: Fri, 08 Nov 2019 10:15:02 +0000
> 
> It's pretty simple.  In that mail I made three statements, you objected
> to the second, which lead me to think that you agree with the other
> two.

I'm sorry for causing you to misunderstand what I was saying.

> 1. Alias 'completions-first-difference' to 'completion-emphasis'
> 2. Alias 'completions-common-part' to 'completion-secondary-emphasis'
> 3. Make 'flex' use only completion-emphasis to highlight the
>    pattern in the candidates.  

We are miscommunicating.  These faces aren't flex-specific, and I was
talking about their names in the context of any completion style,
including, but not limited to flex.

In any case, I consider both completion-emphasis and
completion-secondary-emphasis to be sub-optimal names, because they
say nothing at all regarding what is being emphasized.

> >>   fabrobazor
> >>   ^   ^   ^
> >>   `---`---`----- I want these bold by default in the future
> >> 
> >> no change to the 'basic/prefix' completion.
> >
> > I understand that you want to highlight both f, o, o, and r, but the
> > latter with a different face.  If my understanding is incorrect,
> 
> It is incorrect indeed: 'r' is should have the same face as 'a' or 'b'.

Did I say that _only_ r will be highlighted with that face?

I'm okay with highlighting a and b as well in this example, provided
that typing "faoo" will include "fabrobazor" in the results.

However, please note that r is somewhat different from a and b,
because to type r I don't need to move point, I need just to type the
letter, whereas the other two require me to use cursor motion commands
first.  Not sure if this difference is significant enough to justify
yet another face (and another round of bikeshedding).

> > then what will the other face be used for?
> 
> Nothing.  I don't (yet) see 'flex' as needing secondary kinds of
> emphasis.

What about styles other than flex?

> > I understand that what I called completions-match-emphasis will
> > highlight f, o, and o -- the characters that were in the string typed
> > by the user.  These are currently not highlighted at all.  I also
> > understand that there will be another face, the one we currently call
> > completions-first-difference (and you want another name for it), which
> > will highlight r in this example.
> 
> No, that is again incorrect.  That "other face" is already the most
> important one of the two.  Again, I repeat, the one we currently call
> 'completions-first-difference', is currently the bold one, by default.
> I want to change its name (to 'completion-emphasis' in my proposal) so
> that it makes sense for flex to apply it to flex's most important parts,
> which, in the above example, are the three characters 'f', 'o' and 'o'
> and no other characters.

That would mean this face is used inconsistently: in basic and other
similar styles it highlights the character(s) that narrow(s) the
selection of candidates, while in flex they will highlight the
characters already typed by the user.  Is that correct?  If so, I
object to this inconsistency.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: João Távora <joaotavora@gmail.com>,
>   spacibba@aol.com,
>   emacs-devel@gnu.org,  dgutov@yandex.ru
> Date: Fri, 08 Nov 2019 07:58:43 -0500
> 
> >>   fabrobazor
> >>   ^   ^   ^
> >>   `---`---`----- I want these bold by default in the future
> [...]
> > I understand that what I called completions-match-emphasis will
> > highlight f, o, and o -- the characters that were in the string typed
> > by the user.  These are currently not highlighted at all.
> 
> They're currently "highlighted" with the `completions-common-part` face,
> which by default so far has no visible attribute, so it "looks"
> not-highlighted.

So we agree, yes?  They are currently NOT highlighted, although they
do have a distinct face.

Anyway, what you write seems to contradict what João was saying
regarding the use of these faces.  Or maybe I'm still confused.

> I'd like to change that so the face has a "blue3"

How will the user know the difference between these two highlightings?
Are you counting on them to remember what they typed?  What about
completion in contexts where the user didn't type anything, like when
they type "?"

Re: BIKESHED: completion faces

[João Távora]

On Fri, Nov 8, 2019 at 2:02 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Cc: spacibba@aol.com,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,
> >   dgutov@yandex.ru
> > Date: Fri, 08 Nov 2019 10:15:02 +0000
> >
> > It's pretty simple.  In that mail I made three statements, you objected
> > to the second, which lead me to think that you agree with the other
> > two.
>
> I'm sorry for causing you to misunderstand what I was saying.
>
> > 1. Alias 'completions-first-difference' to 'completion-emphasis'
> > 2. Alias 'completions-common-part' to 'completion-secondary-emphasis'
> > 3. Make 'flex' use only completion-emphasis to highlight the
> >    pattern in the candidates.
>
> We are miscommunicating.  These faces aren't flex-specific, and I was
> talking about their names in the context of any completion style,
> including, but not limited to flex.

I know that. Therefore I chose the names to work in any context, so
they aren't specific to 'flex', nor specific to 'basic' or 'p-c-m' or
any other style.

> In any case, I consider both completion-emphasis and
> completion-secondary-emphasis to be sub-optimal names, because they
> say nothing at all regarding what is being emphasized.

Because, in my opinion, if they are to be useful across different styles,
their names shouldn't make the commitment to say exactly the thing
they highlight. Their names should rather state "I am the face for
whatever the completion style believes is visually relevant"  and "I am the
face for whatever ... is visually relevant at a secondary level".

> > >>   fabrobazor
> > >>   ^   ^   ^
> > >>   `---`---`----- I want these bold by default in the future
> > >>
> > >> no change to the 'basic/prefix' completion.
> > >
> > > I understand that you want to highlight both f, o, o, and r, but the
> > > latter with a different face.  If my understanding is incorrect,
> >
> > It is incorrect indeed: 'r' is should have the same face as 'a' or 'b'.
>
> Did I say that _only_ r will be highlighted with that face?

I don' think so. I think you can read what you said above.

I confirmed your sentence "if my understanding is incorrect". Because
I _don't_ intend flex to  "highlight" r at all, or 'a' or any other character
except 'f', 'o' and ' o'.  So these characters ('a','b','r' and 'z' in this
example) should, for now, have the same face, 'default'. We can think
about the value of highlighting other neighboring characters later.

> I'm okay with highlighting a and b as well in this example, provided
> that typing "faoo" will include "fabrobazor" in the results.

It will indeed include that in the results.  But in that your
example only 4 characters should be highlighted: f,
a (the first one),o, and o .  So no 'b'.

> However, please note that r is somewhat different from a and b,
> because to type r I don't need to move point, I need just to type the
> letter, whereas the other two require me to use cursor motion commands
> first.  Not sure if this difference is significant enough to justify
> yet another face (and another round of bikeshedding).

Exactly, I don't think so. Not at this moment. But your concern
could be addressed.

> > > then what will the other face be used for?
> >
> > Nothing.  I don't (yet) see 'flex' as needing secondary kinds of
> > emphasis.
>
> What about styles other than flex?

They keep using it for whatever they think is secondary.  'basic', for
one, will keep using it for the prefix substring in each candidate.

> That would mean this face is used inconsistently: in basic and other
> similar styles it highlights the character(s) that narrow(s) the
> selection of candidates, while in flex they will highlight the
> characters already typed by the user.  Is that correct?

Yes it is, but "this face" would now be called "completion-emphasis"
, not "completion-narrower" or "completion-already-typed" or
"completion-first-difference".   That's why the renames are
important.

> If so, I object to this inconsistency.

OK. It's good to finally clear that up. But when I talked about this topic
ealier in the thread you replied this.

https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00176.html

JT> However some people have made a point that there should be
JT> some kind of consistency at this level between completion styles, that
JT> the thing emphasized in 'prefix/basic' should have some semantic
JT>relation to the thing emphasized thing for 'flex' and 'substring' too.

EZ > Sorry, but I see no reason for any kind of "consistency" here.  We
EZ> need to highlight to help the user specify the right completion
EZ> candidate, all the rest is secondary IMO.

Maybe you weren't understanding my proposal completely back then,
but in any case your "then-reply" is exactly, verbatim, word for word, what
I would say to you in this situation.

Take care,
João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Nov 2019 15:09:31 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> > > >>   fabrobazor
> > > >>   ^   ^   ^
> > > >>   `---`---`----- I want these bold by default in the future
> > > >>
> > > >> no change to the 'basic/prefix' completion.
> > > >
> > > > I understand that you want to highlight both f, o, o, and r, but the
> > > > latter with a different face.  If my understanding is incorrect,
> > >
> > > It is incorrect indeed: 'r' is should have the same face as 'a' or 'b'.
> >
> > Did I say that _only_ r will be highlighted with that face?
> 
> I don' think so. I think you can read what you said above.
> 
> I confirmed your sentence "if my understanding is incorrect". Because
> I _don't_ intend flex to  "highlight" r at all, or 'a' or any other character
> except 'f', 'o' and ' o'.  So these characters ('a','b','r' and 'z' in this
> example) should, for now, have the same face, 'default'. We can think
> about the value of highlighting other neighboring characters later.
> 
> > I'm okay with highlighting a and b as well in this example, provided
> > that typing "faoo" will include "fabrobazor" in the results.
> 
> It will indeed include that in the results.  But in that your
> example only 4 characters should be highlighted: f,
> a (the first one),o, and o .  So no 'b'.

'b' comes from what you originally wrote.

I admit I don't understand why 'b' won't be highlighted: wouldn't
typing "fobo" narrow the list of candidates shown by "foo"?  If so,
'b' should be highlighted like 'a' and 'r', no?

> > That would mean this face is used inconsistently: in basic and other
> > similar styles it highlights the character(s) that narrow(s) the
> > selection of candidates, while in flex they will highlight the
> > characters already typed by the user.  Is that correct?
> 
> Yes it is, but "this face" would now be called "completion-emphasis"
> , not "completion-narrower" or "completion-already-typed" or
> "completion-first-difference".   That's why the renames are
> important.

I don't think that names that obfuscate the use of the face is a good
idea.  I understood the reason for renaming to be that "first
difference" and "common part" don't necessarily make sense for flex
and other similar styles, but the solution should be to make the names
more general, not obfuscated.  "First" and "second" or "primary" and
"secondary" are too opaque to be useful, IMO.

> > If so, I object to this inconsistency.
> 
> OK. It's good to finally clear that up. But when I talked about this topic
> ealier in the thread you replied this.
> 
> https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00176.html

Yes, I've changed my mind after understanding better what you propose.

Re: BIKESHED: completion faces

[João Távora]

On Fri, Nov 8, 2019 at 3:34 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Fri, 8 Nov 2019 15:09:31 +0000
> > Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>,
> >       Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> >
> > > > >>   fabrobazor
> > > > >>   ^   ^   ^
> > > > >>   `---`---`----- I want these bold by default in the future
> > > > >>
> > > > >> no change to the 'basic/prefix' completion.
> > > > >
> > > > > I understand that you want to highlight both f, o, o, and r, but the
> > > > > latter with a different face.  If my understanding is incorrect,
> > > >
> > > > It is incorrect indeed: 'r' is should have the same face as 'a' or 'b'.
> > >
> > > Did I say that _only_ r will be highlighted with that face?
> >
> > I don' think so. I think you can read what you said above.
> >
> > I confirmed your sentence "if my understanding is incorrect". Because
> > I _don't_ intend flex to  "highlight" r at all, or 'a' or any other character
> > except 'f', 'o' and ' o'.  So these characters ('a','b','r' and 'z' in this
> > example) should, for now, have the same face, 'default'. We can think
> > about the value of highlighting other neighboring characters later.
> >
> > > I'm okay with highlighting a and b as well in this example, provided
> > > that typing "faoo" will include "fabrobazor" in the results.
> >
> > It will indeed include that in the results.  But in that your
> > example only 4 characters should be highlighted: f,
> > a (the first one),o, and o .  So no 'b'.
>
> 'b' comes from what you originally wrote.

My example was different from yours.  I just wrote 'b' there to
exemplify a thing that would not be highlighted when the pattern
is "foo" and the candidate is "fabrobazor"

> I admit I don't understand why 'b' won't be highlighted: wouldn't
> typing "fobo" narrow the list of candidates shown by "foo"?  If so,
> 'b' should be highlighted like 'a' and 'r', no?

When you type "fobo" the 'b' is highlighted .What you seem to
propose later doesn't make much sense in flex, because usually
there can be many such things that you "could" type to narrow
the selection.If 'basic' there is at most one. And in other more
complex styles like Drew's regexp styles or Helm's advances stylee,
there can probably be many more such things, even characters that
are not literally in the candidates.

> > > That would mean this face is used inconsistently: in basic and other
> > > similar styles it highlights the character(s) that narrow(s) the
> > > selection of candidates, while in flex they will highlight the
> > > characters already typed by the user.  Is that correct?
> >
> > Yes it is, but "this face" would now be called "completion-emphasis"
> > , not "completion-narrower" or "completion-already-typed" or
> > "completion-first-difference".   That's why the renames are
> > important.
>
> I don't think that names that obfuscate the use of the face is a good

I don't think it obfuscates, but it displaces the meaning, yes.
Since there can be many styles, which rely on visually
highlighting things according to a particular, well, "style",
I thought it a good strategy to have face names that don't
restrict them from doing so.

So, according to your decision, the way to achieve that
freedom seems to be, to me, to have more face names
like "flex-completion-pattern",  "flex-completion-likely-next-char".
"forgiving-completion-typo",  etc.  (the latter is a hypothetical
example of a face used in the implementation of a "forgiving"
style that allows some typos).  I am correct?

This affords the "consistency" you want, at the expense of
requiring the user to customize a whole new set of faces
if he ever switches styles.  The upside is that the defaults
problem goes away and perhaps face inheritance can hide
part of the complexity from the user.

So, using the opportunity that the subject is fresh now, would
you object that 'flex' starts using a new face, say
'completion-flex-pattern-emphasis' or 'flex-pattern-emphasis' ,
that inherits from the current 'completions-first-difference' face?

Thanks,
João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Nov 2019 16:01:54 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> So, using the opportunity that the subject is fresh now, would
> you object that 'flex' starts using a new face, say
> 'completion-flex-pattern-emphasis' or 'flex-pattern-emphasis' ,
> that inherits from the current 'completions-first-difference' face?

It's possible, though perhaps not optimal.  I'd prefer to try to use
the same faces, and have their names explain what they do in a
sufficiently generalized way which would fit all styles.

Re: BIKESHED: completion faces

[João Távora]

On Fri, Nov 8, 2019 at 4:17 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Fri, 8 Nov 2019 16:01:54 +0000
> > Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>,
> >       Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> >
> > So, using the opportunity that the subject is fresh now, would
> > you object that 'flex' starts using a new face, say
> > 'completion-flex-pattern-emphasis' or 'flex-pattern-emphasis' ,
> > that inherits from the current 'completions-first-difference' face?
>
> It's possible, though perhaps not optimal.  I'd prefer to try to use
> the same faces, and have their names explain what they do in a
> sufficiently generalized way which would fit all styles.

I'm afraid this particular goal is irreconcilable with a freshly
installed Emacs having, __by default__:

1. 'basic' style give prominence to the "first different character"
2. 'flex' style give prominence to the pattern's characters.

UNLESS, of course that elusive face name is something
deliberately vague and generic like 'completion-emphasis' or
'completion-prominence', what you call obfuscation of the specific
meaning.  So "fit all styles" conflicts with "reveal the specific
meaning".

Therefore, I propose we take the the "not-optimal" and
"possible" route.

-- 
João Távora

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Nov 2019 16:27:36 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> > It's possible, though perhaps not optimal.  I'd prefer to try to use
> > the same faces, and have their names explain what they do in a
> > sufficiently generalized way which would fit all styles.
> 
> I'm afraid this particular goal is irreconcilable with a freshly
> installed Emacs having, __by default__:
> 
> 1. 'basic' style give prominence to the "first different character"
> 2. 'flex' style give prominence to the pattern's characters.

AFAIU, Stefan wants to highlight the pattern's characters in basic
style as well.  So the gap is not so large after all.  AFAIU, you are
just saying that you don't want to highlight any characters with the
"first different" face, in which case just don't put that face on any
character.  Though I don't quite understand why you are so against
highlighting possible next character(s).

> UNLESS, of course that elusive face name is something
> deliberately vague and generic like 'completion-emphasis' or
> 'completion-prominence', what you call obfuscation of the specific
> meaning.  So "fit all styles" conflicts with "reveal the specific
> meaning".

I think you assign too much significance to the face names.  We should
instead try to make the highlighting of different styles more
consistent.  Highlighting only the matched part by one style and only
the next suggested characters to type is a starking difference, and I
think it will confuse users who have several styles in the alist.

> Therefore, I propose we take the the "not-optimal" and "possible"
> route.

I think you give up too early.

Re: BIKESHED: completion faces

[João Távora]

On Fri, Nov 8, 2019 at 5:05 PM Eli Zaretskii <eliz@gnu.org> wrote:

> AFAIU, Stefan wants to highlight the pattern's characters in basic
> style as well.

> So the gap is not so large after all.

Stefan deliberately chose a "not in your face" color,
just as a hint that there is a face there, something you see
it after some squinting. So it doesn't fit the bill at all, by
its very inception.

> character.  Though I don't quite understand why you are so against
> highlighting possible next character(s).

Because 99% flex system in the world works like that. And
for good reason. Yes, I must be a cool kid after all.

> I think you give up too early.

I must have written about 50 mails in this thread and explained
the same things some 20 times. I call this about time.

João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Nov 2019 17:21:30 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> > I think you give up too early.
> 
> I must have written about 50 mails in this thread and explained
> the same things some 20 times. I call this about time.

OK, I will shut up from now on.

(You may wish to reflect on how your style of arguing makes it hard to
participate in discussions where you take part.)

FTR, your intention to reverse which parts of completion candidates
are prominent and which aren't is completely against my intuition, and
I'm firmly against that.  I don't care how other flex systems out
there work.

Re: VOTE: Changing completions-common-part face's default

[Stefan Monnier]

Dmitry Gutov <dgutov@yandex.ru> wrote:
> blue3 is a good enough color, and it fits the default theme.  So unless we
> can agree on a better idea, and unless other people object, let's go with
> blue3 and call it a day.

OK, we have one vote here, great.
João, you also mentioned some objections to "blue3", do they still
stand, or are you willing to settle for "blue3" now?

Eli?

I suspect there will be more reactions once(if?) we actually install the
change, so it'd be good to install this soon so we have time to
tweak/revert it before the release.

> If we have more time to think on this,

Of course, we can change it again later, so there will definitely be
more time to think about it.  But I'd like more people to be exposed to
it so opinions are based on more concrete experience (e.g. I was quite
convinced at first that I'd be OK with "common=bold firstdiff=underline"
and it's only after some days of use that I realized that I really
didn't like the "firstdiff=underline" part).


        Stefan

Re: VOTE: Changing completions-common-part face's default

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 08 Nov 2019 14:19:12 -0500
> 
> Dmitry Gutov <dgutov@yandex.ru> wrote:
> > blue3 is a good enough color, and it fits the default theme.  So unless we
> > can agree on a better idea, and unless other people object, let's go with
> > blue3 and call it a day.
> 
> OK, we have one vote here, great.
> João, you also mentioned some objections to "blue3", do they still
> stand, or are you willing to settle for "blue3" now?
> 
> Eli?

Please tell which face will use this, whether as foreground or as
background color, and what will that face highlight in various
completion scenarios.  (I think I know the answers, but I'm so
flabbergasted by the other thread that I no longer believe my memory
and my ability to draw conclusions from what you mentioned in this
one.  Sorry.)

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 08.11.2019 21:19, Stefan Monnier wrote:
>> If we have more time to think on this,
> Of course, we can change it again later, so there will definitely be
> more time to think about it.  But I'd like more people to be exposed to
> it so opinions are based on more concrete experience (e.g. I was quite
> convinced at first that I'd be OK with "common=bold firstdiff=underline"
> and it's only after some days of use that I realized that I really
> didn't like the "firstdiff=underline" part).

FWIW, I think the other suggestion is a radically different idea. So 
whatever experience people have with the proposed change would be 
orthogonal to it.

Re: VOTE: Changing completions-common-part face's default

[Stefan Monnier]

> Please tell which face will use this, whether as foreground or as
> background color, and what will that face highlight in various
> completion scenarios.  (I think I know the answers, but I'm so
> flabbergasted by the other thread that I no longer believe my memory
> and my ability to draw conclusions from what you mentioned in this
> one.  Sorry.)

Concretely the patch I'm suggesting is the one below, which makes it so
the "common-part" (e.g. the common prefix in the case basic completion)
of the completions has a blue3 foreground rather than being identical to
`default` (we'd also want to find a color for the dark background case,
but... one step at a time).


        Stefan


diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 43dd277a2e..e4c4edd114 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -1687,7 +1687,9 @@ completions-first-difference
   "Face for the first character after point in completions.
 See also the face `completions-common-part'.")
 
-(defface completions-common-part '((t nil))
+(defface completions-common-part
+  '((((background light)) :foreground "blue3")
+    (t nil))
   "Face for the parts of completions which matched the pattern.
 See also the face `completions-first-difference'.")
 

Re: BIKESHED: completion faces

[João Távora]

On Fri, Nov 8, 2019 at 7:06 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > > I think you give up too early.
> > I must have written about 50 mails in this thread and explained
> > the same things some 20 times. I call this about time.
> OK, I will shut up from now on.

FTR, nobody asked to shut up or anything like that.
You said I gave up too early, I said that after so much time
arguing with you and others I wouldn't say that it is "early".

I simple have no more arguments to give you or anything new
to contribute. I've made at least 3 different proposals here. You
objected to all of them: I simply have nothing else to contribute.

> (You may wish to reflect on how your style of arguing makes it hard to
> participate in discussions where you take part.)

Thank you very much for your time and energy throughout this
discussion, Eli. I have been in many many discussions with
fruitful outcomes, here and elsewhere, some of them including
you. This wasn't one of them. That's just the way it is, sometimes.
You will, however, understand that I dispense with such unneeded
paternalism.

> FTR, your intention to reverse which parts of completion candidates
> are prominent and which aren't is completely against my intuition,

My proposal was only to control highlighting for a new style, 'flex'
which I believe you don't use.  So I don't think "reversion" is an
adequate characterization. (I did indeed propose that earlier, but
I gave it up after you tried it and told me you didn't like it).

> I don't care how other flex systems out there work.

That's a bad call.  But 100% your prerrogative.

João









-- 
João Távora

Re: VOTE: Changing completions-common-part face's default

[João Távora]

On Fri, Nov 8, 2019 at 7:21 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> Dmitry Gutov <dgutov@yandex.ru> wrote:
> > blue3 is a good enough color, and it fits the default theme.  So unless we
> > can agree on a better idea, and unless other people object, let's go with
> > blue3 and call it a day.
>
> OK, we have one vote here, great.
> João, you also mentioned some objections to "blue3", do they still
> stand, or are you willing to settle for "blue3" now?

As I said earlier, I abstain on "blue3".  It makes me squint, but I won't
be using it.  It will make more new users squint, but less than "blue4".

João

Re: VOTE: Changing completions-common-part face's default

[Dmitry Gutov]

On 07.11.2019 17:54, João Távora wrote:

>> I don't understand this. flex is deterministic. Which part exactly
>> cannot be trusted?
> 
> Precisely, I thought you wouldn't.  Flex is designed to let you
> smash some keys that vaguely fit into your intended completion,
> or at least enter the spectrum where you kinda remember the
> completion to be.  Some Flex implementations even allow you
> to reverse keys so that when my lazy hand types "lyaz" it still
> maches "the-lazy-fox".  When scoring, flex makes some mistakes
> (depending on how good/sophisticated the scoring algorithm is).
> So it's important to understand why the suggestion is at that
> place.

Important for the completion style author mostly, though. It's not like 
a regular user could do something about it. Isn't that right?

> You might think this double-checking "wastes your synapses".
> It's a bargain indeed. it doesn't work for everyone. But it's
> nonsensical to have flex with that crucial part crippled.
> 
> So that in the case of "lzy", I can choose between "the-lazy-fox"
> and "le-zorg-yorg". Both can be reasonable intentions at different
> points in time.

Okay, but... our particular flex implementation doesn't do the free-form 
matching like that. Do you intend to extend it in that direction?

>> I'm not sure I can agree with that. Company highlights the common part,
>>   not simply whatever part completion style chooses to highlight.
> 
> If it's somehow so important for company to highlight always the
> common part, we can put a 'this-is-part-of-the-common-part' prop
> there, too, for company's benetif.  Sniffing for faces, the way I
> implemented it, was really a low-hanging-hack anyway.  The patch
> to sniff for a property is trivial.
> 
> So I wouldn't worry about that.

Okay. That would still mean two different execution flows.

> By the way, doesn't this requirement for company clash with your
> view of the other Emacs frontends?  Why is it important for company
> to always highlight the common part, but not for Emacs's icomplete,
> or *Completions*?

Because its main frontend is a popup which looks and, to an extent, 
behaves like the completion popups in other editors. So it's really 
expected to behave more or less the same. company-tng also implements 
one of the simplified workflows from the other editors.

Re: VOTE: Changing completions-common-part face's default

[João Távora]

On Fri, Nov 8, 2019 at 11:38 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> Important for the completion style author mostly, though. It's not like
> a regular user could do something about it. Isn't that right?

Not at all, no. I was using flex before I became an author of a
flex method and I liked it. I cant unbecome an author of a flex
method.

I'm 100% with Drew here (thou not so scientific).  I use flex (before
and after flex in Emacs proper) to explore a namespace of symbols.  I
work in an application that has may thousands of very long CL symbols.
In these symbols, some words appear inconsistently, middle, end or
beginning, depending on the "era" they were created (this is a system
as old as Emacs, by the way, dates to the 80's).

One of the main use cases I have for the highlighting is to
input that keyword to see what "era" symbol I want.  I don't know
the symbol I want until I start seeing the large list. Then I add another
little sub-concept word to narrow down, and repeat. Flex helps
me explore. I wish I could share with you the function names
I have to work with, but as you can understand, I can't.

As another data point, before I had flex I used a lot of SLIME's
apropos with a regexp to the same effect.  Either it had the
highlighting already or I added it (I don't remember now). I also
used slime-fuzzy, another flex written around 2006 for SLIME.
It already had the highlighting of the pattern (bold :-) Anyway that
highlighting was precious.

> > So that in the case of "lzy", I can choose between "the-lazy-fox"
> > and "le-zorg-yorg". Both can be reasonable intentions at different
> > points in time.
> Okay, but... our particular flex implementation doesn't do the free-form
> matching like that.

It does: "lzy" matches both "the-lazy-fox" and "le-zorg-yorg".

> Do you intend to extend it in that direction?

You must have meant "lyz". Yes it is a possiblity that we want to
extend it, but the scoring becomes much more complex. And
the system becomes slower. We start approaching
google-search-box level smartness and I would need to
read up on that first.

> Okay. That would still mean two diffeerent execution flows.

Don't know what you mean exactly. But yes, I guess. When
using prefix I am focused on one thing, when using flex I am
focused on another thing.  I think that already happens anyway
(the internal focus of the user).

But I don't use prefix much nowadays, fortunately. The beginning
of a name has no special place in my mind.

But Juri has given as an example a search box that does
exactly this mixing _in the same search results_.

> Because its main frontend is a popup which looks and, to an extent,
> behaves like the completion popups in other editors. So it's really
> expected to behave more or less the same. company-tng also implements
> one of the simplified workflows from the other editors.

I don't see much difference in the way I use popups and icomplete's
minibuffer for example.  Or *Completions*. A popup is just placed
in a different part of the screen, and, versus *Completions* it is more
"jittery". But it's exactly the same for me.

João

RE: VOTE: Changing completions-common-part face's default

[Drew Adams]

> > That would still mean two diffeerent execution flows.
> 
> Don't know what you mean exactly. But yes, I guess. When
> using prefix I am focused on one thing, when using flex I am
> focused on another thing.  I think that already happens anyway
> (the internal focus of the user).
> 
> But I don't use prefix much nowadays, fortunately. The beginning
> of a name has no special place in my mind.
> 
> But Juri has given as an example a search box that does
> exactly this mixing _in the same search results_.

I haven't really followed all of the messages,
and I'm not sure what is meant here, by "mixing
completion styles in the same search results".

But if it's about mixing different completion
styles or methods to narrow the same initial
set of candidates, then Icicles does offer that.

You can provide multiple separate search
patterns, and you can change completion method
anytime, including between patterns.

You can, for example, match using a regexp
pattern, then match those matches (also, i.e.,
ANDing) using a flex pattern, and so on.

* `M-(' cycles among the currently available
  completion methods.

* `S-SPC' switches to another pattern (ANDed).

The ability to switch methods anytime is
useful.  The ability to add patterns (getting
the intersection of their matched-candidate
sets) is _very_ useful.  The latter is used
all the time.  The former is used less often.

The former (switching methods on the fly) is
described here:

https://www.emacswiki.org/emacs/Icicles_-_Completion_Methods_and_Styles

The latter (progressive completion) is
described here:

https://www.emacswiki.org/emacs/Icicles_-_Progressive_Completion
___

Which set of completion methods is available
can even be made command-specific.  That's
described here:

https://www.emacswiki.org/emacs/Icicles_-_Completion_Methods_and_Styles#CommandSpecificCompletionMethods
___

Finally, a mention that besides ANDing
matches of multiple patterns (`S-SPC'),
you can complement the current set of
matches, using `C-~'.

Matching A, then also B, then `C-~' gives
you matches of A that are NOT also matches
of B.  (And as mentioned, the completion
methods for A and B need not be the same.)

Each time you use `C-~' you remove matches
of the last pattern: provide a pattern,
then `C-~' to remove its matches, repeat.

This "Chipping away of the non-elephant" is
described here:

https://www.emacswiki.org/emacs/Icicles_-_Nutshell_View#ChippingAway

Re: VOTE: Changing completions-common-part face's default

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Fri, 08 Nov 2019 16:42:50 -0500
> 
> Concretely the patch I'm suggesting is the one below, which makes it so
> the "common-part" (e.g. the common prefix in the case basic completion)
> of the completions has a blue3 foreground rather than being identical to
> `default`

Hm... so the "common part" will be highlighted as blueish and the
"first difference" will stay just bold?  Does that look good?  IOW,
will the bold character be seen right next to a blue one?  Maybe we
should also change the default definition of the first-difference
face?

I'd actually expect this face to use the background, because that's
how other applications behave, and I thought this change was motivated
by that?

Also, what will this do to the output of the more aggressive
completion styles?

> we'd also want to find a color for the dark background case,

Right.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Nov 2019 22:49:36 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> I have been in many many discussions with fruitful outcomes, here
> and elsewhere, some of them including you. This wasn't one of them.

Likewise.

> > FTR, your intention to reverse which parts of completion candidates
> > are prominent and which aren't is completely against my intuition,
> 
> My proposal was only to control highlighting for a new style, 'flex'
> which I believe you don't use.  So I don't think "reversion" is an
> adequate characterization.

By "reversal" I mean that the part we highlight is the part that
matched what the user typed, and the part we don't highlight is what
the user might type to narrow the search.  We currently do the
opposite.

Re: VOTE: Changing completions-common-part face's default

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 9 Nov 2019 00:24:06 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>  Drew Adams <drew.adams@oracle.com>, emacs-devel <emacs-devel@gnu.org>
> 
> On Fri, Nov 8, 2019 at 11:38 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> > Important for the completion style author mostly, though. It's not like
> > a regular user could do something about it. Isn't that right?
> 
> Not at all, no. I was using flex before I became an author of a
> flex method and I liked it. I cant unbecome an author of a flex
> method.
> 
> I'm 100% with Drew here (thou not so scientific).  I use flex (before
> and after flex in Emacs proper) to explore a namespace of symbols.  I
> work in an application that has may thousands of very long CL symbols.
> In these symbols, some words appear inconsistently, middle, end or
> beginning, depending on the "era" they were created (this is a system
> as old as Emacs, by the way, dates to the 80's).
> 
> One of the main use cases I have for the highlighting is to
> input that keyword to see what "era" symbol I want.  I don't know
> the symbol I want until I start seeing the large list. Then I add another
> little sub-concept word to narrow down, and repeat. Flex helps
> me explore. I wish I could share with you the function names
> I have to work with, but as you can understand, I can't.

It is OK to have special needs, and it is OK to program Emacs to cater
to those special needs.  That's why Emacs has minor modes and user
options.

But the important part is to recognize these special needs as such.
It's your and Drew's insistence on making that the default is what at
least I object to, and I think so does Dmitry.  Even if you or someone
else want Emacs to always behave like that, they are still special
needs, in this case special to your usage patterns.

Let's agree to disagree about the importance of the feature you want
to see by default.  The Emacs way of agreeing to disagree is to have
an opt-in feature; it can later become the default if enough users
tell they want it.

Re: BIKESHED: completion faces

[João Távora]

On Sat, Nov 9, 2019 at 6:57 AM Eli Zaretskii <eliz@gnu.org> wrote:

> By "reversal" I mean that the part we highlight is the part that
> matched what the user typed, and the part we don't highlight is what
> the user might type to narrow the search.  We currently do the
> opposite.

Yes. You call this an inconsistency, I don't. Or maybe I could
call it that, but the value that arises from accepting it is
greater than the drawbacks, IMO.

-- 
João Távora

Re: VOTE: Changing completions-common-part face's default

[João Távora]

On Sat, Nov 9, 2019 at 7:30 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > One of the main use cases I have for the highlighting is to
> > input that keyword to see what "era" symbol I want.  I don't know
> > the symbol I want until I start seeing the large list. Then I add another
> > little sub-concept word to narrow down, and repeat. Flex helps
> > me explore. I wish I could share with you the function names
> > I have to work with, but as you can understand, I can't.
> It is OK to have special needs,

I could just as well illustrate your use 'prefix'
need-to-see-the-first-difference case as a special need.
"special" in contrast to what I would suppose is the "norm"
is not very useful here, especially when applied to a new
feature. My position is that your argument is weak also
because you don't have enough first-hand knowledge to
talk about special needs __in the "flex" context__

I view your argument about consistency as a bit stronger,
but also ultimately weaker than mine. Tough the  fact that
it is stronger is why I first proposed the reversion of
faces in the 'basic' case.

I gave my example as a data point. Take my word that it affects
more people that I work with. SLIME's authors must also felt
the "special need" back in 2006.  Probably all the authors/users
of every other flex system in the world also feel that "special
need" (though, they are using other editors and have stated your
don't value their opinion much -- contrary to me.).

> an opt-in feature; it can later become the default if enough users
> tell they want it.

Sure, of course, I hope so.

João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 9 Nov 2019 11:20:30 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> On Sat, Nov 9, 2019 at 6:57 AM Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > By "reversal" I mean that the part we highlight is the part that
> > matched what the user typed, and the part we don't highlight is what
> > the user might type to narrow the search.  We currently do the
> > opposite.
> 
> Yes. You call this an inconsistency, I don't.

I call this "reversal".  I guess we now agree about this part, whereas
previously you said there was no reversal.

> Or maybe I could call it that, but the value that arises from
> accepting it is greater than the drawbacks, IMO.

I object to making this the default behavior, because it's the exact
opposite of what we do now.  I already suggested to make what you want
an optional behavior, but AFAIU you rejected that, not sure I
understand why.

Re: VOTE: Changing completions-common-part face's default

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 9 Nov 2019 11:42:31 +0000
> Cc: Dmitry Gutov <dgutov@yandex.ru>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	Drew Adams <drew.adams@oracle.com>, emacs-devel <emacs-devel@gnu.org>
> 
> > It is OK to have special needs,
> 
> I could just as well illustrate your use 'prefix'
> need-to-see-the-first-difference case as a special need.

You could, but since that's how Emacs worked for the last 11 years, I
don't see how such a claim would stand.

> "special" in contrast to what I would suppose is the "norm"
> is not very useful here, especially when applied to a new
> feature. My position is that your argument is weak also
> because you don't have enough first-hand knowledge to
> talk about special needs __in the "flex" context__

I tried that with different styles, including flex, and I still don't
agree with you about the need to reverse the emphasis.  So what you
think must be the default at least for me doesn't sound 100% correct,
as I find flex very usable for me with the current method of emphasis.

> I gave my example as a data point. Take my word that it affects
> more people that I work with. SLIME's authors must also felt
> the "special need" back in 2006.  Probably all the authors/users
> of every other flex system in the world also feel that "special
> need" (though, they are using other editors and have stated your
> don't value their opinion much -- contrary to me.).

I'm sorry, but that's not how defaults in Emacs change, not because
you give me your word or describe what other applications do.  Not
when the proposed behavior is in such stark contrast to what we've
been doing for years.  We first introduce such a new feature as
optional, and later change it to be the default based on user reports
and complaints.  I suggest to do the same in this case.

Re: BIKESHED: completion faces

[João Távora]

On Sat, Nov 9, 2019 at 11:52 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Sat, 9 Nov 2019 11:20:30 +0000
> > Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>,
> >       Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> >
> > On Sat, Nov 9, 2019 at 6:57 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > By "reversal" I mean that the part we highlight is the part that
> > > matched what the user typed, and the part we don't highlight is what
> > > the user might type to narrow the search.  We currently do the
> > > opposite.
> >
> > Yes. You call this an inconsistency, I don't.
>
> I call this "reversal".  I guess we now agree about this part, whereas
> previously you said there was no reversal.

By "reversal", I understand reversing the application
of the two faces in the 'basic' completion style. I
proposed that, but I withdrew it.  I think Stefan's
original proposal, the very first one that started the
thread, also goes in this direction somewhat. In my
"reversal" proposal, no new faces or aliases need
be added.

By "consistency" I am talking about 'prefix' and
'flex' both highlighting the pattern's characters in
the same face. The reverse, "inconsistency", means
that 'prefix' chooses to highlight the first difference,
and 'flex' chooses to highlight the pattern's characters.
I _don't_ call this "reversal" (though you may, in which
case we need a new word to describe the situation I
explained in the previous paragraph). My "renames"
proposal, which differs from my "reversal" proposal,
brings about this "inconsistency", which I think,
in relative terms, is a drawback dwarfed by the
advantages.

So, to me,  these are two separate things.

> > Or maybe I could call it that, but the value that arises from
> > accepting it is greater than the drawbacks, IMO.
>
> I object to making this the default behavior, because it's the exact
> opposite of what we do now.  I already suggested to make what you want
> an optional behavior, but AFAIU you rejected that, not sure I
> understand why.

Because it is already an optional behaviour. I wrote "we already
have that: themes".  That's the reason. I for example
use M-x customize-face to fix the situation and unless the
situation advances, I will advise users of flex to do the same.
There's no point in adding anything more, IMO.

João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 9 Nov 2019 12:14:28 +0000
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> 
> On Sat, Nov 9, 2019 at 11:52 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: João Távora <joaotavora@gmail.com>
> > > Date: Sat, 9 Nov 2019 11:20:30 +0000
> > > Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>,
> > >       Stefan Monnier <monnier@iro.umontreal.ca>, Dmitry Gutov <dgutov@yandex.ru>
> > >
> > > On Sat, Nov 9, 2019 at 6:57 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > >
> > > > By "reversal" I mean that the part we highlight is the part that
> > > > matched what the user typed, and the part we don't highlight is what
> > > > the user might type to narrow the search.  We currently do the
> > > > opposite.
> > >
> > > Yes. You call this an inconsistency, I don't.
> >
> > I call this "reversal".  I guess we now agree about this part, whereas
> > previously you said there was no reversal.
> 
> By "reversal", I understand reversing the application
> of the two faces in the 'basic' completion style.

I explained above what I meant by "reversal".  And you agreed, so why
are we still arguing about this part?

> > I object to making this the default behavior, because it's the exact
> > opposite of what we do now.  I already suggested to make what you want
> > an optional behavior, but AFAIU you rejected that, not sure I
> > understand why.
> 
> Because it is already an optional behaviour. I wrote "we already
> have that: themes".  That's the reason. I for example
> use M-x customize-face to fix the situation and unless the
> situation advances, I will advise users of flex to do the same.
> There's no point in adding anything more, IMO.

It's up to you, but I think an option that causes the matched part to
be prominent and the next character to type not prominent (which is
AFAIU what you want) is much easier and lightweight for users than
loading a theme or customizing a face.  Later, if it turns out the
majority of users want that, we could make that option the default.

Re: VOTE: Changing completions-common-part face's default

[João Távora]

On Sat, Nov 9, 2019 at 12:04 PM Eli Zaretskii <eliz@gnu.org> wrote:
> You could, but since that's how Emacs worked for the last 11 years, I
> don't see how such a claim would stand.

Well, SLIME's fuzzy is older than that, for example.  So age
is not a major factor factor here. Also, the current behaviour
was adopted when there was NO flex in Emacs.  If there would
have been, the decision would have been different, IMO.

> I tried that with different styles, including flex, and I still don't
> agree with you about the need to reverse the emphasis.  So what you
> think must be the default at least for me doesn't sound 100% correct,
> as I find flex very usable for me with the current method of emphasis.

I'm very happy that you find it usable. However, I have the
conviction -- what I've been trying to persuade you of that
conviction -- that it is _more_ useful for the remaining
population of existing flex users _when_ it has the highlighting
I propose.  If you take one of my proposals, then you, as
an individual from a new population of flex users migrating
from 'prefix' can still customize it to adhere to your longstanding
'prefix' habits. Or you can adapt to what I think is the majority's
use of 'flex'.

> I'm sorry, but that's not how defaults in Emacs change, not because
> you give me your word or describe what other applications do.  Not
> when the proposed behavior is in such stark contrast to what we've
> been doing for years.  We first introduce such a new feature as
> optional, and later change it to be the default based on user reports
> and complaints.  I suggest to do the same in this case.

Again, I must stress that the latest proposal is NOT a change in
defaults because the people starting out emacs with no settings
whatsoever would see _absolutely_ no new behaviour.  It is only
people that _enable_ the new matching style 'flex', which I
implemented some months ago, that would see the highlighting
the way I propose it.

I will, as I have already done, concede that it is a change in what
you and others have called "consistency" between matching styles.
I have explained enough, I think, why I think this isn't a big problem.
But in any case it is NOT a change in defaults, unless you are
considering that there is already a large number of 'flex' users
tracking the Emacs master branch between some months ago and
today that rely on it functioning the way it does.  I do not think that
is the case.

(Note, to avoid a frequent misunderstanding, that I'm talking strictly
about my latest 2 proposals, alright? the "renames" one and the
"new-face-just-for-flex".  Again I did indeed propose earlier a change
in defaults, which I abandoned.)

Also, as another data point, you need not travel beyond Emacs-land
to see examples that corroborate my vision. You needn't even travel
farther than GNU ELPA, I believe.  Company's tooltip behaves like I
describe, and so does Ivy.  A little farther you have Helm, an extremely
popular extension, and SLIME, and SLY.

João

Re: BIKESHED: completion faces

[João Távora]

On Sat, Nov 9, 2019 at 12:20 PM Eli Zaretskii <eliz@gnu.org> wrote:
> I explained above what I meant by "reversal".  And you agreed, so why
> are we still arguing about this part?

Because, with all do respect, I don't want to mischaracterize what
I agreed on. I was clarifying exactly what I think.  I don't have
to agree just for agreement's sake.  I believe those things that
I wrote.  If there's agreement, fine, if there isn't then fine, too.

> It's up to you, but I think an option that causes the matched part to
> be prominent and the next character to type not prominent (which is
> AFAIU what you want) is much easier and lightweight for users than
> loading a theme or customizing a face.

I do believe loading a theme is much simpler than asking the
user to set an option for such a minute aspect. I load themes
because I trust many such good decisions of the theme's
author: I don't want to worry about individual ones.  If I ever do,
I'll customize a face. Faces and variables that control
application of faces are, in my humble opinion of course,
crazy. Even for Emacs :-)

João

Re: BIKESHED: completion faces

[João Távora]

On Sat, Nov 9, 2019 at 1:29 PM João Távora <joaotavora@gmail.com> wrote:

> application of faces are, in my humble opinion of course,
> crazy. Even for Emacs :-)

Using "crazy" was a really bad choice.  I meant "over the
top". Sincere apologies in advance if you thought I was
characterizing you with that word.

João

Re: VOTE: Changing completions-common-part face's default

[Stefan Monnier]

> Hm... so the "common part" will be highlighted as blueish and the
> "first difference" will stay just bold?

Yes.

> Does that look good?

I'll pass judgment on aesthetics.

It doesn't make the "first-difference" less legible IMO, and it does
make the "common-part" easier to find, which are my main criteria, so it
does look good in this sense.  I'm not trying to find a perfect
solution, just an incremental improvement.

> IOW, will the bold character be seen right next to a blue one?

Yes.

> Maybe we should also change the default definition of the
> first-difference face?

I'd rather postpone this decision to later.

> I'd actually expect this face to use the background, because that's
> how other applications behave, and I thought this change was motivated
> by that?

While highlighting the "common-part" is definitely standard, there's
definitely not a standard *way* to highlight it.  While a background
color is used in some cases I have the vague impression that it might be
a minority rather than a majority of cases.

> Also, what will this do to the output of the more aggressive
> completion styles?

For flex's "foo matched to frobot" it highlights the "f" the "o"s in
blue3 and the "t" in bold.

>> we'd also want to find a color for the dark background case,
> Right.

But I'd also rather postpone this decision to later.


        Stefan

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 9 Nov 2019 13:29:30 +0000
> Cc: Ergus <spacibba@aol.com>, Dmitry Gutov <dgutov@yandex.ru>,
>  Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel <emacs-devel@gnu.org>
> 
> > It's up to you, but I think an option that causes the matched part to
> > be prominent and the next character to type not prominent (which is
> > AFAIU what you want) is much easier and lightweight for users than
> > loading a theme or customizing a face.
> 
> I do believe loading a theme is much simpler than asking the
> user to set an option for such a minute aspect. I load themes
> because I trust many such good decisions of the theme's
> author: I don't want to worry about individual ones.  If I ever do,
> I'll customize a face. Faces and variables that control
> application of faces are, in my humble opinion of course,
> crazy. Even for Emacs :-)

I think you are looking at this from the implementation POV.  From
users' POV, an option (or a minor mode) is a better way when we are
talking not just about changing colors and other face attributes, but
about changing behavior in significant ways.  In this case, what is
implemented via faces changes the behavior, because a face prominently
different from the default becomes like the default, and another face
makes the reverse transformation.  Think of this as a binary mode that
makes either the first-difference or the common part prominent:
flipping a variable is an easily understood and easily discovered way
of getting each user the behavior he/she wants.

By contrast, loading a theme or customizing 2 faces in coordinated way
is much less discoverable and much more complex.

So I think providing an option is better then telling users to
customize themes or faces.  It also has a clear and simple path for
becoming the default in the future.  Thus, it is IMO a compromise that
should leave us happier than the all-or-nothing kind of solution that
you are defending.

Re: VOTE: Changing completions-common-part face's default

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Sat, 09 Nov 2019 10:25:52 -0500
> 
> > I'd actually expect this face to use the background, because that's
> > how other applications behave, and I thought this change was motivated
> > by that?
> 
> While highlighting the "common-part" is definitely standard, there's
> definitely not a standard *way* to highlight it.  While a background
> color is used in some cases I have the vague impression that it might be
> a minority rather than a majority of cases.

Don't "the other" applications use bold instead of color?

> >> we'd also want to find a color for the dark background case,
> > Right.
> 
> But I'd also rather postpone this decision to later.

Not sure why: a lot of people use dark background in Emacs, AFAIK.

Re: VOTE: Changing completions-common-part face's default

[Stefan Monnier]

>> While highlighting the "common-part" is definitely standard, there's
>> definitely not a standard *way* to highlight it.  While a background
>> color is used in some cases I have the vague impression that it might be
>> a minority rather than a majority of cases.
> Don't "the other" applications use bold instead of color?

It seems many do, but not all.  Not sure about what is more common in
this respect within the context of text editors.

>> >> we'd also want to find a color for the dark background case,
>> > Right.
>> But I'd also rather postpone this decision to later.
> Not sure why: a lot of people use dark background in Emacs, AFAIK.

Just because I prefer to do it one step at a time (I have the
impression(delusion?) it can help keep the discussion more focused).
I do hope the second step of choosing a color for the dark-background
case will come before Emacs-27, tho.


        Stefan

RE: VOTE: Changing completions-common-part face's default

[Drew Adams]

> But the important part is to recognize these
> special needs as such. It's your and
> Drew's insistence on making that the default
  ^^^^^^^^^^^^^^^^^
> is what at least I object to, and I think
> so does Dmitry.  

Huh?  I know I'm the bogeyman for you, Eli
(or one of them), but I've said _nothing_
about any defaults here.

In fact, I made it clear that this whole
question, about vanilla faces etc. for
completions doesn't affect my use at all.
_Not my problem._

The only effect of any of this discussion on
what I do, AFAICT, might be that I'll have to
modify Icicles a bit to ignore different
face-fiddling from vanilla Emacs, in order to
continue doing what Icicles has been doing.

My contribution to this and the companion
thread has been _only_ to speak to what
Icicles does (and has long done), as a point
of comparison/reference, and perhaps as food
for thought wrt completion behavior.

You want to draw battle lines, but I'm not
in this battle - sorry.

Let me be very clear: not my problem.  Do
whatever you guys want wrt vanilla Emacs
highlighting completion (non-)matches.
Doesn't affect me.

Do I care at all?  Sure - I care about
vanilla Emacs - a lot.  But what you decide
here is unlikely to affect my use or that
of anyone who might use my code.  At least
I think (and hope) it's unlikely.  And I
take no part in the current arguments.

Back off, please.  Not interested in being
one of your punching bags.

Re: BIKESHED: completion faces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1103 bytes --]

On Sat, Nov 9, 2019, 15:31 Eli Zaretskii <eliz@gnu.org> wrote:
> > I'll customize a face. Faces and variables that control
> > application of faces are, in my humble opinion of course,
> > crazy. Even for Emacs :-)
> I think you are looking at this from the implementation POV.  From
> users' POV, an option (or a minor mode) is a better way when we are

Implementation-wise, I think I would handle an `if`
and a variable.  It a very bad idea for usability,
that's all. Just what would you call this thing? No,
no, forgot I asked :-)
Anyway, whatever the name I strongly oppose it.
I can't stop you from adding it, of course.

Eli, although I do feel some itch to reply to your
characterization of my solution as "all-or-nothing"
I will leave the discussion here, since I have nothing
new to add.  Contrary to what you might believe from
the number of words I spent on this, I am not sad
or demotivated. I plan on continuing to contribute to
Emacs and some of these things will require (or are
already waiting for <wink>) your feedback, so I suggest
we move on to those.

João

[-- Attachment #2: Type: text/html, Size: 1488 bytes --]

Re: VOTE: Changing completions-common-part face's default

[Juri Linkov]

>> > That would still mean two diffeerent execution flows.
>>
>> Don't know what you mean exactly. But yes, I guess. When
>> using prefix I am focused on one thing, when using flex I am
>> focused on another thing.  I think that already happens anyway
>> (the internal focus of the user).
>>
>> But I don't use prefix much nowadays, fortunately. The beginning
>> of a name has no special place in my mind.
>>
>> But Juri has given as an example a search box that does
>> exactly this mixing _in the same search results_.
>
> I haven't really followed all of the messages,
> and I'm not sure what is meant here, by "mixing
> completion styles in the same search results".

Chromium mixes different completion styles in one completion list,
for example: https://www.chromestory.com/2018/05/omnibox-redesign/

It highlights completion matches using bold that works also for blue links
(e.g. "nytimes.com" in the example above).  I'm not sure what completion style
supports listing links in the *Completions* buffer, but using blue for links
and blue for matches will make them indistinguishable from each other
in the *Completions* buffer.

If changing the defaults is so highly sensitive topic, I don't see
what would be wrong with adding a new face inheriting from
completions-first-difference and applying only to flex-like
completion styles.  No change in defaults will maintain the status quo.

Another advantage of adding a new face is that anyone can
customize flex highlighting without affecting the faces
of the prefix completion style.

Re: VOTE: Changing completions-common-part face's default

[João Távora]

On Sat, Nov 9, 2019 at 10:07 PM Juri Linkov <juri@linkov.net> wrote:

> If changing the defaults is so highly sensitive topic, I don't see
> what would be wrong with adding a new face inheriting from
> completions-first-difference and applying only to flex-like
> completion styles.  No change in defaults will maintain the status quo.
>
> Another advantage of adding a new face is that anyone can
> customize flex highlighting without affecting the faces
> of the prefix completion style.

My latest proposal was exactly this, but it has been rejected.  Much
as I agree with you, I think it's better we leave this alone for now,
and wait until more people start using flex, so we can get more feedback.

-- 
João Távora

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 09.11.2019 17:31, Eli Zaretskii wrote:
> I think you are looking at this from the implementation POV.  From
> users' POV, an option (or a minor mode) is a better way when we are
> talking not just about changing colors and other face attributes, but
> about changing behavior in significant ways.  In this case, what is
> implemented via faces changes the behavior, because a face prominently
> different from the default becomes like the default, and another face
> makes the reverse transformation.  Think of this as a binary mode that
> makes either the first-difference or the common part prominent:
> flipping a variable is an easily understood and easily discovered way
> of getting each user the behavior he/she wants.

How would that work? Having two faces have different default definitions 
depending on the value of the variable?

Any custom face would override that decision. And chasing all theme 
authors to make them honor the variable is a lot of effort.

Re: BIKESHED: completion faces

[Eli Zaretskii]

> Cc: spacibba@aol.com, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 10 Nov 2019 11:18:46 +0200
> 
> On 09.11.2019 17:31, Eli Zaretskii wrote:
> > I think you are looking at this from the implementation POV.  From
> > users' POV, an option (or a minor mode) is a better way when we are
> > talking not just about changing colors and other face attributes, but
> > about changing behavior in significant ways.  In this case, what is
> > implemented via faces changes the behavior, because a face prominently
> > different from the default becomes like the default, and another face
> > makes the reverse transformation.  Think of this as a binary mode that
> > makes either the first-difference or the common part prominent:
> > flipping a variable is an easily understood and easily discovered way
> > of getting each user the behavior he/she wants.
> 
> How would that work? Having two faces have different default definitions 
> depending on the value of the variable?

Either that or a function that redefines the face definitions, e.g. by
aliasing/copying from other faces.

> Any custom face would override that decision. And chasing all theme 
> authors to make them honor the variable is a lot of effort.

I'm not sure I understand what customization scenarios you have in
mind, so I don't think I see the problem.

Re: BIKESHED: completion faces

[Dmitry Gutov]

On 14.11.2019 11:45, Eli Zaretskii wrote:
> I'm not sure I understand what customization scenarios you have in
> mind, so I don't think I see the problem.

Basically any custom theme that sets the colors of these two faces?

Luckily, they're currently normally left alone in custom themes. But 
that could change when we add color to completions-common-face.

An arbitrary theme like that is likely not to honor the said variable. 
Would aliasing help somehow?

Re: BIKESHED: completion faces

[Eli Zaretskii]

> Cc: joaotavora@gmail.com, spacibba@aol.com, monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Thu, 14 Nov 2019 12:40:39 +0200
> 
> On 14.11.2019 11:45, Eli Zaretskii wrote:
> > I'm not sure I understand what customization scenarios you have in
> > mind, so I don't think I see the problem.
> 
> Basically any custom theme that sets the colors of these two faces?
> 
> Luckily, they're currently normally left alone in custom themes. But 
> that could change when we add color to completions-common-face.
> 
> An arbitrary theme like that is likely not to honor the said variable. 
> Would aliasing help somehow?

Maybe.  But since my proposal was rejected, it sounds like waste of
time to discuss its details, no?

Re: BIKESHED: completion faces

[João Távora]

On Thu, Nov 14, 2019 at 2:16 PM Eli Zaretskii <eliz@gnu.org> wrote:
> Maybe.  But since my proposal was rejected, it sounds like waste of
> time to discuss its details, no?

If you're talking about my input, let it be known that I have
no power to reject any proposal (probably a good idea,
too ;-) ).

So you shouldn't let my principled opposition demotivate you,
if you really do think it's the right way forward.

João Távora

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 14 Nov 2019 14:19:18 +0000
> Cc: Dmitry Gutov <dgutov@yandex.ru>, Ergus <spacibba@aol.com>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel <emacs-devel@gnu.org>
> 
> On Thu, Nov 14, 2019 at 2:16 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > Maybe.  But since my proposal was rejected, it sounds like waste of
> > time to discuss its details, no?
> 
> If you're talking about my input, let it be known that I have
> no power to reject any proposal

Of course, you do: you write the code, so you have all the power to
refuse to put something in it.

Re: BIKESHED: completion faces

[João Távora]

On Thu, Nov 14, 2019 at 2:42 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > If you're talking about my input, let it be known that I have
> > no power to reject any proposal
>
> Of course, you do: you write the code, so you have all the power to
> refuse to put something in it.

Hmmmm.  OK.  Cool, I guess. :)

Then let me phrase my position as such. Even though I am
skeptic of the merits of the change, I don't wish to exercise that
power fully, i.e. I won't use that veto in this particular case,
especially given that you are the head Emacs maintainer.

João

Re: BIKESHED: completion faces

[Juri Linkov]

>> > I think you are looking at this from the implementation POV.  From
>> > users' POV, an option (or a minor mode) is a better way when we are
>> > talking not just about changing colors and other face attributes, but
>> > about changing behavior in significant ways.  In this case, what is
>> > implemented via faces changes the behavior, because a face prominently
>> > different from the default becomes like the default, and another face
>> > makes the reverse transformation.  Think of this as a binary mode that
>> > makes either the first-difference or the common part prominent:
>> > flipping a variable is an easily understood and easily discovered way
>> > of getting each user the behavior he/she wants.
>>
>> How would that work? Having two faces have different default definitions
>> depending on the value of the variable?
>
> Either that or a function that redefines the face definitions, e.g. by
> aliasing/copying from other faces.

I thought you meant an option/variable or a minor mode for flex style,
so when it's enabled in the *Completions* buffer with flex styles, then
it puts the same face on different parts of the buffer than prefix style,
i.e. the flex style activates this minor mode and it puts bold on common
parts.  Then there is no need to meddle with face definitions.  Isn't it so?

Re: BIKESHED: completion faces

[João Távora]

On Sat, Nov 16, 2019 at 9:07 PM Juri Linkov <juri@linkov.net> wrote:

> I thought you meant an option/variable or a minor mode for flex style,
> so when it's enabled in the *Completions* buffer with flex styles, then
> it puts the same face on different parts of the buffer than prefix style,
> i.e. the flex style activates this minor mode and it puts bold on common
> parts.  Then there is no need to meddle with face definitions.  Isn't it so?

A style can't activate minor modes: a style is a symbol
designating a couple of methods for matching patterns to
possible completions.  The preferred style is indicated by
the user, a minor mode or even by the completion table
itself.

It's true that something like fido-mode could activate the
variable or minor mode that i think Eli is proposing, the
same way it activates the flex style and other variables.
Emacs -Q + M-x fido-mode would then produce a familiar flex
experience.  But it would be a terrible experience for users
who have loaded themes designed to make flex look good on
modes other than fido-mode.

So I don't think it's an excellent solution.

João

Re: BIKESHED: completion faces

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 16 Nov 2019 23:40:35 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, Dmitry Gutov <dgutov@yandex.ru>, Ergus <spacibba@aol.com>, 
> 	emacs-devel <emacs-devel@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>
> 
> A style can't activate minor modes

But a minor mode can change how a style behaves.

Re: BIKESHED: completion faces

[João Távora]

On Sun, Nov 17, 2019 at 5:27 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Sat, 16 Nov 2019 23:40:35 +0000
> > Cc: Eli Zaretskii <eliz@gnu.org>, Dmitry Gutov <dgutov@yandex.ru>, Ergus <spacibba@aol.com>,
> >       emacs-devel <emacs-devel@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>
> >
> > A style can't activate minor modes
>
> But a minor mode can change how a style behaves.

Sure. That's what I meant when I wrote fido-mode
could do that.  But that would bring about the
problem I described.

João