bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Morgan Willcock]

I believe there is an issue with completion-preview-mode in that it
doesn't seem to find all completions for a case-insensitive completion
at point function.

Evaluating the code below in `emacs -Q` will put the current buffer into
fundamental mode with a case-insensitive completion at point function:

  (defun test-completion-at-point-function ()
    "Complete the symbol at point."
    (pcase (bounds-of-thing-at-point 'symbol)
      (`(,beg . ,end)
       (list beg end
             (completion-table-case-fold
              (completion-table-dynamic
               (lambda (string)
                 (list "TestSymbol"))))))))

  (fundamental-mode)
  (remove-hook 'completion-at-point-functions #'tags-completion-at-point-function)
  (add-hook 'completion-at-point-functions #'test-completion-at-point-function)
  (completion-preview-mode 1)

Once the code is evaluated:

- Typing "tes" into the buffer will not display the completion preview.

- The default completion mechanism (as initiated with complete-symbol
  via C-M-i) will work with no problems, and replace "tes" with the
  completion "TestSymbol".

- Typing "Tes" into the buffer will display the completion preview.

I would expect that the completion preview should be displayed for both
"tes" and "Tes" since a completion is available to use in both cases.



In GNU Emacs 30.0.91 (build 2, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0, Xaw3d scroll bars) of 2024-09-12 built on inspiron
Windowing system distributor 'The X.Org Foundation', version 11.0.12101007
System Description: Debian GNU/Linux 12 (bookworm)

Configured using:
 'configure --with-native-compilation=aot --with-xml2
 --with-x-toolkit=lucid'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG
LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP X11 XAW3D
XDBE XIM XINPUT2 XPM LUCID ZLIB

Important settings:
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: ELisp/d

Minor modes in effect:
  server-mode: t
  global-corfu-mode: t
  corfu-mode: t
  jabber-activity-mode: t
  etags-regen-mode: t
  which-key-mode: t
  global-devil-mode: t
  devil-mode: t
  erc-ring-mode: t
  erc-netsplit-mode: t
  erc-menu-mode: t
  erc-list-mode: t
  erc-irccontrols-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  erc-imenu-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-spelling-mode: t
  erc-track-mode: t
  erc-match-mode: t
  erc-autojoin-mode: t
  erc-networks-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  minibuffer-depth-indicate-mode: t
  ido-everywhere: t
  recentf-mode: t
  global-display-fill-column-indicator-mode: t
  display-fill-column-indicator-mode: t
  global-hl-line-mode: t
  display-time-mode: t
  flyspell-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/mwillcock/.emacs.d/custom/powershell.el/powershell hides /home/mwillcock/.emacs.d/custom/powershell
/home/mwillcock/.emacs.d/elpa/which-key-3.6.1/which-key hides /usr/local/share/emacs/30.0.91/lisp/which-key
/home/mwillcock/.emacs.d/elpa/modus-themes-4.5.0/theme-loaddefs hides /usr/local/share/emacs/30.0.91/lisp/theme-loaddefs

Features:
(shadow sort ecomplete mail-extr emacsbug cursor-sensor server cal-iso
autorevert face-remap init corfu sly sly-completion sly-buttons
sly-messages sly-common apropos etags fileloop arc-mode archive-mode
hyperspec powershell php-mode mode-local consult-imenu consult php-face
php php-project jabber jabber-ourversion hippie-exp hexrgb fsm sgml-mode
facemenu srv dns starttls tls goto-addr yaml-mode markdown-mode lua-mode
advice edmacro kmacro etags-regen kixtart-docstrings cape kixtart-mode
tempo org-msg let-alist color ox-odt rng-loc rng-uri rng-parse rng-match
rng-dt rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util
ox-latex ox-icalendar ox-html table ox-ascii ox-publish ox org-attach
htmlize gnus-msg gnus-icalendar icalendar gnus-dired gnus-cite which-key
devil delight comp comp-cstr ags-mode speedbar ezimage dframe shadowfile
eglot jsonrpc xref flymake diff ert ewoc debug backtrace warnings python
project cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align
cc-engine cc-vars cc-defs erc-ring erc-netsplit erc-menu erc-list
erc-goodies erc-imenu imenu erc-pcomplete erc-button erc-fill erc-stamp
erc-sasl erc-sasl-ecdsa-nist256p-challenge erc-sasl-scram-sha-512
erc-sasl-scram-sha-256 erc-sasl-scram-sha-1 erc-sasl-external
erc-sasl-plain sasl-scram-sha256 sasl-scram-rfc sasl-scram-sha-1 rfc2104
hex-util sasl sasl-anonymous sasl-login sasl-plain erc-spelling
erc-track erc-match erc-join erc erc-backend erc-networks erc-common
erc-compat compat erc-loaddefs ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util vc-git diff-mode
track-changes vc-dispatcher org-indent oc-basic cl-extra ol-eww eww
url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect
gnus-art mm-uu mml2015 gnus-sum ol-docview doc-view filenotify jka-compr
image-mode exif ol-bibtex bibtex ol-bbdb ol-w3m ol-doi org-link-doi appt
diary-lib diary-loaddefs org-capture ob-shell ob-plantuml ob-dot
org-goto org-clock comp-run comp-common org-duration org-agenda
org-element org-persist org-id org-element-ast inline avl-tree generator
org-refile org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src sh-script smie treesit executable ob-comint org-pcomplete
org-list org-footnote org-faces org-entities noutline outline
ob-emacs-lisp ob-core ob-eval org-version org-cycle org-table ol
org-fold org-fold-core org-keys oc org-loaddefs thingatpt org-compat
org-macs autoinsert compile bookmark savehist crm minibuf-eldef mb-depth
ido tramp-cache time-stamp tramp-sh tramp rx trampver tramp-integration
files-x tramp-message tramp-compat xdg format-spec tramp-loaddefs
recentf tree-widget shell pcomplete comint ansi-osc ansi-color ring
easy-mmode display-fill-column-indicator hl-line time gnus-group
gnus-undo gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail mail-source
utf7 nnoo parse-time iso8601 gnus-spec gnus-int gnus-range gnus-win gnus
nnheader range cus-edit pp cus-load wid-edit dictionary
external-completion dictionary-connection flyspell ispell shr pixel-fill
kinsoku url-file svg xml dom modus-vivendi-theme modus-themes mm-view
mml-smime smime gnutls dig smtpmail message sendmail yank-media puny
dired dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068
gnus-util text-property-search time-date mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader epg-config help-mode dabbrev
find-func cal-menu calendar cal-loaddefs desktop frameset pcase
cape-autoloads consult-autoloads corfu-autoloads delight-autoloads
do-at-point-autoloads fsm-autoloads htmlize-autoloads lua-mode-autoloads
markdown-mode-autoloads modus-themes-autoloads package-lint-autoloads
php-mode-autoloads rainbow-mode-autoloads renpy-mode-autoloads
sly-autoloads totp-auth-autoloads base32-autoloads info
vertico-autoloads wgrep-autoloads which-key-autoloads
yaml-mode-autoloads package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie generate-lisp-file
url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs icons password-cache json subr-x map byte-opt
gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
touch-screen tool-bar dnd fontset image regexp-opt fringe tabulated-list
replace newcomment text-mode lisp-mode prog-mode register page tab-bar
menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse
jit-lock font-lock syntax font-core term/tty-colors frame minibuffer
nadvice seq simple cl-generic indonesian philippine cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop
case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo x-toolkit xinput2 x multi-tty move-toolbar
make-network-process native-compile emacs)

Memory information:
((conses 16 1662552 108846) (symbols 48 50603 5)
 (strings 32 260961 5577) (string-bytes 1 7128489) (vectors 16 148988)
 (vector-slots 8 2543291 190764) (floats 8 781 818)
 (intervals 56 123265 150) (buffers 984 21))

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hello Morgan,

Morgan Willcock <morgan@ice9.digital> writes:

> I believe there is an issue with completion-preview-mode in that it
> doesn't seem to find all completions for a case-insensitive completion
> at point function.

Thanks for this report.

> Evaluating the code below in `emacs -Q` will put the current buffer into
> fundamental mode with a case-insensitive completion at point function:
>
>   (defun test-completion-at-point-function ()
>     "Complete the symbol at point."
>     (pcase (bounds-of-thing-at-point 'symbol)
>       (`(,beg . ,end)
>        (list beg end
>              (completion-table-case-fold
>               (completion-table-dynamic
>                (lambda (string)
>                  (list "TestSymbol"))))))))
>
>   (fundamental-mode)
>   (remove-hook 'completion-at-point-functions #'tags-completion-at-point-function)
>   (add-hook 'completion-at-point-functions #'test-completion-at-point-function)
>   (completion-preview-mode 1)
>
> Once the code is evaluated:
>
> - Typing "tes" into the buffer will not display the completion preview.
>
> - The default completion mechanism (as initiated with complete-symbol
>   via C-M-i) will work with no problems, and replace "tes" with the
>   completion "TestSymbol".
>
> - Typing "Tes" into the buffer will display the completion preview.
>
> I would expect that the completion preview should be displayed for both
> "tes" and "Tes" since a completion is available to use in both cases.

We could display the completion preview also when you type a prefix that
differs from a candidate prefix only in case (and in fact we already do,
when completion-ignore-case is non-nil), but the question is what to do
when you accept/insert this completion.  Namely, changing the prefix
from "tes" to "Tes" when you accept the completion suggestion adds
complications both in terms of implementation and in terms of UX, and
I'm not sure that this added complexity would be justified.

Do you have, or know of, a concrete use case with a completion table
that behaves like your test-completion-at-point-function?


Best,

Eshel

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Morgan Willcock]

Eshel Yaron <me@eshelyaron.com> writes:

Hi Eshel,

> We could display the completion preview also when you type a prefix that
> differs from a candidate prefix only in case (and in fact we already do,
> when completion-ignore-case is non-nil), but the question is what to do
> when you accept/insert this completion.  Namely, changing the prefix
> from "tes" to "Tes" when you accept the completion suggestion adds
> complications both in terms of implementation and in terms of UX, and
> I'm not sure that this added complexity would be justified.

My expectation was that a "completion preview" is using exactly the same
criteria as the default completion mechanism, and would indicate when a
completion was available to use.

The difference in the behaviour also means completion-preview-mode
cannot reliably be used as the entry-point (by repeatedly calling
completion-preview-complete) for the completion interface.

> Do you have, or know of, a concrete use case with a completion table
> that behaves like your test-completion-at-point-function?

I think it would be rare to see one explicitly set to be
case-insensitive, but I found the issue because I use a Cape capf
transformer to wrap an existing capf backend:

https://elpa.gnu.org/packages/doc/cape.html#Capf-transformers

It is feasible to do that for any language where the syntax is
case-insensitive.  Here is an example of it being done:

https://git.sr.ht/~mew/kixtart-mode/tree/v1.3.2/item/doc/kixtart-mode.texi#L804

(I wrote this mode and its manual, but anyone using cape-capf-case-fold
is likely to see the same problem.)

Thanks,
Morgan

-- 
Morgan Willcock

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Morgan Willcock <morgan@ice9.digital> writes:

> Eshel Yaron <me@eshelyaron.com> writes:
>
> Hi Eshel,
>
>> We could display the completion preview also when you type a prefix that
>> differs from a candidate prefix only in case (and in fact we already do,
>> when completion-ignore-case is non-nil), but the question is what to do
>> when you accept/insert this completion.  Namely, changing the prefix
>> from "tes" to "Tes" when you accept the completion suggestion adds
>> complications both in terms of implementation and in terms of UX, and
>> I'm not sure that this added complexity would be justified.
>
> My expectation was that a "completion preview" is using exactly the same
> criteria as the default completion mechanism, 

This expectation is (at least presently) not warranted.  Completion
Preview mode draws from the same pool of completions as
completion-at-point (it uses the same completion-at-point-functions),
but as a different frontend it makes different use of these completions.
Namely, Completion Preview mode is for prefix completion.  This is the
purpose of this feature.  We can discuss generalizing Completion Preview
mode to incorporate other kinds (or styles) of completion, that does
require some non-trivial considerations, though.

In the benefit of other users, could you please share what led you to
have this expectation?  Perhaps we can improve the documentation to
avoid such confusions.

> and would indicate when a completion was available to use.

That it does, for prefix completions.

> The difference in the behaviour also means completion-preview-mode
> cannot reliably be used as the entry-point (by repeatedly calling
> completion-preview-complete) for the completion interface.

That's not the goal of this command, so that's OK, I think :)
Completion Preview mode is not intended to replace completion-at-point,
you can use C-M-i with the preview enabled, and get the same results
that would without it.

>> Do you have, or know of, a concrete use case with a completion table
>> that behaves like your test-completion-at-point-function?
>
> I think it would be rare to see one explicitly set to be
> case-insensitive, but I found the issue because I use a Cape capf
> transformer to wrap an existing capf backend:
>
> https://elpa.gnu.org/packages/doc/cape.html#Capf-transformers
>
> It is feasible to do that for any language where the syntax is
> case-insensitive.  

If case is not important, and you're fine with completing "tes" in your
example to "testSymbol", then setting completion-ignore-case to non-nil
should get you there, IIUC.

Another option, if case is insignificant, is to use a completion table
that adopts the case of the input and returns completion candidates
adjusted accordingly, instead of ignoring the case of the input and
returning candidates that match up to case differences.  We do something
like that in ispell-completion-at-point, for example, and it works well
with Completion Preview mode.

> Here is an example of it being done:
>
> https://git.sr.ht/~mew/kixtart-mode/tree/v1.3.2/item/doc/kixtart-mode.texi#L804
>
> (I wrote this mode and its manual, but anyone using cape-capf-case-fold
> is likely to see the same problem.)

Thanks, but this points to example code in the documentation of a
package I'm not familiar with, so it's hard to discern its significance.
(The manual looks really nice BTW!)

If my suggestions above don't help with your use case, would you like to
try and propose a patch that does?


Eshel

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Morgan Willcock]

Eshel Yaron <me@eshelyaron.com> writes:

>>> We could display the completion preview also when you type a prefix that
>>> differs from a candidate prefix only in case (and in fact we already do,
>>> when completion-ignore-case is non-nil), but the question is what to do
>>> when you accept/insert this completion.  Namely, changing the prefix
>>> from "tes" to "Tes" when you accept the completion suggestion adds
>>> complications both in terms of implementation and in terms of UX, and
>>> I'm not sure that this added complexity would be justified.
>>
>> My expectation was that a "completion preview" is using exactly the same
>> criteria as the default completion mechanism,
>
> This expectation is (at least presently) not warranted.  Completion
> Preview mode draws from the same pool of completions as
> completion-at-point (it uses the same completion-at-point-functions),
> but as a different frontend it makes different use of these completions.
> Namely, Completion Preview mode is for prefix completion.  This is the
> purpose of this feature.  We can discuss generalizing Completion Preview
> mode to incorporate other kinds (or styles) of completion, that does
> require some non-trivial considerations, though.
>
> In the benefit of other users, could you please share what led you to
> have this expectation?  Perhaps we can improve the documentation to
> avoid such confusions.

It was just the name.  I assumed it to be a preview of the same
completions which are in normal use, rather than a separate mechanism.

>> and would indicate when a completion was available to use.
>
> That it does, for prefix completions.

There are gaps in my knowledge here, but is there something about a
prefix completion which means that it has to force a case-sensitive
match, or is there something else about the preview interface which
imposes the restriction?

If the completion at point function is case-insensitive at which point
in the process is that case-insensitivity being lost?

>> The difference in the behaviour also means completion-preview-mode
>> cannot reliably be used as the entry-point (by repeatedly calling
>> completion-preview-complete) for the completion interface.
>
> That's not the goal of this command, so that's OK, I think :)

Yes, I appreciate that the aim is just to show the preview, but it does
actually function as an entry point if the input produces multiple
matches.

(It works less well with the default completion interface because it is
fairly easy to leave the completion buffer open, but the default
behaviour of Corfu aligns with it quite well because the Corfu frame
automatically closes.)

> Completion Preview mode is not intended to replace completion-at-point,
> you can use C-M-i with the preview enabled, and get the same results
> that would without it.

I think my main point is that I wouldn't necessarily know to press C-M-i
because of the lack of preview.

>>> Do you have, or know of, a concrete use case with a completion table
>>> that behaves like your test-completion-at-point-function?
>>
>> I think it would be rare to see one explicitly set to be
>> case-insensitive, but I found the issue because I use a Cape capf
>> transformer to wrap an existing capf backend:
>>
>> https://elpa.gnu.org/packages/doc/cape.html#Capf-transformers
>>
>> It is feasible to do that for any language where the syntax is
>> case-insensitive.
>
> If case is not important, and you're fine with completing "tes" in your
> example to "testSymbol", then setting completion-ignore-case to non-nil
> should get you there, IIUC.

I found out the hard way that completion-ignore-case isn't something
that can be set as buffer-local:

https://lists.gnu.org/archive/html/emacs-devel/2024-03/msg00302.html

I wouldn't want to set case-insensitivity globally.

> Another option, if case is insignificant, is to use a completion table
> that adopts the case of the input and returns completion candidates
> adjusted accordingly, instead of ignoring the case of the input and
> returning candidates that match up to case differences.  We do something
> like that in ispell-completion-at-point, for example, and it works well
> with Completion Preview mode.

Do you mean this would need changes to the completion at point function
rather than to completion-preview-mode?

>> Here is an example of it being done:
>>
>> https://git.sr.ht/~mew/kixtart-mode/tree/v1.3.2/item/doc/kixtart-mode.texi#L804
>>
>> (I wrote this mode and its manual, but anyone using cape-capf-case-fold
>> is likely to see the same problem.)
>
> Thanks, but this points to example code in the documentation of a
> package I'm not familiar with, so it's hard to discern its significance.
> (The manual looks really nice BTW!)

I don't think there are any restrictions on where cape-capf-case-fold
can be used, but fundamentally the only goal is to insert the completion
in the same form that it was returned from the completion at point
function, but not require the user to match the case.

Or to put it another way, because completion-ignore-case cannot be
buffer-local, cape-capf-case-fold is the only feasible way I've seen to
make a case-insensitive language easier to complete.

> If my suggestions above don't help with your use case, would you like to
> try and propose a patch that does?

I would be willing to give it a go, but I think I am missing some fairly
critical knowledge about why the same completion at point function is in
use but the match result is different.

Thank you for the quick and detailed reply,
Morgan

-- 
Morgan Willcock

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi,

Morgan Willcock <morgan@ice9.digital> writes:

> There are gaps in my knowledge here, but is there something about a
> prefix completion which means that it has to force a case-sensitive
> match, or is there something else about the preview interface which
> imposes the restriction?

Prefix completion does not force case-sensitivity.  However, in
Completion Preview mode we only show (preview) the suffix.  Since we
only suggest a suffix, the completion preview cannot convey to you, the
user, suggested alternations for the prefix, such as case alternations.
This is in contrast with e.g. the *Completions* buffer where you see a
list of whole completion candidates.  So the preview is inherently all
about adding text, not changing existing text.

We can conceive of a way to indicate suggested changes to the existing
prefix, probably with an overlay.  I think that's an interesting idea to
explore, but it would make Completion Preview mode more complex (both in
terms of implementation, and more importantly, from a user's
perspective).  So we'll only do it if enables an important use case.

> If the completion at point function is case-insensitive at which point
> in the process is that case-insensitivity being lost?

completion-preview--try-table is where we first filter appropriate
completion candidates.

[...]

>> If case is not important, and you're fine with completing "tes" in your
>> example to "testSymbol", then setting completion-ignore-case to non-nil
>> should get you there, IIUC.
>
> I found out the hard way that completion-ignore-case isn't something
> that can be set as buffer-local:
>
> https://lists.gnu.org/archive/html/emacs-devel/2024-03/msg00302.html
>
> I wouldn't want to set case-insensitivity globally.

I'll take a closer look at that discussion, but I'm pretty sure you can
in fact set completion-ignore-case buffer-locally.  It works as expected
in my quick tests, and it's done in other places in Emacs already :|

>> Another option, if case is insignificant, is to use a completion table
>> that adopts the case of the input and returns completion candidates
>> adjusted accordingly, instead of ignoring the case of the input and
>> returning candidates that match up to case differences.  We do something
>> like that in ispell-completion-at-point, for example, and it works well
>> with Completion Preview mode.
>
> Do you mean this would need changes to the completion at point function
> rather than to completion-preview-mode?

Yes.  Basically, use a different wrapper instead of completion-table-case-fold.

[...]

>> If my suggestions above don't help with your use case, would you like to
>> try and propose a patch that does?
>
> I would be willing to give it a go, but I think I am missing some fairly
> critical knowledge about why the same completion at point function is in
> use but the match result is different.

Great.  The "matches" (the return value of the completion functions) are
the same.  We post-process them, including filtering them, differently,
since we present them differently (we only display the suffix, etc.).
Check out completion-preview--try-table in particular, and feel free to
ask if you find anything unclear.  Admittedly, the documentation of some
of these internal functions could be a bit more detailed :)


Best,

Eshel

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eshel Yaron <me@eshelyaron.com> writes:

[...]

>>> If case is not important, and you're fine with completing "tes" in your
>>> example to "testSymbol", then setting completion-ignore-case to non-nil
>>> should get you there, IIUC.
>>
>> I found out the hard way that completion-ignore-case isn't something
>> that can be set as buffer-local:
>>
>> https://lists.gnu.org/archive/html/emacs-devel/2024-03/msg00302.html
>>
>> I wouldn't want to set case-insensitivity globally.
>
> I'll take a closer look at that discussion, but I'm pretty sure you can
> in fact set completion-ignore-case buffer-locally.  It works as expected
> in my quick tests, and it's done in other places in Emacs already :|

FYI, I've added a new user option completion-preview-ignore-case on the
master branch (commit f1794a17b62), which controls case sensitivity for
Completion Preview mode specifically.  This is definitely safe to set
buffer-locally.  The docstring of this new variable should hopefully
also explain exactly what's the expected behavior in this regard.


Cheers,

Eshel

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Morgan Willcock]

Eshel Yaron <me@eshelyaron.com> writes:

> FYI, I've added a new user option completion-preview-ignore-case on the
> master branch (commit f1794a17b62), which controls case sensitivity for
> Completion Preview mode specifically.  This is definitely safe to set
> buffer-locally.  The docstring of this new variable should hopefully
> also explain exactly what's the expected behavior in this regard.

Thanks for adding the new option.  For my use case I don't think I would
be able to use it because the case mismatch between the text already
typed and the completion will effectively be generating a completion
which may not actually exist.

e.g. If a valid completion was "FooBar", and I typed "foo", having the
final result as "fooBar" wouldn't be ideal.

I've been trying to think of a way to get it to work without disturbing
the user interface that already exists.  Maybe it would be possible
fixup the result in a post-insertion hook if one existed - such a hook
might be generally useful anyway.

In the long term though, I really think that the preview should be
allowed to overlap with the text already typed and potentially replace
it when the completion is inserted, but I appreciate that it would be a
big change in the internals to please a minority.

Morgan

-- 
Morgan Willcock

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi,

Morgan Willcock <morgan@ice9.digital> writes:

> Eshel Yaron <me@eshelyaron.com> writes:
>
>> FYI, I've added a new user option completion-preview-ignore-case on the
>> master branch (commit f1794a17b62), which controls case sensitivity for
>> Completion Preview mode specifically.  This is definitely safe to set
>> buffer-locally.  The docstring of this new variable should hopefully
>> also explain exactly what's the expected behavior in this regard.
>
> Thanks for adding the new option.  For my use case I don't think I would
> be able to use it because the case mismatch between the text already
> typed and the completion will effectively be generating a completion
> which may not actually exist.
>
> e.g. If a valid completion was "FooBar", and I typed "foo", having the
> final result as "fooBar" wouldn't be ideal.

Fair enough, although earlier you mentioned that your use case was
completing symbols in a case-insensitive language, so I wonder what's
wrong with "fooBar", or "foobar" for that matter.  They are all the same
as the completion candidate, "FooBar", ignoring case differences, no?

> I've been trying to think of a way to get it to work without disturbing
> the user interface that already exists.  Maybe it would be possible
> fixup the result in a post-insertion hook if one existed - such a hook
> might be generally useful anyway.

We have the :exit-function (which your capf can provide as an extra
property alongside the completion table it returns), and indeed you
could use that to "fix" the case of your prefix after completing it.
That's an interesting solution, I think.  Or do you have some other kind
of hook in mind?


Best,

Eshel

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Morgan Willcock]

Eshel Yaron <me@eshelyaron.com> writes:

>> e.g. If a valid completion was "FooBar", and I typed "foo", having the
>> final result as "fooBar" wouldn't be ideal.
>
> Fair enough, although earlier you mentioned that your use case was
> completing symbols in a case-insensitive language, so I wonder what's
> wrong with "fooBar", or "foobar" for that matter.  They are all the same
> as the completion candidate, "FooBar", ignoring case differences, no?

The symbols would technically be valid in the language, but the
completion candidates will be using a particular style of naming.

Fundamentally, if I am meant to be writing using PascalCase then I don't
want the result to be accidentally transformed into camelCase.

>> I've been trying to think of a way to get it to work without disturbing
>> the user interface that already exists.  Maybe it would be possible
>> fixup the result in a post-insertion hook if one existed - such a hook
>> might be generally useful anyway.
>
> We have the :exit-function (which your capf can provide as an extra
> property alongside the completion table it returns), and indeed you
> could use that to "fix" the case of your prefix after completing it.

I don't think it is the responsibility of the completion function
because it is not the completion function that is modifying the buffer.

> That's an interesting solution, I think.  Or do you have some other kind
> of hook in mind?

Just a hook that runs after it was completion-preview-mode that inserted
a completion.

I thought that completion-preview-active-mode-hook might have been
suitable, but it looks like that is being called on every key-press when
typing in the buffer (without the completion preview).

Thanks,
Morgan

-- 
Morgan Willcock

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi,

Morgan Willcock <morgan@ice9.digital> writes:

> Eshel Yaron <me@eshelyaron.com> writes:
>
>>> I've been trying to think of a way to get it to work without disturbing
>>> the user interface that already exists.  Maybe it would be possible
>>> fixup the result in a post-insertion hook if one existed - such a hook
>>> might be generally useful anyway.
>>
>> We have the :exit-function (which your capf can provide as an extra
>> property alongside the completion table it returns), and indeed you
>> could use that to "fix" the case of your prefix after completing it.
>
> I don't think it is the responsibility of the completion function
> because it is not the completion function that is modifying the buffer.
>>
>> That's an interesting solution, I think.  Or do you have some other kind
>> of hook in mind?
>
> Just a hook that runs after it was completion-preview-mode that inserted
> a completion.

I'm quite not sure what you mean by responsibility, but in terms of
functionality, :exit-function is the existing interface for doing
something after a completion is inserted, and Completion Preview mode
supports this interface.

If :exit-function doesn't work for your use case, please explain why,
and if there's indeed a need for another hook, it's easy to add one :)


Thanks,

Eshel

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Morgan Willcock]

Eshel Yaron <me@eshelyaron.com> writes:

>>>> I've been trying to think of a way to get it to work without disturbing
>>>> the user interface that already exists.  Maybe it would be possible
>>>> fixup the result in a post-insertion hook if one existed - such a hook
>>>> might be generally useful anyway.
>>>
>>> We have the :exit-function (which your capf can provide as an extra
>>> property alongside the completion table it returns), and indeed you
>>> could use that to "fix" the case of your prefix after completing it.
>>
>> I don't think it is the responsibility of the completion function
>> because it is not the completion function that is modifying the buffer.
>>>
>>> That's an interesting solution, I think.  Or do you have some other kind
>>> of hook in mind?
>>
>> Just a hook that runs after it was completion-preview-mode that inserted
>> a completion.
>
> I'm quite not sure what you mean by responsibility, but in terms of
> functionality, :exit-function is the existing interface for doing
> something after a completion is inserted, and Completion Preview mode
> supports this interface.

The hook would be interface specific because it is the interface which
has modified the case of the completion.

> If :exit-function doesn't work for your use case, please explain why,
> and if there's indeed a need for another hook, it's easy to add one :)

My reasons for not using :exit-function would be:

- If the completion was inserted using a different interface then the
  function would be running unnecessarily.

- For completion functions that I didn't write, :exit-function isn't
  going to be set with the expectation that a particular completion
  interface and case preference is used to insert the candidate.

That said, I am just trying to think of an easy mechanism to preserve
the case of the candidate.  A hook may generally be useful, but adding a
hook does not solve the underlying problem that I am describing here.

Perhaps it is best to just pause for the moment while you get additional
feedback from other users.

Thanks,
Morgan

-- 
Morgan Willcock

bug#73234: 30.0.91; completion-preview-mode doesn't trigger for case-insensitive capf

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Morgan Willcock <morgan@ice9.digital> writes:

> Eshel Yaron <me@eshelyaron.com> writes:
>
>> If :exit-function doesn't work for your use case, please explain why,
>> and if there's indeed a need for another hook, it's easy to add one :)
>
> My reasons for not using :exit-function would be:

Thanks, these make sense.

> - If the completion was inserted using a different interface then the
>   function would be running unnecessarily.

Right, although this needn't be significant in terms of performance.

> - For completion functions that I didn't write, :exit-function isn't
>   going to be set with the expectation that a particular completion
>   interface and case preference is used to insert the candidate.

The idea is that you use a capf wrapper that adds the :exit-function,
when needed, like cape-capf-case-fold in your example.

> That said, I am just trying to think of an easy mechanism to preserve
> the case of the candidate.  A hook may generally be useful, but adding a
> hook does not solve the underlying problem that I am describing here.
>
> Perhaps it is best to just pause for the moment while you get additional
> feedback from other users.

All right, let me know if you have further thoughts or a potential patch,
and thank you for your engagement!


Best,

Eshel