bug#68699: Eglot completion has stopped working in some cases

bug#68699: Eglot completion has stopped working in some cases

[Brennan Vincent]

I bisected the issue to e93d99a4a0ce578249304dce350465c580a49892, so
including João.

In some cases when attempting to complete using eglot, I am getting
errors rather than a list of completions. I haven't been able to debug
why but it seems to be happening in "pcm" completion style.

For example, in a Rust crate with the following main file:

fn foo_len() {
}

fn main() {
    len
}

positioning my cursor after "len" and attempting completion yields the
following error message:

completion-pcm--merge-completions: Internal error: _mm_blend_epi16(…)
(use std::arch::x86_64::_mm_blend_epi16) doesn’t match \`len

Full backtrace:

Debugger entered--Lisp error: (error #("Internal error: _mm_blend_epi16(…) (use std::arch::x86_64::_mm_blend_epi16) doesn’t match \\`len" 16 17 (eglot--lsp-item (:label #("_mm_blend_epi16(…) (use std::arch::x86_64::_mm_blend_epi16)" 0 1 (eglot--lsp-item #5) 5 8 (face completions-common-part)) :kind 3 :detail "unsafe fn(__m128i, __m128i) -> __m128i" :documentation (:kind "markdown" :value "Blend packed 16-bit integers from `a` and `b` using the mask `IMM8`.\n\nThe mask bits determine the selection. A clear bit selects the\ncorresponding element of `a`, and a set bit the corresponding\nelement of `b`.\n\n[Intel's documentation](https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm_blend_epi16)") :deprecated :json-false :sortText "fffffff0" :filterText #("_mm_blend_epi16" 5 8 (face completions-common-part)) :insertTextFormat 2 :textEdit (:range (:start (:line 4 :character 4) :end (:line 4 :character 7)) :newText "_mm_blend_epi16(${1:a}, ${2:b})$0") :additionalTextEdits [] :command (:title "triggerParameterHints" :command "rust-analyzer.triggerParameterHints") :data (:position (:textDocument (:uri "file:///home/brennan/test_rust/src/bin/test_rust.rs") :position (:line 4 :character 7)) :imports [(:full_import_path "std::arch::x86_64::_mm_blend_epi16" :imported_name "_mm_blend_epi16")]))) 21 24 (face completions-common-part)))
  signal(error (#("Internal error: _mm_blend_epi16(…) (use std::arch::x86_64::_mm_blend_epi16) doesn’t match \\`len" 16 17 (eglot--lsp-item (:label #("_mm_blend_epi16(…) (use std::arch::x86_64::_mm_blend_epi16)" 0 1 (eglot--lsp-item #6) 5 8 (face completions-common-part)) :kind 3 :detail "unsafe fn(__m128i, __m128i) -> __m128i" :documentation (:kind "markdown" :value "Blend packed 16-bit integers from `a` and `b` using the mask `IMM8`.\n\nThe mask bits determine the selection. A clear bit selects the\ncorresponding element of `a`, and a set bit the corresponding\nelement of `b`.\n\n[Intel's documentation](https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm_blend_epi16)") :deprecated :json-false :sortText "fffffff0" :filterText #("_mm_blend_epi16" 5 8 (face completions-common-part)) :insertTextFormat 2 :textEdit (:range (:start (:line 4 :character 4) :end (:line 4 :character 7)) :newText "_mm_blend_epi16(${1:a}, ${2:b})$0") :additionalTextEdits [] :command (:title "triggerParameterHints" :command "rust-analyzer.triggerParameterHints") :data (:position (:textDocument (:uri "file:///home/brennan/test_rust/src/bin/test_rust.rs") :position (:line 4 :character 7)) :imports [(:full_import_path "std::arch::x86_64::_mm_blend_epi16" :imported_name "_mm_blend_epi16")]))) 21 24 (face completions-common-part))))
  error("Internal error: %s doesn't match %s" #("_mm_blend_epi16(…) (use std::arch::x86_64::_mm_blend_epi16)" 0 1 (eglot--lsp-item (:label #2 :kind 3 :detail "unsafe fn(__m128i, __m128i) -> __m128i" :documentation (:kind "markdown" :value "Blend packed 16-bit integers from `a` and `b` using the mask `IMM8`.\n\nThe mask bits determine the selection. A clear bit selects the\ncorresponding element of `a`, and a set bit the corresponding\nelement of `b`.\n\n[Intel's documentation](https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm_blend_epi16)") :deprecated :json-false :sortText "fffffff0" :filterText #("_mm_blend_epi16" 5 8 (face completions-common-part)) :insertTextFormat 2 :textEdit (:range (:start (:line 4 :character 4) :end (:line 4 :character 7)) :newText "_mm_blend_epi16(${1:a}, ${2:b})$0") :additionalTextEdits [] :command (:title "triggerParameterHints" :command "rust-analyzer.triggerParameterHints") :data (:position (:textDocument (:uri "file:///home/brennan/test_rust/src/bin/test_rust.rs") :position (:line 4 :character 7)) :imports [(:full_import_path "std::arch::x86_64::_mm_blend_epi16" :imported_name "_mm_blend_epi16")]))) 5 8 (face completions-common-part)) "\\`len")
  completion-pcm--merge-completions((... ... ... ... ... ... ... ... ... ... ... ... ... ... ...) ("len"))
  completion-pcm--merge-try(("len") (... ... ... ... ... ... ... ... ... ... ... ... ... ... ...) "" "")
  completion-pcm-try-completion("len" #f(compiled-function (pattern pred action) #<bytecode 0x19ec37e26e8f5965>) nil 3)
  #f(compiled-function (style) #<bytecode -0x32c2df51d2b464c>)(partial-completion)
  #f(compiled-function (elt) #<bytecode 0x19004498146b82cc>)(partial-completion)
  mapc(#f(compiled-function (elt) #<bytecode 0x19004498146b82cc>) (eglot--dumb-flex basic partial-completion emacs22))
  seq-do(#f(compiled-function (elt) #<bytecode 0x19004498146b82cc>) (eglot--dumb-flex basic partial-completion emacs22))
  seq-some(#f(compiled-function (style) #<bytecode -0x32c2df51d2b464c>) (eglot--dumb-flex basic partial-completion emacs22))
  completion--nth-completion(1 "len" #f(compiled-function (pattern pred action) #<bytecode 0x19ec37e26e8f5965>) nil 3 (metadata (category . eglot-capf) (display-sort-function . #f(compiled-function (completions) #<bytecode -0x67c261ef9006e89>))))
  completion-try-completion("len" #f(compiled-function (pattern pred action) #<bytecode 0x19ec37e26e8f5965>) nil 3 (metadata (category . eglot-capf) (display-sort-function . #f(compiled-function (completions) #<bytecode -0x67c261ef9006e89>))))
  ivy-completion-in-region(#<marker at 35 in test_rust.rs> 38 #f(compiled-function (pattern pred action) #<bytecode 0x19ec37e26e8f5965>) nil)
  completion-in-region(#<marker at 35 in test_rust.rs> 38 #f(compiled-function (pattern pred action) #<bytecode 0x19ec37e26e8f5965>) nil)
  completion-at-point()
  complete-symbol(nil)
  funcall-interactively(complete-symbol nil)
  call-interactively(complete-symbol nil nil)
  command-execute(complete-symbol)

bug#68699: Eglot completion has stopped working in some cases

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 6991 bytes --]

Ok, but please explain exactly what "attempting completion" amounts to.
Read the manual about how Eglot bugs should be reported, Emacs -Q etc.
Thanks

On Wed, Jan 24, 2024, 20:37 Brennan Vincent <brennan@umanwizard.com> wrote:

> I bisected the issue to e93d99a4a0ce578249304dce350465c580a49892, so
> including João.
>
> In some cases when attempting to complete using eglot, I am getting
> errors rather than a list of completions. I haven't been able to debug
> why but it seems to be happening in "pcm" completion style.
>
> For example, in a Rust crate with the following main file:
>
> fn foo_len() {
> }
>
> fn main() {
>     len
> }
>
> positioning my cursor after "len" and attempting completion yields the
> following error message:
>
> completion-pcm--merge-completions: Internal error: _mm_blend_epi16(…)
> (use std::arch::x86_64::_mm_blend_epi16) doesn’t match \`len
>
> Full backtrace:
>
> Debugger entered--Lisp error: (error #("Internal error: _mm_blend_epi16(…)
> (use std::arch::x86_64::_mm_blend_epi16) doesn’t match \\`len" 16 17
> (eglot--lsp-item (:label #("_mm_blend_epi16(…) (use
> std::arch::x86_64::_mm_blend_epi16)" 0 1 (eglot--lsp-item #5) 5 8 (face
> completions-common-part)) :kind 3 :detail "unsafe fn(__m128i, __m128i) ->
> __m128i" :documentation (:kind "markdown" :value "Blend packed 16-bit
> integers from `a` and `b` using the mask `IMM8`.\n\nThe mask bits determine
> the selection. A clear bit selects the\ncorresponding element of `a`, and a
> set bit the corresponding\nelement of `b`.\n\n[Intel's documentation](
> https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm_blend_epi16)")
> :deprecated :json-false :sortText "fffffff0" :filterText
> #("_mm_blend_epi16" 5 8 (face completions-common-part)) :insertTextFormat 2
> :textEdit (:range (:start (:line 4 :character 4) :end (:line 4 :character
> 7)) :newText "_mm_blend_epi16(${1:a}, ${2:b})$0") :additionalTextEdits []
> :command (:title "triggerParameterHints" :command
> "rust-analyzer.triggerParameterHints") :data (:position (:textDocument
> (:uri "file:///home/brennan/test_rust/src/bin/test_rust.rs") :position
> (:line 4 :character 7)) :imports [(:full_import_path
> "std::arch::x86_64::_mm_blend_epi16" :imported_name "_mm_blend_epi16")])))
> 21 24 (face completions-common-part)))
>   signal(error (#("Internal error: _mm_blend_epi16(…) (use
> std::arch::x86_64::_mm_blend_epi16) doesn’t match \\`len" 16 17
> (eglot--lsp-item (:label #("_mm_blend_epi16(…) (use
> std::arch::x86_64::_mm_blend_epi16)" 0 1 (eglot--lsp-item #6) 5 8 (face
> completions-common-part)) :kind 3 :detail "unsafe fn(__m128i, __m128i) ->
> __m128i" :documentation (:kind "markdown" :value "Blend packed 16-bit
> integers from `a` and `b` using the mask `IMM8`.\n\nThe mask bits determine
> the selection. A clear bit selects the\ncorresponding element of `a`, and a
> set bit the corresponding\nelement of `b`.\n\n[Intel's documentation](
> https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm_blend_epi16)")
> :deprecated :json-false :sortText "fffffff0" :filterText
> #("_mm_blend_epi16" 5 8 (face completions-common-part)) :insertTextFormat 2
> :textEdit (:range (:start (:line 4 :character 4) :end (:line 4 :character
> 7)) :newText "_mm_blend_epi16(${1:a}, ${2:b})$0") :additionalTextEdits []
> :command (:title "triggerParameterHints" :command
> "rust-analyzer.triggerParameterHints") :data (:position (:textDocument
> (:uri "file:///home/brennan/test_rust/src/bin/test_rust.rs") :position
> (:line 4 :character 7)) :imports [(:full_import_path
> "std::arch::x86_64::_mm_blend_epi16" :imported_name "_mm_blend_epi16")])))
> 21 24 (face completions-common-part))))
>   error("Internal error: %s doesn't match %s" #("_mm_blend_epi16(…) (use
> std::arch::x86_64::_mm_blend_epi16)" 0 1 (eglot--lsp-item (:label #2 :kind
> 3 :detail "unsafe fn(__m128i, __m128i) -> __m128i" :documentation (:kind
> "markdown" :value "Blend packed 16-bit integers from `a` and `b` using the
> mask `IMM8`.\n\nThe mask bits determine the selection. A clear bit selects
> the\ncorresponding element of `a`, and a set bit the corresponding\nelement
> of `b`.\n\n[Intel's documentation](
> https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm_blend_epi16)")
> :deprecated :json-false :sortText "fffffff0" :filterText
> #("_mm_blend_epi16" 5 8 (face completions-common-part)) :insertTextFormat 2
> :textEdit (:range (:start (:line 4 :character 4) :end (:line 4 :character
> 7)) :newText "_mm_blend_epi16(${1:a}, ${2:b})$0") :additionalTextEdits []
> :command (:title "triggerParameterHints" :command
> "rust-analyzer.triggerParameterHints") :data (:position (:textDocument
> (:uri "file:///home/brennan/test_rust/src/bin/test_rust.rs") :position
> (:line 4 :character 7)) :imports [(:full_import_path
> "std::arch::x86_64::_mm_blend_epi16" :imported_name "_mm_blend_epi16")])))
> 5 8 (face completions-common-part)) "\\`len")
>   completion-pcm--merge-completions((... ... ... ... ... ... ... ... ...
> ... ... ... ... ... ...) ("len"))
>   completion-pcm--merge-try(("len") (... ... ... ... ... ... ... ... ...
> ... ... ... ... ... ...) "" "")
>   completion-pcm-try-completion("len" #f(compiled-function (pattern pred
> action) #<bytecode 0x19ec37e26e8f5965>) nil 3)
>   #f(compiled-function (style) #<bytecode
> -0x32c2df51d2b464c>)(partial-completion)
>   #f(compiled-function (elt) #<bytecode
> 0x19004498146b82cc>)(partial-completion)
>   mapc(#f(compiled-function (elt) #<bytecode 0x19004498146b82cc>)
> (eglot--dumb-flex basic partial-completion emacs22))
>   seq-do(#f(compiled-function (elt) #<bytecode 0x19004498146b82cc>)
> (eglot--dumb-flex basic partial-completion emacs22))
>   seq-some(#f(compiled-function (style) #<bytecode -0x32c2df51d2b464c>)
> (eglot--dumb-flex basic partial-completion emacs22))
>   completion--nth-completion(1 "len" #f(compiled-function (pattern pred
> action) #<bytecode 0x19ec37e26e8f5965>) nil 3 (metadata (category .
> eglot-capf) (display-sort-function . #f(compiled-function (completions)
> #<bytecode -0x67c261ef9006e89>))))
>   completion-try-completion("len" #f(compiled-function (pattern pred
> action) #<bytecode 0x19ec37e26e8f5965>) nil 3 (metadata (category .
> eglot-capf) (display-sort-function . #f(compiled-function (completions)
> #<bytecode -0x67c261ef9006e89>))))
>   ivy-completion-in-region(#<marker at 35 in test_rust.rs> 38
> #f(compiled-function (pattern pred action) #<bytecode 0x19ec37e26e8f5965>)
> nil)
>   completion-in-region(#<marker at 35 in test_rust.rs> 38
> #f(compiled-function (pattern pred action) #<bytecode 0x19ec37e26e8f5965>)
> nil)
>   completion-at-point()
>   complete-symbol(nil)
>   funcall-interactively(complete-symbol nil)
>   call-interactively(complete-symbol nil nil)
>   command-execute(complete-symbol)
>
>

[-- Attachment #2: Type: text/html, Size: 8605 bytes --]

bug#68699: Eglot completion has stopped working in some cases

[João Távora]

On Wed, Jan 24, 2024 at 8:37 PM Brennan Vincent <brennan@umanwizard.com> wrote:

> For example, in a Rust crate with the following main file:
>
> fn foo_len() {
> }
>
> fn main() {
>     len
> }

I've reproduced this with latest master:

~/Source/Emacs/emacs/src/emacs -Q ~/MoreSource/Rust/hello/src/main.rs
-f rust-ts-mode -f eglot

where main.rs is the provided snippet.

Pressing C-M-i with point after 'len' yields the error.

The error only happens in partial-completion is in the completion-styles
variable, so a workaround is to remove partial-completion from
completion-styles, at least in Eglot enabled buffers.

I will look now for the underlying cause.

bug#68699: Eglot completion has stopped working in some cases

[João Távora]

On Thu, Jan 25, 2024 at 12:43 AM João Távora <joaotavora@gmail.com> wrote:
>
> On Wed, Jan 24, 2024 at 8:37 PM Brennan Vincent <brennan@umanwizard.com> wrote:
>
> > For example, in a Rust crate with the following main file:
> >
> > fn foo_len() {
> > }
> >
> > fn main() {
> >     len
> > }
>
> I've reproduced this with latest master:
>
> ~/Source/Emacs/emacs/src/emacs -Q ~/MoreSource/Rust/hello/src/main.rs
> -f rust-ts-mode -f eglot
>
> where main.rs is the provided snippet.
>
> Pressing C-M-i with point after 'len' yields the error.
>
> The error only happens in partial-completion is in the completion-styles
> variable, so a workaround is to remove partial-completion from
> completion-styles, at least in Eglot enabled buffers.
>
> I will look now for the underlying cause.

Stefan, this is down to partial-completion somehow thinking
that "\\'len" should match at least some LSP-provided
completion candidate, otherwise it's an "internal error"

I wish I could turn off all the styles from the table but I can
only add to them.  I simply cannot write a reasonable
try-completion thing for LSP completions.  LSP completions
are really designed to be navigated using a company-like
popup.  Without it they should almost always use the *Completions*
buffer, and rarely ever Emacs partial completion shenanigans.

Nevertheless, I need to plug this bug somehow in Eglot.

Brennan, can you try this patch I've just pushed?

diff --git a/lisp/progmodes/eglot.el b/lisp/progmodes/eglot.el
index 511000927cf..c5fbf5eb9d5 100644
--- a/lisp/progmodes/eglot.el
+++ b/lisp/progmodes/eglot.el
@@ -3060,9 +3060,13 @@ eglot--dumb-flex
            finally (cl-return comp)))

 (defun eglot--dumb-allc (pat table pred _point) (funcall table pat pred t))
+(defun eglot--dumb-tryc (pat table pred point)
+  (if-let ((probe (funcall table pat pred nil)))
+      (cons probe (length probe))
+    (cons pat point)))

 (add-to-list 'completion-category-defaults '(eglot-capf (styles
eglot--dumb-flex)))
-(add-to-list 'completion-styles-alist '(eglot--dumb-flex ignore
eglot--dumb-allc))
+(add-to-list 'completion-styles-alist '(eglot--dumb-flex
eglot--dumb-tryc eglot--dumb-allc))

 (defun eglot-completion-at-point ()
   "Eglot's `completion-at-point' function."

João

bug#68699: Eglot completion has stopped working in some cases

[Brennan Vincent]

The issue is fixed for me on latest master, which includes the patch you
mentioned.

João Távora <joaotavora@gmail.com> writes:

> On Thu, Jan 25, 2024 at 12:43 AM João Távora <joaotavora@gmail.com> wrote:
>>
>> On Wed, Jan 24, 2024 at 8:37 PM Brennan Vincent <brennan@umanwizard.com> wrote:
>>
>> > For example, in a Rust crate with the following main file:
>> >
>> > fn foo_len() {
>> > }
>> >
>> > fn main() {
>> >     len
>> > }
>>
>> I've reproduced this with latest master:
>>
>> ~/Source/Emacs/emacs/src/emacs -Q ~/MoreSource/Rust/hello/src/main.rs
>> -f rust-ts-mode -f eglot
>>
>> where main.rs is the provided snippet.
>>
>> Pressing C-M-i with point after 'len' yields the error.
>>
>> The error only happens in partial-completion is in the completion-styles
>> variable, so a workaround is to remove partial-completion from
>> completion-styles, at least in Eglot enabled buffers.
>>
>> I will look now for the underlying cause.
>
> Stefan, this is down to partial-completion somehow thinking
> that "\\'len" should match at least some LSP-provided
> completion candidate, otherwise it's an "internal error"
>
> I wish I could turn off all the styles from the table but I can
> only add to them.  I simply cannot write a reasonable
> try-completion thing for LSP completions.  LSP completions
> are really designed to be navigated using a company-like
> popup.  Without it they should almost always use the *Completions*
> buffer, and rarely ever Emacs partial completion shenanigans.
>
> Nevertheless, I need to plug this bug somehow in Eglot.
>
> Brennan, can you try this patch I've just pushed?
>
> diff --git a/lisp/progmodes/eglot.el b/lisp/progmodes/eglot.el
> index 511000927cf..c5fbf5eb9d5 100644
> --- a/lisp/progmodes/eglot.el
> +++ b/lisp/progmodes/eglot.el
> @@ -3060,9 +3060,13 @@ eglot--dumb-flex
>             finally (cl-return comp)))
>
>  (defun eglot--dumb-allc (pat table pred _point) (funcall table pat pred t))
> +(defun eglot--dumb-tryc (pat table pred point)
> +  (if-let ((probe (funcall table pat pred nil)))
> +      (cons probe (length probe))
> +    (cons pat point)))
>
>  (add-to-list 'completion-category-defaults '(eglot-capf (styles
> eglot--dumb-flex)))
> -(add-to-list 'completion-styles-alist '(eglot--dumb-flex ignore
> eglot--dumb-allc))
> +(add-to-list 'completion-styles-alist '(eglot--dumb-flex
> eglot--dumb-tryc eglot--dumb-allc))
>
>  (defun eglot-completion-at-point ()
>    "Eglot's `completion-at-point' function."
>
> João

bug#68699: Eglot completion has stopped working in some cases

[João Távora]

Great, thanks for testing.  Closing.  I've release Eglot 1.17
with this fix and another completion-related fix.

João