TAGS completion with (setq-local completion-ignore-case t)

TAGS completion with (setq-local completion-ignore-case t)

[Morgan Willcock]

I was trying to track down an issue with inconsistencies in completion
matching between a custom capf function and the default TAGS capf
function.

It seems that if a buffer is configured for case-insensitive completion
matching, by setting the buffer-local value of completion-ignore-case to
t, this does not have any effect on TAGS completion.

In order to get case-insensitive completion matching for TAGS, I had to
also set the buffer local value of completion-ignore-case to t in the
TAGS buffer.

Is there a technical reason why TAGS completion cannot use the value of
completion-ignore-case that is set in the buffer where completion is
initiated, or is this a bug in the implementation?

-- 
Morgan Willcock

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Date: Thu, 07 Mar 2024 09:14:04 +0000
> 
> I was trying to track down an issue with inconsistencies in completion
> matching between a custom capf function and the default TAGS capf
> function.
> 
> It seems that if a buffer is configured for case-insensitive completion
> matching, by setting the buffer-local value of completion-ignore-case to
> t, this does not have any effect on TAGS completion.
> 
> In order to get case-insensitive completion matching for TAGS, I had to
> also set the buffer local value of completion-ignore-case to t in the
> TAGS buffer.
> 
> Is there a technical reason why TAGS completion cannot use the value of
> completion-ignore-case that is set in the buffer where completion is
> initiated, or is this a bug in the implementation?

If you mean completion when you type "M-.", then this has its own user
option, tags-case-fold-search.  Is that what you want?

Re: TAGS completion with (setq-local completion-ignore-case t)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Morgan Willcock <morgan@ice9.digital>
>> Date: Thu, 07 Mar 2024 09:14:04 +0000
>> 
>> I was trying to track down an issue with inconsistencies in completion
>> matching between a custom capf function and the default TAGS capf
>> function.
>> 
>> It seems that if a buffer is configured for case-insensitive completion
>> matching, by setting the buffer-local value of completion-ignore-case to
>> t, this does not have any effect on TAGS completion.
>> 
>> In order to get case-insensitive completion matching for TAGS, I had to
>> also set the buffer local value of completion-ignore-case to t in the
>> TAGS buffer.
>> 
>> Is there a technical reason why TAGS completion cannot use the value of
>> completion-ignore-case that is set in the buffer where completion is
>> initiated, or is this a bug in the implementation?
>
> If you mean completion when you type "M-.", then this has its own user
> option, tags-case-fold-search.  Is that what you want?

I am talking about completion as initiated by the complete-tag command.

(Setting tags-case-fold-search to t does not change the result, whether
set buffer-local or globally).

-- 
Morgan Willcock

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Cc: emacs-devel@gnu.org
> Date: Thu, 07 Mar 2024 09:54:31 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Is there a technical reason why TAGS completion cannot use the value of
> >> completion-ignore-case that is set in the buffer where completion is
> >> initiated, or is this a bug in the implementation?
> >
> > If you mean completion when you type "M-.", then this has its own user
> > option, tags-case-fold-search.  Is that what you want?
> 
> I am talking about completion as initiated by the complete-tag command.
> 
> (Setting tags-case-fold-search to t does not change the result, whether
> set buffer-local or globally).

Can you show a complete recipe, starting from "emacs -Q", to reproduce
the issue?  And what version of Emacs is that?

Re: TAGS completion with (setq-local completion-ignore-case t)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Morgan Willcock <morgan@ice9.digital>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 07 Mar 2024 09:54:31 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Is there a technical reason why TAGS completion cannot use the value of
>> >> completion-ignore-case that is set in the buffer where completion is
>> >> initiated, or is this a bug in the implementation?
>> >
>> > If you mean completion when you type "M-.", then this has its own user
>> > option, tags-case-fold-search.  Is that what you want?
>> 
>> I am talking about completion as initiated by the complete-tag command.
>> 
>> (Setting tags-case-fold-search to t does not change the result, whether
>> set buffer-local or globally).
>
> Can you show a complete recipe, starting from "emacs -Q", to reproduce
> the issue?

I've included a script below which should demonstrate the problem,
i.e. no completions are generated when the buffer-local value of
completion-ignore-case is t until the value of completion-ignore-case is
also set to t for the TAGS buffer.

> And what version of Emacs is that?

I've tested on 29.2 and the master branch (as of a few hours ago).

  ## Generate a TAGS file.
  printf "Function IsSingle\nFunction IsString\n" > functions
  etags --lang=none --regex='/^function[ \t]+\([0-9A-Za-z:_]+\)/\1/i' functions


  ## Test default completion (case-sensitive).
  emacs -Q --eval "(progn \
                    (pop-to-buffer (get-buffer-create \"test\")) \
                    (visit-tags-table \"TAGS\") \
                    (insert \"Is\") \
                    (complete-tag) \
                    (complete-tag))"

  # 2 possible completions:
  # IsSingle
  # IsString


  ## Test case-insensitive completion (global).
  emacs -Q --eval "(progn \
                    (pop-to-buffer (get-buffer-create \"test\")) \
                    (setq completion-ignore-case t) \
                    (visit-tags-table \"TAGS\") \
                    (insert \"is\") \
                    (complete-tag) \
                    (complete-tag))"

  # 2 possible completions:
  # IsSingle
  # IsString


  ## Test case-insensitive completion (buffer-local).
  emacs -Q --eval "(progn \
                    (pop-to-buffer (get-buffer-create \"test\")) \
                    (setq-local completion-ignore-case t) \
                    (visit-tags-table \"TAGS\") \
                    (insert \"is\") \
                    (complete-tag) \
                    (complete-tag))"

  # No completions.


  ## Test case-insensitive completion (buffer-local, including TAGS buffer).
  emacs -Q --eval "(progn \
                    (pop-to-buffer (get-buffer-create \"test\")) \
                    (setq-local completion-ignore-case t) \
                    (visit-tags-table \"TAGS\") \
                    (with-current-buffer \"TAGS\" \
                      (setq-local completion-ignore-case t)) \
                    (insert \"is\") \
                    (complete-tag) \
                    (complete-tag))"

  # 2 possible completions:
  # IsSingle
  # IsString

-- 
Morgan Willcock

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Cc: emacs-devel@gnu.org
> Date: Thu, 07 Mar 2024 11:37:27 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Morgan Willcock <morgan@ice9.digital>
> >> Cc: emacs-devel@gnu.org
> >> Date: Thu, 07 Mar 2024 09:54:31 +0000
> >> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> >> Is there a technical reason why TAGS completion cannot use the value of
> >> >> completion-ignore-case that is set in the buffer where completion is
> >> >> initiated, or is this a bug in the implementation?
> >> >
> >> > If you mean completion when you type "M-.", then this has its own user
> >> > option, tags-case-fold-search.  Is that what you want?
> >> 
> >> I am talking about completion as initiated by the complete-tag command.
> >> 
> >> (Setting tags-case-fold-search to t does not change the result, whether
> >> set buffer-local or globally).
> >
> > Can you show a complete recipe, starting from "emacs -Q", to reproduce
> > the issue?
> 
> I've included a script below which should demonstrate the problem,
> i.e. no completions are generated when the buffer-local value of
> completion-ignore-case is t until the value of completion-ignore-case is
> also set to t for the TAGS buffer.

Does the patch below give good results?

diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
index 476037e..5976121 100644
--- a/lisp/progmodes/etags.el
+++ b/lisp/progmodes/etags.el
@@ -2065,7 +2065,8 @@ complete-tag
       (user-error "%s"
                   (substitute-command-keys
                    "No tags table loaded; try \\[visit-tags-table]")))
-  (let ((comp-data (tags-completion-at-point-function)))
+  (let ((comp-data (tags-completion-at-point-function))
+        (completion-ignore-case (find-tag--completion-ignore-case)))
     (if (null comp-data)
 	(user-error "Nothing to complete")
       (completion-in-region (car comp-data) (cadr comp-data)

Re: TAGS completion with (setq-local completion-ignore-case t)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

> Does the patch below give good results?
>
> diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
> index 476037e..5976121 100644
> --- a/lisp/progmodes/etags.el
> +++ b/lisp/progmodes/etags.el
> @@ -2065,7 +2065,8 @@ complete-tag
>        (user-error "%s"
>                    (substitute-command-keys
>                     "No tags table loaded; try \\[visit-tags-table]")))
> -  (let ((comp-data (tags-completion-at-point-function)))
> +  (let ((comp-data (tags-completion-at-point-function))
> +        (completion-ignore-case (find-tag--completion-ignore-case)))
>      (if (null comp-data)
>  	(user-error "Nothing to complete")
>        (completion-in-region (car comp-data) (cadr comp-data)

I rebuilt from the latest commit on the master branch to incorporate the
patch.  Unfortunately, I do not see any change in the result of the
failing test case; the completion still appears to be case-sensitive.

  printf "Function IsSingle\nFunction IsString\n" > functions
  etags --lang=none --regex='/^function[ \t]+\([0-9A-Za-z:_]+\)/\1/i' functions
  emacs -Q --eval "(progn \
                     (pop-to-buffer (get-buffer-create \"test\")) \
                     (setq-local completion-ignore-case t) \
                     (visit-tags-table \"TAGS\") \
                     (insert \"is\") \
                     (complete-tag) \
                     (complete-tag))"

  # No completions.

-- 
Morgan Willcock

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Cc: emacs-devel@gnu.org
> Date: Sat, 09 Mar 2024 13:23:08 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Does the patch below give good results?
> >
> > diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
> > index 476037e..5976121 100644
> > --- a/lisp/progmodes/etags.el
> > +++ b/lisp/progmodes/etags.el
> > @@ -2065,7 +2065,8 @@ complete-tag
> >        (user-error "%s"
> >                    (substitute-command-keys
> >                     "No tags table loaded; try \\[visit-tags-table]")))
> > -  (let ((comp-data (tags-completion-at-point-function)))
> > +  (let ((comp-data (tags-completion-at-point-function))
> > +        (completion-ignore-case (find-tag--completion-ignore-case)))
> >      (if (null comp-data)
> >  	(user-error "Nothing to complete")
> >        (completion-in-region (car comp-data) (cadr comp-data)
> 
> I rebuilt from the latest commit on the master branch to incorporate the
> patch.  Unfortunately, I do not see any change in the result of the
> failing test case; the completion still appears to be case-sensitive.
> 
>   printf "Function IsSingle\nFunction IsString\n" > functions
>   etags --lang=none --regex='/^function[ \t]+\([0-9A-Za-z:_]+\)/\1/i' functions
>   emacs -Q --eval "(progn \
>                      (pop-to-buffer (get-buffer-create \"test\")) \
>                      (setq-local completion-ignore-case t) \
>                      (visit-tags-table \"TAGS\") \
>                      (insert \"is\") \
>                      (complete-tag) \
>                      (complete-tag))"
> 
>   # No completions.

You need to set tags-case-fold-search, not the buffer-local value of
completion-ignore-case.  That's because the commands in etags.el look
at tags-case-fold-search to decide whether they should ignore
letter-case.

Re: TAGS completion with (setq-local completion-ignore-case t)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

>> I rebuilt from the latest commit on the master branch to incorporate the
>> patch.  Unfortunately, I do not see any change in the result of the
>> failing test case; the completion still appears to be case-sensitive.
>> 
>>   printf "Function IsSingle\nFunction IsString\n" > functions
>>   etags --lang=none --regex='/^function[ \t]+\([0-9A-Za-z:_]+\)/\1/i' functions
>>   emacs -Q --eval "(progn \
>>                      (pop-to-buffer (get-buffer-create \"test\")) \
>>                      (setq-local completion-ignore-case t) \
>>                      (visit-tags-table \"TAGS\") \
>>                      (insert \"is\") \
>>                      (complete-tag) \
>>                      (complete-tag))"
>> 
>>   # No completions.
>
> You need to set tags-case-fold-search, not the buffer-local value of
> completion-ignore-case.  That's because the commands in etags.el look
> at tags-case-fold-search to decide whether they should ignore
> letter-case.

Setting tags-case-fold-search to t (buffer-local or not) doesn't seem to
have any effect when completion-ignore-case is also set to t.

  emacs -Q --eval "(progn \
                     (pop-to-buffer (get-buffer-create \"test\")) \
                     (setq-local completion-ignore-case t) \
                     (setq-local tags-case-fold-search t) \
                     (visit-tags-table \"TAGS\") \
                     (insert \"is\") \
                     (complete-tag) \
                     (complete-tag))"

  # No completions.

  emacs -Q --eval "(progn \
                     (pop-to-buffer (get-buffer-create \"test\")) \
                     (setq-local completion-ignore-case t) \
                     (setq tags-case-fold-search t) \
                     (visit-tags-table \"TAGS\") \
                     (insert \"is\") \
                     (complete-tag) \
                     (complete-tag))"

  # No completions.

I would still need to set completion-ignore-case to t because that is
still relevant for other capf backends for the buffer (but not for all
buffers).

It also pretty strange that setting completion-ignore-case to t has done
the opposite of what it advertises.

-- 
Morgan Willcock

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Cc: emacs-devel@gnu.org
> Date: Sat, 09 Mar 2024 14:06:39 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > You need to set tags-case-fold-search, not the buffer-local value of
> > completion-ignore-case.  That's because the commands in etags.el look
> > at tags-case-fold-search to decide whether they should ignore
> > letter-case.
> 
> Setting tags-case-fold-search to t (buffer-local or not) doesn't seem to
> have any effect when completion-ignore-case is also set to t.
> 
>   emacs -Q --eval "(progn \
>                      (pop-to-buffer (get-buffer-create \"test\")) \
>                      (setq-local completion-ignore-case t) \
>                      (setq-local tags-case-fold-search t) \

Don't use setq-local, use setq.  Or let-bind it, like this:

  (progn
    (pop-to-buffer (get-buffer-create "test"))
    (let ((tags-case-fold-search t))
      (insert "is")
      (complete-tag)
      (complete-tag)))

> I would still need to set completion-ignore-case to t because that is
> still relevant for other capf backends for the buffer (but not for all
> buffers).

That's fine, and should not interfere.

> It also pretty strange that setting completion-ignore-case to t has done
> the opposite of what it advertises.

It just is not heeded to.  Like with
read-file-name-completion-ignore-case in case of reading file names,
and a few other similar variables.

Re: TAGS completion with (setq-local completion-ignore-case t)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Morgan Willcock <morgan@ice9.digital>
>> Cc: emacs-devel@gnu.org
>> Date: Sat, 09 Mar 2024 14:06:39 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > You need to set tags-case-fold-search, not the buffer-local value of
>> > completion-ignore-case.  That's because the commands in etags.el look
>> > at tags-case-fold-search to decide whether they should ignore
>> > letter-case.
>> 
>> Setting tags-case-fold-search to t (buffer-local or not) doesn't seem to
>> have any effect when completion-ignore-case is also set to t.
>> 
>>   emacs -Q --eval "(progn \
>>                      (pop-to-buffer (get-buffer-create \"test\")) \
>>                      (setq-local completion-ignore-case t) \
>>                      (setq-local tags-case-fold-search t) \
>
> Don't use setq-local, use setq.  Or let-bind it, like this:
>
>   (progn
>     (pop-to-buffer (get-buffer-create "test"))
>     (let ((tags-case-fold-search t))
>       (insert "is")
>       (complete-tag)
>       (complete-tag)))

The code example is just to recreate the problem.  This affects the
completion mechanism in the buffer that the user has direct access to
with key-bindings.

i.e. If the user types "is" and then presses C-M-i there are no
completions available.

-- 
Morgan Willcock

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> Date: Sat, 09 Mar 2024 16:26:35 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > Setting tags-case-fold-search to t (buffer-local or not) doesn't seem to
> > have any effect when completion-ignore-case is also set to t.
> > 
> >   emacs -Q --eval "(progn \
> >                      (pop-to-buffer (get-buffer-create \"test\")) \
> >                      (setq-local completion-ignore-case t) \
> >                      (setq-local tags-case-fold-search t) \
> 
> Don't use setq-local, use setq.  Or let-bind it, like this:
> 
>   (progn
>     (pop-to-buffer (get-buffer-create "test"))
>     (let ((tags-case-fold-search t))
>       (insert "is")
>       (complete-tag)
>       (complete-tag)))
> 
> > I would still need to set completion-ignore-case to t because that is
> > still relevant for other capf backends for the buffer (but not for all
> > buffers).
> 
> That's fine, and should not interfere.

Actually, I take that back: you are well advised NOT to give
completion-ignore-case buffer-local bindings.  That's because many
Emacs commands let-bind that variable to produce the desired behavior,
and let-binding of a buffer-local variable can have strange effects
when the code inside the let-binding switches to a different buffer
(as etags commands do, because they switch to the TAGS buffer).

Why exactly do you need to have a buffer-local value of
completion-ignore-case?  This is not a user option, and is meant to be
either set globally or let-bound for specific code.

> > It also pretty strange that setting completion-ignore-case to t has done
> > the opposite of what it advertises.

That's the effect of what I describe above, I think.

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Cc: emacs-devel@gnu.org
> Date: Sat, 09 Mar 2024 14:33:28 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Don't use setq-local, use setq.  Or let-bind it, like this:
> >
> >   (progn
> >     (pop-to-buffer (get-buffer-create "test"))
> >     (let ((tags-case-fold-search t))
> >       (insert "is")
> >       (complete-tag)
> >       (complete-tag)))
> 
> The code example is just to recreate the problem.  This affects the
> completion mechanism in the buffer that the user has direct access to
> with key-bindings.
> 
> i.e. If the user types "is" and then presses C-M-i there are no
> completions available.

C-M-i is not bound to complete-tag, it is bound to complete-symbol, at
least by default.  If I customize tags-case-fold-search to t, and then
invoke complete-tag, I get case-insensitive completion based on TAGS.

Re: TAGS completion with (setq-local completion-ignore-case t)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

> Why exactly do you need to have a buffer-local value of
> completion-ignore-case?  This is not a user option, and is meant to be
> either set globally or let-bound for specific code.

I was using it as a configuration example for a major-mode setup hook,
where the language itself is not case-sensitive, but completions through
capf functions (including TAG completion) can be displayed and inserted
in mixed case.

Is there a supported way to have user configurable case-insensitive
completion that is configured per major mode?

-- 
Morgan Willcock

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Cc: emacs-devel@gnu.org
> Date: Sat, 09 Mar 2024 14:57:47 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Why exactly do you need to have a buffer-local value of
> > completion-ignore-case?  This is not a user option, and is meant to be
> > either set globally or let-bound for specific code.
> 
> I was using it as a configuration example for a major-mode setup hook,
> where the language itself is not case-sensitive, but completions through
> capf functions (including TAG completion) can be displayed and inserted
> in mixed case.
> 
> Is there a supported way to have user configurable case-insensitive
> completion that is configured per major mode?

use the user options specific for each type of completion, I'd say.
For TAGS that's tags-case-fold-search; for file names that's
read-file-name-completion-ignore-case; for buffers its
read-buffer-completion-ignore-case.  And there may be others.

Stefan, any better ideas?

Re: TAGS completion with (setq-local completion-ignore-case t)

[Eli Zaretskii]

> Date: Sat, 09 Mar 2024 17:49:25 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > Is there a supported way to have user configurable case-insensitive
> > completion that is configured per major mode?
> 
> use the user options specific for each type of completion, I'd say.
> For TAGS that's tags-case-fold-search; for file names that's
> read-file-name-completion-ignore-case; for buffers its
> read-buffer-completion-ignore-case.  And there may be others.

Btw, I'd expect major-mode specific completion-at-point-functions to
DTRT wrt case-sensitivity by default, as appropriate for each major
mode.  tags-completion-at-point-function is mode-agnostic, which is
why it has its own knob to control that.

Re: TAGS completion with (setq-local completion-ignore-case t)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Why exactly do you need to have a buffer-local value of
>> > completion-ignore-case?  This is not a user option, and is meant to be
>> > either set globally or let-bound for specific code.
>>
>> I was using it as a configuration example for a major-mode setup hook,
>> where the language itself is not case-sensitive, but completions through
>> capf functions (including TAG completion) can be displayed and inserted
>> in mixed case.
>>
>> Is there a supported way to have user configurable case-insensitive
>> completion that is configured per major mode?
>
> use the user options specific for each type of completion, I'd say.
> For TAGS that's tags-case-fold-search; for file names that's
> read-file-name-completion-ignore-case; for buffers its
> read-buffer-completion-ignore-case.  And there may be others.
>
> Stefan, any better ideas?

I've tested wrapping the capf functions with Cape's cape-capf-case-fold
transformer.  This seems to give the ideal result for the actual
completions and is fine to be restricted to the major-mode (the wrapped
version is just another capf function which can be buffer-local).

https://elpa.gnu.org/packages/cape.html

I'll likely stick with this solution because it works so well, but thank
you for looking at the built-in options.

-- 
Morgan Willcock