bug#18643: 25.0.50; elisp--expect-function-p

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

emacs should only constrain itself to function completion when
absolutely sure.

It is not a big deal if function names creep into variable completion or
vice versa. Often it is handy to complete to an existing symbol and then
edit it into something else. For example, one might need to create a
variable name based on a function name.

There are a few places where I expect to have a completion based on my
past experience but fail now. For example, in (pred ...) pattern of
pcase. Other failures, (let (|)) and (let ((|))) where | is point.

I have experienced annoyances here and there and I think the fundamental
solution is not to second guess but complete liberally as we did before.

HTH,
Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

Hi Leo,

Leo Liu <sdl.web@gmail.com> writes:
> emacs should only constrain itself to function completion when
> absolutely sure.

How can we make sure? I think macro-expansion could help in some cases,
but implementing that approach is a bit beyond me.

> It is not a big deal if function names creep into variable completion or
> vice versa. Often it is handy to complete to an existing symbol and then
> edit it into something else.

IME, context-sensitive completion is useful, and people like it.

Narrowing down the list of possible completions is often quite useful,
especially when it can turn 2-3 completions into just one, which can be
inserted with one key. I think we can sacrifice completion in certain
rare cases to retain this advantage in general.

> For example, one might need to create a variable name based on a
> function name.

I'm pretty sure writing a new variable definition is a much less
frequent operation than referring to a variable in a function, or
writing a function call. And code completion is most useful when one is
referring to existing variables and functions.

> There are a few places where I expect to have a completion based on my
> past experience but fail now. For example, in (pred ...) pattern of
> pcase. Other failures, (let (|)) and (let ((|))) where | is point.

If you care to enumerate the main problem cases, and how they fail, it
shouldn't be too hard to add support for them. But of course, we'll
continue to have false negatives in some cases, like third-party macros.

> I have experienced annoyances here and there and I think the fundamental
> solution is not to second guess but complete liberally as we did before.

That suggestions sounds like giving up to me.

For what you're describing, you might like dabbrev-style completion, or
a simple function that looks up symbols in obarray. If you're not using
company-mode, maybe try porting company-dabbrev-code to
completion-at-point-functions, and use that.

I haven't spent a lot of time using the current trunk, but the recent
changes replicate the logic from company-elisp, and I quite liked it
(before it was deprecated in favor of company-capf). I haven't seen many
complaints about function-variable separation there.

That backend does have a `company-elisp-detect-function-context' user
option (so that's another solution for the issue discussed here), but it
doesn't seem to be used much (if at all), judging by the publicly
available configurations.

-- Dmitry

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2014-10-09 06:50 +0400, Dmitry Gutov wrote:
> IME, context-sensitive completion is useful, and people like it.

I agree.

> Narrowing down the list of possible completions is often quite useful,
> especially when it can turn 2-3 completions into just one, which can be
> inserted with one key. I think we can sacrifice completion in certain
> rare cases to retain this advantage in general.

It is the false negatives that kill its usefulness in the case of elisp.
For example, (ert-deftest |) and (defun |) no longer have the
completions I need.

>> For example, one might need to create a variable name based on a
>> function name.
>
> I'm pretty sure writing a new variable definition is a much less
> frequent operation than referring to a variable in a function, or
> writing a function call. And code completion is most useful when one is
> referring to existing variables and functions.

I won't jump to such conclusions so soon. You are guessing users'
editing habits too much.

> If you care to enumerate the main problem cases, and how they fail, it
> shouldn't be too hard to add support for them. But of course, we'll
> continue to have false negatives in some cases, like third-party macros.

What is hard is having a good algorithm to determine completion types
correctly. Unfortunately what we have has too many false negatives. I'd
much prefer offering multiple candidates instead of showing no
completions and with completion-cycling I can get to most completions in
under a second.

>> I have experienced annoyances here and there and I think the fundamental
>> solution is not to second guess but complete liberally as we did before.
>
> That suggestions sounds like giving up to me.
>
> For what you're describing, you might like dabbrev-style completion, or
> a simple function that looks up symbols in obarray. If you're not using
> company-mode, maybe try porting company-dabbrev-code to
> completion-at-point-functions, and use that.
>
> I haven't spent a lot of time using the current trunk, but the recent
> changes replicate the logic from company-elisp, and I quite liked it
> (before it was deprecated in favor of company-capf). I haven't seen many
> complaints about function-variable separation there.
>
> That backend does have a `company-elisp-detect-function-context' user
> option (so that's another solution for the issue discussed here), but it
> doesn't seem to be used much (if at all), judging by the publicly
> available configurations.

My bug report is not based on anything other than completion-at-point,
the plain/standard emacs completion UI, which is what I use for elisp
for a long while and I am quite content with it. It can be less optimal
for other languages. but for elisp it has worked well in the past.

HTH,
Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/09/2014 07:31 AM, Leo Liu wrote:
> It is the false negatives that kill its usefulness in the case of elisp.

I disagree with the word "kill" here. Somewhat diminish, maybe.

> For example, (ert-deftest |) and (defun |) no longer have the
> completions I need.

These would be trivial to improve.

>> I'm pretty sure writing a new variable definition is a much less
>> frequent operation than referring to a variable in a function, or
>> writing a function call. And code completion is most useful when one is
>> referring to existing variables and functions.
>
> I won't jump to such conclusions so soon. You are guessing users'
> editing habits too much.

In which part? The lower frequency of new definitions is self-evident, I 
think.

> What is hard is having a good algorithm to determine completion types
> correctly. Unfortunately what we have has too many false negatives. I'd
> much prefer offering multiple candidates instead of showing no
> completions and with completion-cycling I can get to most completions in
> under a second.

In that case, I guess you'll have to add a user option. 
`lisp-completion-at-point-uses-context', or something like that.

> My bug report is not based on anything other than completion-at-point,

Does that mean my suggestions can't be based on anything else?

> the plain/standard emacs completion UI, which is what I use for elisp
> for a long while and I am quite content with it. It can be less optimal
> for other languages. but for elisp it has worked well in the past.

I've been using it for a while as well, and I wouldn't say it was, or 
is, perfect.

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

> It is the false negatives that kill its usefulness in the case of elisp.
> For example, (ert-deftest |) and (defun |) no longer have the
> completions I need.

Dmitry, what is it exactly that changed?
I mean, Elisp completion has been context-sensitive "for ever", so the
change is not that it now is, but that it now reacts to the
context differently.

Oh, I think I see: the change is the introduction of
elisp--form-quoted-p such that if elisp--form-quoted-p returns nil
(and we're not expecting a function), we only complete against var names
instead of the previous behavior of expanding against "anything".

I think this is not right: we should have a elisp--expect-exp-p
which returns non-nil in a context where we know the symbol at point
should be a valid expression (i.e. a variable).  Not sure how to write
it, tho, but in its absence I think we're better off completing against
"anything" than restricting ourselves to variables.

BTW, to improve on these behaviors, one approach is to take the
macro-expansion code used in elisp--local-variables and use it
"everywhere".  I.e. expand elisp--local-variables so it doesn't just
return the list of local vars, but also returns whether we're inside
a quote, or we're expecting a function, or we're expecting an expression.


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2014-10-09 11:27 -0400, Stefan Monnier wrote:
> BTW, to improve on these behaviors, one approach is to take the
> macro-expansion code used in elisp--local-variables and use it
> "everywhere".  I.e. expand elisp--local-variables so it doesn't just
> return the list of local vars, but also returns whether we're inside
> a quote, or we're expecting a function, or we're expecting an expression.

Is it possible to use weight or priority i.e. the context decided
doesn't cut completion space to a subset but instead prioritise those
fit into the context by putting them at the front. Or find a way to
allow other completions to show up when the input matches none of the
subset.

Also could we not force users to insert ` before any completion in
strings or comments.

BTW, this is what happens when false negative hits:

1. If I remember the completion and it is short I type it output but
   often I double check with C-h f or C-h v to make sure it is correct.

2. When it is long or I don't remember I have to use C-h f or C-h v and
   copy it over from the *Help* buffer.

The cost is high and painful.

Thanks,
Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

>> BTW, to improve on these behaviors, one approach is to take the
>> macro-expansion code used in elisp--local-variables and use it
>> "everywhere".  I.e. expand elisp--local-variables so it doesn't just
>> return the list of local vars, but also returns whether we're inside
>> a quote, or we're expecting a function, or we're expecting an expression.
> Is it possible to use weight or priority i.e. the context decided
> doesn't cut completion space to a subset but instead prioritise those
> fit into the context by putting them at the front.  Or find a way to
> allow other completions to show up when the input matches none of the
> subset.

Yes, we can do that with completion-table-in-turn.
Note that the elisp--local-variables is pretty precise, so it might not
be necessary.

> Also could we not force users to insert ` before any completion in
> strings or comments.

We could provide a config option, but I'd rather we don't offer those
completions by default.


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2014-10-09 21:16 -0400, Stefan Monnier wrote:
> Yes, we can do that with completion-table-in-turn.
> Note that the elisp--local-variables is pretty precise, so it might not
> be necessary.

Indeed.

> We could provide a config option, but I'd rather we don't offer those
> completions by default.

For example, I need to add comment to a line with a bit of lisp form. I
already M-; and typed in a few words followed by `:':

        ;; Another way: (average (compute-weight ....) ...)

Now I won't be able to complete anything while finishing the form. I
have to uncomment first then finish the form and comment again.

Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

>> We could provide a config option, but I'd rather we don't offer those
>> completions by default.
> For example, I need to add comment to a line with a bit of lisp form. I
> already M-; and typed in a few words followed by `:':
>         ;; Another way: (average (compute-weight ....) ...)
> Now I won't be able to complete anything while finishing the form. I
> have to uncomment first then finish the form and comment again.

I know, but there's really no way for Emacs to know what you intend to
complete here.

So I think the right solution for this is to write
a completion-at-point-function which is designed specifically for
"completion in strings and comments" and which would probably rely on
something like dabbrev, complemented by a major-mode-supplied list of
additional candidates.

WDYT?


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

[-- Attachment #1: Type: text/html, Size: 681 bytes --]

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2014-10-11 00:53 +0400, Dmitry Gutov wrote:
> The "complemented" part there gives me pause: we just got out of the
> situation when a person writing prose in a docstring kept getting Lisp
> functions in completions.
>
> I rather think the right solution would start with a stricter
> convention for code snippets in docstrings.

This gives me a sense that you are thinking about a situation when
someone uses company and has the completions displayed immediately.

If true I am dubious with this change. Are we to switch to company by
default?

In standard default emacs behaviour, I had never getting anything
without me asking for it, be it in comments or strings or code. They
only appear when I ask.

Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

> If true I am dubious with this change. Are we to switch to company by
> default?

I'd like to include company in core Emacs, yes.  Enable it by default,
maybe not, or not yet.  But it's an important use case.


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2014-10-11 09:48 -0400, Stefan Monnier wrote:
> I'd like to include company in core Emacs, yes.  Enable it by default,
> maybe not, or not yet.  But it's an important use case.

That's my impression. It is perfectly fine to make things work better
for company. I am a bit worried seeing the default get worse.

Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

>> I'd like to include company in core Emacs, yes.  Enable it by default,
>> maybe not, or not yet.  But it's an important use case.
> That's my impression. It is perfectly fine to make things work better
> for company. I am a bit worried seeing the default get worse.

I agree, we should try to handle both cases better.  I.e. have a way to
distinguish "the user explicitly requested completion, so try to come up
with something" from the company case where we only want to auto pop up
a completion menu if we're sufficiently confident that the suggestions
are valuable.

Question is how to do it.  One way I see we could do it is by adding
a property to completion data (e.g. in the data returned by the
completion-at-point-function) which says something like "unsure" or
"low-quality" so we could ignore those when the user hasn't explicitly
requested completion.


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/14/2014 10:32 PM, Stefan Monnier wrote:

> I agree, we should try to handle both cases better.  I.e. have a way to
> distinguish "the user explicitly requested completion, so try to come up
> with something" from the company case where we only want to auto pop up
> a completion menu if we're sufficiently confident that the suggestions
> are valuable.

The idea makes a certain amount of sense, but I'm not sure if "try to 
come up with something" should be performed in contexts where other 
completion functions are likely to be more valuable.

> Question is how to do it.  One way I see we could do it is by adding
> a property to completion data (e.g. in the data returned by the
> completion-at-point-function) which says something like "unsure" or
> "low-quality" so we could ignore those when the user hasn't explicitly
> requested completion.

How will that interact with other elements in 
completion-at-point-functions? Suppose there's another function there, 
named ispell-complete-at-point. And suppose it always returns non-nil 
when in strings or comments, which looks like a reasonable behavior.

Which function will be used in comment if the user explicitly requested 
completion, and if lisp-completion-at-point returned `unsure'? Will that 
depend on the order of the elements in c-a-p-f?

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

> The idea makes a certain amount of sense, but I'm not sure if "try to come
> up with something" should be performed in contexts where other completion
> functions are likely to be more valuable.

Most likely the "unsure" tables would also be ":exclusive no".

> How will that interact with other elements in completion-at-point-functions?

We get to decide.

> Which function will be used in comment if the user explicitly requested
> completion, and if lisp-completion-at-point returned `unsure'? Will that
> depend on the order of the elements in c-a-p-f?

Yes, and it will depend on whether lisp-completion-at-point and
ispell-complete-at-point return ":exclusive no".  And maybe it will
depend on some new-feature-to-be-invented-to-make-it-work-better.


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

> 5:25 PM, 10 октября 2014 г., Stefan Monnier <monnier@iro.umontreal.ca>:
>     a completion-at-point-function which is designed specifically for
>     "completion in strings and comments" and which would probably rely on
>     something like dabbrev, complemented by a major-mode-supplied list of
>     additional candidates.

> The "complemented" part there gives me pause: we just got out of the
> situation when a person writing prose in a docstring kept getting Lisp
> functions in completions.

But by being a separate function, users can add/remove it at will,
depending on their preference.

> I rather think the right solution would start with a stricter convention for
> code snippets in docstrings. 

That's a non-starter, because while we can apply this to our own code,
we can't enforce it on others's,


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/11/2014 05:47 PM, Stefan Monnier wrote:

> But by being a separate function, users can add/remove it at will,
> depending on their preference.

Hmm, that's one option, yes.

>> I rather think the right solution would start with a stricter convention for
>> code snippets in docstrings.
>
> That's a non-starter, because while we can apply this to our own code,
> we can't enforce it on others's,

Then those others won't get Lisp completion is docstrings. Which is 
their choice, and not that big a deal.

The convention doesn't have to be complicated. For example, we can limit 
ourselves to considering blocks within comments or docstrings separated 
by newlines from the rest of the text, and only when each line starts 
with 2+ (or 4+?) spaces. As a plus, we'll be able to modify the 
indentation function to work in these blocks, too.

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

>>> I rather think the right solution would start with a stricter
>>> convention for code snippets in docstrings.
>> That's a non-starter, because while we can apply this to our own code,
>> we can't enforce it on others's,
> Then those others won't get Lisp completion is docstrings.  Which is their
> choice, and not that big a deal.

There's also the case where text happens to match your convention, so
the user will get completion when she doesn't want it.


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/14/2014 10:34 PM, Stefan Monnier wrote:
> There's also the case where text happens to match your convention, so
> the user will get completion when she doesn't want it.

Then the result still won't be worse than the Emacs 24.4 behavior.

Still, the text written in a way I described may be not Lisp code, but 
it's unlikely to be prose (unless it's a way of displaying a block 
quote?), so this kind of heuristic can be useful anyway.

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2014-10-16 06:35 +0400, Dmitry Gutov wrote:
> Still, the text written in a way I described may be not Lisp code, but
> it's unlikely to be prose (unless it's a way of displaying a block
> quote?), so this kind of heuristic can be useful anyway.

In the past 1-2 months I happen to write an elisp tool from scratch,
which has the following stats:

  component 1:
     file1: 2744        lines
     file2: 275         lines

  component 2:
     file1: 3194        lines

with component 2 v0.7.2 released on 08 Oct 2014. The packages are
developed using emacs trunk and happened to have some parts developed
before the change and others after the change. When I told you I tried
to live with it, I did try very hard.

Speaking from experience, the recent change to elisp-completion-at-point
has gotten in the way more than just once. It is painfully difficult to
see how this is an improvement to the default features.

(BTW, I hope to submit the packages for inclusion in emacs sometime
later)

Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/16/2014 07:36 AM, Leo Liu wrote:
> On 2014-10-16 06:35 +0400, Dmitry Gutov wrote:
>> Still, the text written in a way I described may be not Lisp code, but
>> it's unlikely to be prose (unless it's a way of displaying a block
>> quote?), so this kind of heuristic can be useful anyway.
 > ...
> Speaking from experience, the recent change to elisp-completion-at-point
> has gotten in the way more than just once. It is painfully difficult to
> see how this is an improvement to the default features.

Sorry, I don't know what you're trying to say. The heuristic I mention 
above is a proposal, not something already implemented, so it couldn't 
have helped or hurt you over these last 1-2 months.

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2014-10-10 09:25 -0400, Stefan Monnier wrote:
> I know, but there's really no way for Emacs to know what you intend to
> complete here.

I am fine with giving me anything from obarray. elisp names tend to be
very long anything that you can give me help me save time.

> So I think the right solution for this is to write
> a completion-at-point-function which is designed specifically for
> "completion in strings and comments" and which would probably rely on
> something like dabbrev, complemented by a major-mode-supplied list of
> additional candidates.
>
> WDYT?

Is this a solution for elisp or all modes? I am only concerned about
elisp mode though, which already has a good completion source ie
obarray.

For example, in common lisp I prefer the fuzzy completion (from slime)
which basically gives me anything sorted by score (with a user settable
cutoff). I haven't had any problem quickly getting the completion I have
in mind.

I don't understand why we work so hard to prevent completion in strings
and comments. Sorry if this has been discussed.

Thanks,
Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/10/2014 03:43 AM, Leo Liu wrote:

> Is it possible to use weight or priority i.e. the context decided
> doesn't cut completion space to a subset but instead prioritise those
> fit into the context by putting them at the front. Or find a way to
> allow other completions to show up when the input matches none of the
> subset.

It wouldn't be the worst approach, but we should be able to do better.

> Also could we not force users to insert ` before any completion in
> strings or comments.

Please don't ignore the available information about why things work as 
they do currently. Have you looked at the revision that introduced that 
change? Do you have anything to add to the discussion in the related bug?

> BTW, this is what happens when false negative hits:
>
> 1. If I remember the completion and it is short I type it output but
>     often I double check with C-h f or C-h v to make sure it is correct.
>
> 2. When it is long or I don't remember I have to use C-h f or C-h v and
>     copy it over from the *Help* buffer.
>
> The cost is high and painful.

Personally, I use `hippie-expand' as an escape hatch, in the rare cases 
I have this problem. It might be painful sometimes, but I don't think 
it's frequent.

Here's what happens because of false positives:

1. Write some function call, need to pass in a dynamic variable, invoke 
completion. Yeah, this one looks kinda right. `eval-defun', run... Nope! 
This one was just a poorly named function.

2. The reverse, for a symbol in funcall position.

3. Type a short prefix, invoke completion. Get a myriad things to sort 
through.

So, with code completion working this way you also have to use C-h f and 
C-h v more often than you might have had to otherwise, if only to verify 
that your code makes sense.

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2014-10-10 07:56 +0400, Dmitry Gutov wrote:
> It wouldn't be the worst approach, but we should be able to do better.
>
>> Also could we not force users to insert ` before any completion in
>> strings or comments.
>
> Please don't ignore the available information about why things work as
> they do currently. Have you looked at the revision that introduced
> that change? Do you have anything to add to the discussion in the
> related bug?

Sorry, I am merely reporting this as a usability bug and I don't intend
to get too deep into it (too little time :(). If I can work around it I
do that and get back to my work. For this bug I actually tried to get
used to it. I failed so I spoke up but not to blame any change.

You seem to be fairly confident on the approach chosen. Good and I look
forward to its improvements. Please be considerate on two things:

1. Leave room for other usage habits;

2. Rely less on heuristics. As also suggested by Stefan, do what he did
   with elisp--local-variables, i.e. code walk the expansion of the sexp
   and pick out the var/function postions/names. elisp--local-variables
   performs really well because it `guess' it right.

HTH,
Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/10/2014 08:33 AM, Leo Liu wrote:

> Sorry, I am merely reporting this as a usability bug and I don't intend
> to get too deep into it (too little time :().

I'm not asking you to code, just to read the available info. Otherwise 
you're assuming others have the time to repeat the discussions.

> I failed so I spoke up but not to blame any change.

Why not? vc-annotate is a standard tool.

> 1. Leave room for other usage habits;

Sure. But unfortunately, supporting a set of different usage habits 
means the result may be suboptimal for some of them.

On 10/11/2014 04:25 AM, Leo Liu wrote:

 > This gives me a sense that you are thinking about a situation when
 > someone uses company and has the completions displayed immediately.

Yes, that's my point of reference. But on the other hand, you could say 
that it just emphasises the existing problem: Lisp code outside of 
quotes usually constitutes the minority of the text in docstrings (and 
moreso in comments). The rest of the text would be better served by a 
different completion function (maybe dictionary-based, like Ispell). And 
if `lisp-completion-at-point' offers completions there, no other 
completion function can do so.

 > In standard default emacs behaviour, I had never getting anything
 > without me asking for it, be it in comments or strings or code. They
 > only appear when I ask.

Maybe someone else would prefer to see actual words as completions, when 
they are writing prose?

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/09/2014 07:27 PM, Stefan Monnier wrote:

> I mean, Elisp completion has been context-sensitive "for ever", so the
> change is not that it now is, but that it now reacts to the
> context differently.

It completed against "all available symbols" in way too many situations. 
And the logic was even simpler before 24.4, where IIRC you adopted some 
code from company-elisp (and also reimplemented some).

> I think this is not right: we should have a elisp--expect-exp-p
> which returns non-nil in a context where we know the symbol at point
> should be a valid expression (i.e. a variable).

Yes, I guess it would make more sense (this function can whitelist 
certain cases, like function name in a defun, for example).

 > Not sure how to write
> it, tho, but in its absence I think we're better off completing against
> "anything" than restricting ourselves to variables.

Emacs 25.1 is probably still far away enough for us to get this right. 
No need to revert useful changes.

> BTW, to improve on these behaviors, one approach is to take the
> macro-expansion code used in elisp--local-variables and use it
> "everywhere".  I.e. expand elisp--local-variables so it doesn't just
> return the list of local vars, but also returns whether we're inside
> a quote, or we're expecting a function, or we're expecting an expression.

I'm glad that you think this approach will work. Sounds like then the 
code will just have to handle a fixed number of forms, which is solvable.

Now I just have to wrap my head around `elisp--local-variables'. :)

bug#18643: 25.0.50; elisp--expect-function-p

[Stefan Monnier]

> I'm glad that you think this approach will work. Sounds like then the code
> will just have to handle a fixed number of forms, which is solvable.

I'm not sure it will, but experience seems to indicate that it works
well for local variables, so there's hope.

> Now I just have to wrap my head around `elisp--local-variables'. :)

It's very simple:

It takes the string between the first opening paren and point, appends
a special symbol (the witness) followed by the number of missing closing
parens to make the whole string a valid sexp, than reads&macroexpands
that.  And then we traverse the expanded code (a tree) straight from the
root to the leaf corresponding to point (which is done in
elisp--local-variables-1), collecting various info along the way.

Of course, traversing the tree efficiently is hard/impossible because
we don't know for sure where is the witness (which represents point) in
the tree, so we'd have to go through all the nodes of the tree, whereas
we want to limit ourselves to going down from the root straight to the
right leaf without backtracking.

Luckily, we know that point started "at the very end" of the sexp, and
macroexpansion tends to preserve the order, so if we always follow "the
rightmost child", we often find the witness.

Of course, it doesn't always work: e.g. if we start from (with-foo
witness), we'll generally end up with a macro-expanded code of the form
(unwind-protect (progn (something) witness) (somethingelse)) and going
down the rightmost child won't find the witness.

Maybe we could/should do a full backtracking traversal of the whole tree
instead, so it works more often.  After all, we use macroexpand-all
already, so we do perform such a traversal elsewhere anyway (and we
don't really have to: we could perform the macroexpansion only along the
spine we descend).


        Stefan

bug#18643: 25.0.50; elisp--expect-function-p

[Dmitry Gutov]

On 10/10/2014 05:20 PM, Stefan Monnier wrote:

>> Now I just have to wrap my head around `elisp--local-variables'. :)
>
> It's very simple:

:)

> It takes the string between the first opening paren and point, appends
> a special symbol (the witness) followed by the number of missing closing
> parens to make the whole string a valid sexp, than reads&macroexpands
> that.  And then we traverse the expanded code (a tree) straight from the
> root to the leaf corresponding to point (which is done in
> elisp--local-variables-1), collecting various info along the way.

Thanks!

I guess it might fail if some macros along the way require more 
arguments than we actually give them. But that can be worked around on 
case-by-case basis.

> Of course, traversing the tree efficiently is hard/impossible because
> we don't know for sure where is the witness (which represents point) in
> the tree, so we'd have to go through all the nodes of the tree, whereas
> we want to limit ourselves to going down from the root straight to the
> right leaf without backtracking.

I can see how performance could be a problem, but so far it's never been 
the bottleneck for me. In the cases I measured, (lisp--local-variables) 
takes around 1-2 ms (and up to 10ms near the end of the huge 
`cl--parse-loop-clause').

So even the caching mechanism you have in 
`lisp--local-variables-completion-table' could be unnecessary.

On the other hand, I've managed to find a specific case when it fails 
(will file the bug shortly).

bug#18643: 25.0.50; elisp--expect-function-p

[Lars Ingebrigtsen]

Leo Liu <sdl.web@gmail.com> writes:

> emacs should only constrain itself to function completion when
> absolutely sure.
>
> It is not a big deal if function names creep into variable completion or
> vice versa. Often it is handy to complete to an existing symbol and then
> edit it into something else. For example, one might need to create a
> variable name based on a function name.
>
> There are a few places where I expect to have a completion based on my
> past experience but fail now. For example, in (pred ...) pattern of
> pcase. Other failures, (let (|)) and (let ((|))) where | is point.
>
> I have experienced annoyances here and there and I think the fundamental
> solution is not to second guess but complete liberally as we did before.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

Skimming this bug report, it's not exactly clear what it's all about,
but perhaps things have changed since this was reported seven years ago.
That is, if you have

(defun lalabar ())
(defvar lalavar nil)

(let ((lala| ))

and hit `C-M-i', it'll expand to lalavar, and not lalabar.

But perhaps you had something other in mind?  Or does this work as you
wanted it to in recent Emacs versions?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#18643: 25.0.50; elisp--expect-function-p

[Leo Liu]

On 2022-04-26 15:44 +0200, Lars Ingebrigtsen wrote:
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
>
> Skimming this bug report, it's not exactly clear what it's all about,
> but perhaps things have changed since this was reported seven years ago.
> That is, if you have
>
> (defun lalabar ())
> (defvar lalavar nil)
>
> (let ((lala| ))
>
> and hit `C-M-i', it'll expand to lalavar, and not lalabar.
>
> But perhaps you had something other in mind?  Or does this work as you
> wanted it to in recent Emacs versions?

It's probably too late since this is like a long time ago. I remember
there were some changes to elisp that caused completion to fail in some
places. It was counter-productive for those who are used to the old way.
For example, (pcase x ((pred |) )).

Leo

bug#18643: 25.0.50; elisp--expect-function-p

[Lars Ingebrigtsen]

Leo Liu <sdl.web@gmail.com> writes:

> It's probably too late since this is like a long time ago. I remember
> there were some changes to elisp that caused completion to fail in some
> places. It was counter-productive for those who are used to the old way.
> For example, (pcase x ((pred |) )).

OK; I'm closing this bug report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no