CEDET completion-at-point-function

CEDET completion-at-point-function

[Stefan Monnier]

I've taken a look at improving semantic.el's
completion-at-point-function, and I'm wondering:

Can we use lexical-binding in there (completion-at-point-functions was
designed assuming closures are not a problem), or would it be a problem
for compatibility reasons?


        Stefan

Re: CEDET completion-at-point-function

[Eric M. Ludlam]

On 05/11/2014 02:24 PM, Stefan Monnier wrote:
> I've taken a look at improving semantic.el's
> completion-at-point-function, and I'm wondering:
>
> Can we use lexical-binding in there (completion-at-point-functions was
> designed assuming closures are not a problem), or would it be a problem
> for compatibility reasons?

Hi,

As far as I know, semantic-ia-complete-symbol (which is assigned 
indirectly to completion-at-point-function) has no external dependencies 
other than frequently being bound to a key.  For your reference, the 
entirety of "semantic/ia.el" was originally setup to be examples on how 
to use some of semantic's features to create different kinds of useful 
commands.  If this section can be updated as an improved example, I'm 
all for it.

Of course, semantic-ia-complete-symbol calls into other parts of the 
system for calculating completions, and I am not familiar enough with 
the lexical-binding topic to know if they would be negatively affected 
or not.  Is there a brief reference I could read about the topic?

Thanks
Eric

------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs

Re: [CEDET-devel] CEDET completion-at-point-function

[Stefan Monnier]

> Of course, semantic-ia-complete-symbol calls into other parts of the system
> for calculating completions, and I am not familiar enough with the
> lexical-binding topic to know if they would be negatively affected or not.
> Is there a brief reference I could read about the topic?

The issue of lexical-binding is just that it's a feature that was
introduced in Emacs-24.1, so code that relies on it won't work in
older Emacsen.

Anyway, I've since figured that it's simpler to use a new file.

Here is my work-in-progress code.  I'd welcome comments on it, since
I don't know much about Semantic I've had to make a few changes to it
(see the overloaded methods I had to change in semantic/analyze.el and
semantic/ia.el, meaning that the "overloadability" was moved to a new
method) as well as make some assumptions about some of its code (see
comment in semantic/analyze/complete.el).


        Stefan


=== modified file 'lisp/cedet/semantic.el'
--- lisp/cedet/semantic.el	2014-01-01 07:43:34 +0000
+++ lisp/cedet/semantic.el	2014-05-23 02:47:18 +0000
@@ -1155,9 +1157,6 @@
     ;; re-activated.
     (setq semantic-new-buffer-fcn-was-run nil)))
 
-(defun semantic-completion-at-point-function ()
-  'semantic-ia-complete-symbol)
-
 ;;; Autoload some functions that are not in semantic/loaddefs
 
 (autoload 'global-semantic-idle-completions-mode "semantic/idle"

=== modified file 'lisp/cedet/semantic/analyze.el'
--- lisp/cedet/semantic/analyze.el	2014-01-01 07:43:34 +0000
+++ lisp/cedet/semantic/analyze.el	2014-06-15 02:47:17 +0000
@@ -466,32 +466,24 @@
 ;;
 ;; Create a full-up context analysis.
 ;;
+
 ;;;###autoload
-(define-overloadable-function semantic-analyze-current-context (&optional position)
-  "Analyze the current context at optional POSITION.
-If called interactively, display interesting information about POSITION
-in a separate buffer.
+(define-overloadable-function semantic-analyze-sab-context (sab)
+  "Analyze the current context at SAB.
 Returns an object based on symbol `semantic-analyze-context'.
+SAB is an object as returned by `semantic-ctxt-current-symbol-and-bounds'.
 
-This function can be overridden with the symbol `analyze-context'.
-When overriding this function, your override will be called while
-cursor is at POSITION.  In addition, your function will not be called
-if a cached copy of the return object is found."
-  (interactive "d")
-  ;; Only do this in a Semantic enabled buffer.
-  (when (not (semantic-active-p))
-    (error "Cannot analyze buffers not supported by Semantic"))
+If a cached copy of the return object is found for symbol `current-context',
+it takes precedence."
+  (cl-assert (semantic-active-p))
   ;; Always refresh out tags in a safe way before doing the
   ;; context.
   (semantic-refresh-tags-safe)
-  ;; Do the rest of the analysis.
-  (if (not position) (setq position (point)))
   (save-excursion
-    (goto-char position)
-    (let* ((answer (semantic-get-cache-data 'current-context)))
+    (goto-char (car (nth 2 sab)))
+    (or (semantic-get-cache-data 'current-context)
       (with-syntax-table semantic-lex-syntax-table
-	(when (not answer)
-	  (setq answer (:override))
+	  (let ((answer (:override)))
 	  (when (and answer (oref answer bounds))
 	    (with-slots (bounds) answer
 	      (semantic-cache-data-to-buffer (current-buffer)
@@ -500,27 +492,19 @@
 					     answer
 					     'current-context
 					     'exit-cache-zone)))
-	  ;; Check for interactivity
-	  (when (called-interactively-p 'any)
-	    (if answer
-		(semantic-analyze-pop-to-context answer)
-	      (message "No Context."))
-	    ))
-
-	answer))))
+	    answer)))))
 
-(defun semantic-analyze-current-context-default (position)
-  "Analyze the current context at POSITION.
+(defun semantic-analyze-sab-context-default (sab)
+  "Analyze the current context SAB.
 Returns an object based on symbol `semantic-analyze-context'."
   (let* ((semantic-analyze-error-stack nil)
-	 (context-return nil)
-	 (prefixandbounds (semantic-ctxt-current-symbol-and-bounds (or position (point))))
+	 (prefixandbounds sab)
 	 (prefix (car prefixandbounds))
 	 (bounds (nth 2 prefixandbounds))
 	 ;; @todo - vv too early to really know this answer! vv
 	 (prefixclass (semantic-ctxt-current-class-list))
 	 (prefixtypes nil)
-	 (scope (semantic-calculate-scope position))
+	 (scope (semantic-calculate-scope (car bounds)))
 	 (function nil)
 	 (fntag nil)
 	 arg fntagend argtag
@@ -596,6 +580,8 @@
     ;; Only do work if we have bounds (meaning a prefix to complete)
     (when bounds
 
+      ;; FIXME: Code duplication!  This should use something like
+      ;; condition-case-unless-debug!
       (if debug-on-error
 	  (catch 'unfindable
 	    (setq prefix (semantic-analyze-find-tag-sequence
@@ -628,7 +614,6 @@
       ;; If we found a tag for our function, we can go into
       ;; functional context analysis mode, meaning we have a type
       ;; for the argument.
-      (setq context-return
 	    (semantic-analyze-context-functionarg
 	     "functionargument"
 	     :buffer (current-buffer)
@@ -640,7 +625,7 @@
 	     :prefixclass prefixclass
 	     :bounds bounds
 	     :prefixtypes prefixtypes
-	     :errors semantic-analyze-error-stack)))
+       :errors semantic-analyze-error-stack))
 
       ;; No function, try assignment
      ((and (setq assign (semantic-ctxt-current-assignment))
@@ -651,7 +636,6 @@
 	     (error (semantic-analyze-push-error err)
 		    nil)))
 
-      (setq context-return
 	    (semantic-analyze-context-assignment
 	     "assignment"
 	     :buffer (current-buffer)
@@ -661,7 +645,7 @@
 	     :prefix prefix
 	     :prefixclass prefixclass
 	     :prefixtypes prefixtypes
-	     :errors semantic-analyze-error-stack)))
+       :errors semantic-analyze-error-stack))
 
      ;; TODO: Identify return value condition.
      ;;((setq return .... what to do?)
@@ -669,7 +653,6 @@
 
      (bounds
       ;; Nothing in particular
-      (setq context-return
 	    (semantic-analyze-context
 	     "context"
 	     :buffer (current-buffer)
@@ -678,13 +661,34 @@
 	     :prefix prefix
 	     :prefixclass prefixclass
 	     :prefixtypes prefixtypes
-	     :errors semantic-analyze-error-stack)))
+       :errors semantic-analyze-error-stack))
 
-     (t (setq context-return nil))
-     )
+     (t nil)
+     )))
 
-    ;; Return our context.
-    context-return))
+;;;###autoload
+(defun semantic-analyze-current-context (&optional position interactive)
+  "Analyze the current context at optional POSITION.
+If called interactively, display interesting information about POSITION
+in a separate buffer.
+Returns an object based on symbol `semantic-analyze-context'.
+
+This function can be overridden with the symbol `analyze-context'.
+When overriding this function, your override will be called while
+cursor is at POSITION.  In addition, your function will not be called
+if a cached copy of the return object is found."
+  ;; FIXME: Shouldn't `analyze-context' above be `current-context'?
+  (interactive "d\np")
+  ;; Only do this in a Semantic enabled buffer.
+  (when (not (semantic-active-p))
+    (error "Cannot analyze buffers not supported by Semantic"))
+  (let* ((sab (semantic-ctxt-current-symbol-and-bounds position))
+         (answer (when sab (semantic-analyze-sab-context sab))))
+    (when interactive
+      (if answer
+          (semantic-analyze-pop-to-context answer)
+        (message "No Context.")))
+    answer))
 
 (defun semantic-analyze-dereference-alias (taglist)
   "Dereference first tag in TAGLIST if it is an alias.
@@ -742,7 +746,7 @@
   :group 'semantic
   :type semantic-format-tag-custom-list)
 
-(defun semantic-analyze-princ-sequence (sequence &optional prefix buff)
+(defun semantic-analyze-princ-sequence (sequence &optional prefix _buff)
   "Send the tag SEQUENCE to standard out.
 Use PREFIX as a label.
 Use BUFF as a source of override methods."
@@ -760,8 +764,7 @@
 	(princ (format "'%S" (car sequence)))))
       (princ "\n")
       (setq sequence (cdr sequence))
-      (setq prefix (make-string (length prefix) ? ))
-      ))
+    (setq prefix (make-string (length prefix) ?\s))))
 
 (defmethod semantic-analyze-show ((context semantic-analyze-context))
   "Insert CONTEXT into the current buffer in a nice way."

=== modified file 'lisp/cedet/semantic/analyze/complete.el'
--- lisp/cedet/semantic/analyze/complete.el	2014-01-01 07:43:34 +0000
+++ lisp/cedet/semantic/analyze/complete.el	2014-06-15 03:00:28 +0000
@@ -129,6 +129,14 @@
 	 (do-typeconstraint (not (memq 'no-tc flags)))
 	 (do-unique (not (memq 'no-unique flags)))
 	 )
+    ;; If the buffer text is "p->f_a", this code will only give us the fields
+    ;; of "p" which start with "f_a".  But we may want to complete it to
+    ;; "p->fastmap_accurate".
+    ;; In semantic/capf.el we hack around it by fudging `prefix' so it doesn't
+    ;; exactly contain the buffer text (e.g. it might pretend the user only
+    ;; typed "p->f" and let the generic completion code take responsibility for
+    ;; filtering out completions which don't contain the "_a").
+    ;; So don't assume that `prefix' really reflects the content of the buffer.
 
     ;; Calculate what our prefix string is so that we can
     ;; find all our matching text.

=== added file 'lisp/cedet/semantic/capf.el'
--- lisp/cedet/semantic/capf.el	1970-01-01 00:00:00 +0000
+++ lisp/cedet/semantic/capf.el	2014-06-15 03:03:44 +0000
@@ -0,0 +1,139 @@
+;;; capf.el --- Completion at point function for Semantic  -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2014  Free Software Foundation, Inc.
+
+;; Author: Stefan Monnier <monnier@iro.umontreal.ca>
+;; Keywords:
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;
+
+;;; Code:
+
+(require 'semantic/fw)
+(require 'semantic/ia)
+
+;; FIXME: compared to company-semantic:
+;; - we don't distinguish string-or-comment from code.
+;; - missing support for :company-docsig, :company-doc-buffer.
+
+;;;###autoload
+(defun semantic-completion-at-point-function ()
+  (when (semantic-active-p)
+    (let ((sab (semantic-ctxt-current-symbol-and-bounds)))
+      (when (nth 2 sab)
+        (let* ((bounds (nth 2 sab))
+               (cache (list nil))
+               (current (buffer-substring (car bounds) (cdr bounds)))
+               (table (semantic-capf-completion-table sab cache)))
+          (when (eq completion-ignore-case
+                    (string= (downcase current) current))
+            (setq table
+                  (completion-table-case-fold table completion-ignore-case)))
+          (list (car (nth 2 sab)) (cdr (nth 2 sab)) table
+                :exit-function
+                ;; FIXME: Use lexical-binding!
+                (lambda (str status)
+                  (when (eq status 'finished)
+                    (let* ((as (semantic-capf-context-and-syms str sab cache))
+                           (syms (cdr as))
+                           (tag (semantic-find-first-tag-by-name str syms)))
+                      (when tag
+                        (semantic-ia-completion-completed tag)))))
+                :company-location
+                (apply-partially #'semantic-capf-company-location sab cache)
+                ;; :company-doc-buffer
+                ;; (apply-partially #'semantic-capf-company-docbuffer sab cache)
+                ;; :company-docsig
+                ;; (apply-partially #'semantic-capf-company-docsig sab cache)
+                ))))))
+
+(defun semantic-capf-context-and-syms (prefix sab cache)
+  (unless (and (car cache)
+               (string-prefix-p (car cache) prefix))
+    (let* ((a (if (car cache) (cadr cache)
+                (semantic-analyze-sab-context sab)))
+           ;; `sab' and `a' embed the current buffer's "completion text", and
+           ;; semantic-analyze-possible-completions will compute the completion
+           ;; of that text, so replace it with the `prefix' which we want to
+           ;; complete.  Otherwise partial or substring completion can't work.
+           (_ (setf (car (last (oref a prefix))) prefix))
+           (syms (semantic-analyze-possible-completions a)))
+      (setcar cache prefix)
+      (setcdr cache (cons a syms))))
+  (cdr cache))
+
+
+(defun semantic-capf-completion-table (sab cache)
+  ;; Calculating completions is a two step process.
+  ;;
+  ;; The first analyzes the current context, which finds tags for
+  ;; all the stuff that may be referenced by the code around POS.
+  ;;
+  ;; The second step derives completions from that context.
+  (let ((buf (current-buffer)))
+    (completion-table-dynamic
+     (lambda (pre)
+       (with-current-buffer buf
+         ;; FIXME: Figure out how to use completion-boundaries to be able to do
+         ;; partial completion of "p->f" to "port->fastmap".
+         (let* ((as (semantic-capf-context-and-syms pre sab cache))
+                (a (car as))
+                (syms (cdr as)))
+           ;; Complete this symbol.
+           (or syms
+               (if (semantic-analyze-context-p a)
+                   ;; This is a clever hack.  If we were unable to find any
+                   ;; smart completions, let's divert to how senator derives
+                   ;; completions.
+                   ;;
+                   ;; This is a way of making this fcn more useful since
+                   ;; the smart completion engine sometimes fails.
+                   (all-completions pre
+                                    (semantic--completion-table
+                                     buf sab))))))))))
+
+(defun semantic-capf-company-location (sab cache str)
+  (let* ((as (semantic-capf-context-and-syms str sab cache))
+         (syms (cdr as))
+         (tag (assoc arg syms)))
+    (when (buffer-live-p (semantic-tag-buffer tag))
+      (cons (semantic-tag-buffer tag)
+            (semantic-tag-start tag)))))
+
+;; (defun semantic-capf-company-docbuffer (sab cache str)
+;;   (let* ((as (semantic-capf-context-and-syms str sab cache))
+;;          (syms (cdr as))
+;;          (tag (assoc arg syms))
+;;          (doc (company-semantic-documentation-for-tag tag)))
+;;     (when doc
+;;       (with-current-buffer (help-buffer)
+;;         (help-setup-xref `(semantic-capf-company-docbuffer ,sab ,cache)
+;;       (company-doc-buffer
+;;        (concat (funcall semantic-idle-summary-function tag nil t)
+;;                "\n"
+;;                doc)))))
+;;     (when (buffer-live-p (semantic-tag-buffer tag))
+;;       (cons (semantic-tag-buffer tag)
+;;             (semantic-tag-start tag)))))
+
+;; Local variables:
+;; generated-autoload-file: "loaddefs.el"
+;; End:
+
+(provide 'semantic/capf)
+;;; capf.el ends here

=== modified file 'lisp/cedet/semantic/ctxt.el'
--- lisp/cedet/semantic/ctxt.el	2014-01-01 07:43:34 +0000
+++ lisp/cedet/semantic/ctxt.el	2014-05-03 15:15:20 +0000
@@ -519,7 +519,7 @@
 	    (error nil)))
 	symlist))))
 
-
+;;;###autoload
 (define-overloadable-function semantic-ctxt-current-symbol-and-bounds (&optional point)
   "Return the current symbol and bounds the cursor is on at POINT.
 The symbol should be the same as returned by `semantic-ctxt-current-symbol'.

=== modified file 'lisp/cedet/semantic/ia.el'
--- lisp/cedet/semantic/ia.el	2014-01-13 20:04:08 +0000
+++ lisp/cedet/semantic/ia.el	2014-06-14 14:39:27 +0000
@@ -68,22 +68,30 @@
 ;; character after function names.  For Lisp, it might check
 ;; to put a "(" in front of a function name.
 
-(define-overloadable-function semantic-ia-insert-tag (tag)
-  "Insert TAG into the current buffer based on completion.")
+(define-overloadable-function semantic-ia-completion-completed (tag)
+  "Add extra text once completion is completed.")
 
-(defun semantic-ia-insert-tag-default (tag)
-  "Insert TAG into the current buffer based on completion."
-  (insert (semantic-tag-name tag))
+(defun semantic-ia-completion-completed-default (tag)
   (let ((tt (semantic-tag-class tag)))
-    (cond ((eq tt 'function)
-	   (insert "("))
+    (cond ((and (eq tt 'function)
+                (not (looking-at "[ \t]*(")))
+           ;; FIXME: GNU style (among others) wants a space before the
+           ;; open paren!
+           (let ((last-command-event ?\())
+             (call-interactively #'self-insert-command)))
 	  (t nil))))
 
+(defun semantic-ia-insert-tag (tag)
+  (insert (semantic-tag-name tag))
+  (semantic-ia-completion-completed tag))
+(make-obsolete 'semantic-ia-insert-tag
+               'semantic-ia-completion-completed "24.5")
+
 (defalias 'semantic-ia-get-completions 'semantic-ia-get-completions-deprecated
   "`Semantic-ia-get-completions' is obsolete.
 Use `semantic-analyze-possible-completions' instead.")
 
-(defun semantic-ia-get-completions-deprecated (context point)
+(defun semantic-ia-get-completions-deprecated (context _point)
   "A function to help transition away from `semantic-ia-get-completions'.
 Return completions based on CONTEXT at POINT.
 You should not use this, nor the aliased version.

Re: CEDET completion-at-point-function

[Eric M. Ludlam]

On 06/14/2014 11:14 PM, Stefan Monnier wrote:
> Anyway, I've since figured that it's simpler to use a new file.
>
> Here is my work-in-progress code.  I'd welcome comments on it, since
> I don't know much about Semantic I've had to make a few changes to it
> (see the overloaded methods I had to change in semantic/analyze.el and
> semantic/ia.el, meaning that the "overloadability" was moved to a new
> method) as well as make some assumptions about some of its code (see
> comment in semantic/analyze/complete.el).

Thanks for looking into this.  I have a few thoughts, though I wasn't 
always certain what your goal was in a particular change;


> === modified file 'lisp/cedet/semantic/analyze.el'
> --- lisp/cedet/semantic/analyze.el	2014-01-01 07:43:34 +0000
> +++ lisp/cedet/semantic/analyze.el	2014-06-15 02:47:17 +0000

[...]


> +      ;; FIXME: Code duplication!  This should use something like
> +      ;; condition-case-unless-debug!
>         (if debug-on-error

I was unaware of this.  I'll start using it instead.

> +;;;###autoload
> +(defun semantic-analyze-current-context (&optional position interactive)
> +  "Analyze the current context at optional POSITION.
> +If called interactively, display interesting information about POSITION
> +in a separate buffer.
> +Returns an object based on symbol `semantic-analyze-context'.
> +
> +This function can be overridden with the symbol `analyze-context'.
> +When overriding this function, your override will be called while
> +cursor is at POSITION.  In addition, your function will not be called
> +if a cached copy of the return object is found."
> +  ;; FIXME: Shouldn't `analyze-context' above be `current-context'?

That whole piece of doc should be removed, as there is no override 
symbol interface to 'mode-local' anymore.   Thanks for noticing that.

Splitting the implementation apart seems like a good idea based on the 
way you are using it.

> === modified file 'lisp/cedet/semantic/analyze/complete.el'
> --- lisp/cedet/semantic/analyze/complete.el	2014-01-01 07:43:34 +0000
> +++ lisp/cedet/semantic/analyze/complete.el	2014-06-15 03:00:28 +0000
> @@ -129,6 +129,14 @@
>   	 (do-typeconstraint (not (memq 'no-tc flags)))
>   	 (do-unique (not (memq 'no-unique flags)))
>   	 )
> +    ;; If the buffer text is "p->f_a", this code will only give us the fields
> +    ;; of "p" which start with "f_a".  But we may want to complete it to
> +    ;; "p->fastmap_accurate".
> +    ;; In semantic/capf.el we hack around it by fudging `prefix' so it doesn't
> +    ;; exactly contain the buffer text (e.g. it might pretend the user only
> +    ;; typed "p->f" and let the generic completion code take responsibility for
> +    ;; filtering out completions which don't contain the "_a").
> +    ;; So don't assume that `prefix' really reflects the content of the buffer.

I use old school completion in here from before the new fuzzy matching 
was introduced.

The ideal solution would be to have a new function such as 
`semantic-find-tags-for-completion-fuzzy' or whichever word you use to 
describe the behavior to fit in with all the other semantic-find-* 
functions.  Since the completion engine also accepts a set of flags, 
semantic/capf could just pass in a flag for fuzzy matching instead of 
trying to work around the missing feature.

> === added file 'lisp/cedet/semantic/capf.el'
> --- lisp/cedet/semantic/capf.el	1970-01-01 00:00:00 +0000
> +++ lisp/cedet/semantic/capf.el	2014-06-15 03:03:44 +0000

[...]

> +(defun semantic-capf-completion-table (sab cache)
> +  ;; Calculating completions is a two step process.
> +  ;;
> +  ;; The first analyzes the current context, which finds tags for
> +  ;; all the stuff that may be referenced by the code around POS.
> +  ;;
> +  ;; The second step derives completions from that context.
> +  (let ((buf (current-buffer)))
> +    (completion-table-dynamic
> +     (lambda (pre)
> +       (with-current-buffer buf
> +         ;; FIXME: Figure out how to use completion-boundaries to be able to do
> +         ;; partial completion of "p->f" to "port->fastmap".

Interesting idea.  As I'm only inspecting the patch and I'm not quite 
sure how we got here or what completion-boundaries are, but 
`semantic-ctxt-current-symbol-and-bounds' will provide back both "p" and 
"f", and the completion engine would throw an error if there is no 'p'. 
  When deriving the current-context the returned class shows that more 
than one symbol in the prefix is unfound.  This is a notice that you 
could do some fuzzy cross-symbol matching for a case like this.

I think the only way to have the semantic completion engine provide the 
data to do that is for the s-a-b function to return bounds for each 
found symbol instead of just the last one.  There are a bunch of 
assumptions around this so it would be a fundamental change, but also 
valuable.  It would certainly be possible to start with 'p' and walk 
through the unknown strings doing completion along the way if we knew 
where those text strings were in the buffer.

What is particularly interesting is that if you know you have p->f, then 
p must be some sort of struct/class, so you could filter out ll the 
symbols that are not one of those.  That is a new kind of filter the 
engine doesn't do yet.   The s-a-b function would need to be upgraded to 
interpret the boundary text into an indication of the completion filter.

Sounds fun!  I'd be tempted to investigate that when I'm done with my 
current EDE enhancements task.

Eric





------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems

Re: [CEDET-devel] CEDET completion-at-point-function

[Stefan Monnier]

> Splitting the implementation apart seems like a good idea based on the way
> you are using it.

It's an incompatible change, so I'm interested in your opinion about the
consequences.  IOW how likely is it that there are existing overrides of
those thingies "out in the wild"?  Those in Emacs's CEDET code are of course
easy/trivial to fix.
I (currently) need two such incompatible changes:
- one to split the "get symbol-and-bounds at point" from the actual
  analysis of the current context.
- one to separate the insertion of the tag's name from the insertion of
  supplemental text after it such as "()".

> The ideal solution would be to have a new function such as
> `semantic-find-tags-for-completion-fuzzy' or whichever word you use to
> describe the behavior to fit in with all the other semantic-find-* 
> functions.  Since the completion engine also accepts a set of flags,
> semantic/capf could just pass in a flag for fuzzy matching instead of trying
> to work around the missing feature.

It could be useful for CEDET to do the fuzzy matching in some cases, but
so far I'm not interested in this option.  Instead, I want to handle the
case where the fuzzy matching is performed by the generic completion
code, according to `completion-styles'.

The patch I sent does correctly let one complete "p->f_a" to
"p->fastmap_accurate".

> I think the only way to have the semantic completion engine provide the data
> to do that is for the s-a-b function to return bounds for each found symbol
> instead of just the last one.

Indeed, that's the first step.

> There are a bunch of assumptions around this
> so it would be a fundamental change, but also valuable.  It would certainly
> be possible to start with 'p' and walk through the unknown strings doing
> completion along the way if we knew where those text strings were in
> the buffer.

That's what the generic completion code would do if we could give it the
right info.  It does that already when you complete "/us/s/man" to
"/usr/share/man".

> What is particularly interesting is that if you know you have p->f, then
> p must be some sort of struct/class, so you could filter out ll the symbols
> that are not one of those.  That is a new kind of filter the engine doesn't
> do yet.

I think the generic completion code would already take care of that.
E.g. it currently completes "/us/s/man" to "/usr/share/man" even if
"/us/s/m" only completes to "/usr/s/m" because there are "m" completions
under "/usr/sbin", "/usr/src", and "/usr/share".

Tho admittedly, the above works because the completions for "/usr/s"
look like "share/" rather than "share".  So we might need to arrange for
the completion table to return "port->" rather than "port" when "port"
is a variable of type "pointer to struct/class", and of course only do
it when completing "the left hand side".

The "completion-boundaries" is a way for the completion table to tell
the completion code about how "/u/b/em" is split into 3 parts that are
completed "separately".


        Stefan

Re: CEDET completion-at-point-function

[Eric M. Ludlam]

On 06/16/2014 04:52 PM, Stefan Monnier wrote:
>> Splitting the implementation apart seems like a good idea based on the way
>> you are using it.
>
> It's an incompatible change, so I'm interested in your opinion about the
> consequences.  IOW how likely is it that there are existing overrides of
> those thingies "out in the wild"?  Those in Emacs's CEDET code are of course
> easy/trivial to fix.
> I (currently) need two such incompatible changes:
> - one to split the "get symbol-and-bounds at point" from the actual
>    analysis of the current context.

I do not think this will be a compatibility issue.  The srecoder mode 
has an overload here you could test with to see if it becomes a problem 
in the .srt (template) files.

> - one to separate the insertion of the tag's name from the insertion of
>    supplemental text after it such as "()".

I doubt this was ever overloaded.  Almost all uses of semantic's 
completion API are directly via semantic-analyze-possible-completions, 
and they handle their own insertion.

[...]

>> What is particularly interesting is that if you know you have p->f, then
>> p must be some sort of struct/class, so you could filter out ll the symbols
>> that are not one of those.  That is a new kind of filter the engine doesn't
>> do yet.
>
> I think the generic completion code would already take care of that.
> E.g. it currently completes "/us/s/man" to "/usr/share/man" even if
> "/us/s/m" only completes to "/usr/s/m" because there are "m" completions
> under "/usr/sbin", "/usr/src", and "/usr/share".
>
> Tho admittedly, the above works because the completions for "/usr/s"
> look like "share/" rather than "share".  So we might need to arrange for
> the completion table to return "port->" rather than "port" when "port"
> is a variable of type "pointer to struct/class", and of course only do
> it when completing "the left hand side".
>
> The "completion-boundaries" is a way for the completion table to tell
> the completion code about how "/u/b/em" is split into 3 parts that are
> completed "separately".

I think I'll need to play with completion tables and completion 
boundaries to better comment.  There is an expense expanding from some 
symbol p (which could be port) past the ->, and to the next symbol.  If 
there happen to be 10 possible "p" expansions that are all also some 
sort of struct, then p->f, the f could be a rather large  number of 
possible things, and thus be expensive if not handled carefully.  For 
example if the code had 4 variables "port1" "port2", etc that were all 
the same type.  When expanding the ->f, that should happen once instead 
of 4 times because we know all the types are the same.

When expanding p->f->a->d there are 3 different kinds of completion.

The first symbol "p" could be a rather large number of things from the 
global name space.   "f" and "a" are pretty easy, since you are now in 
the "expand a data type" space, semantic builds a nice datatype table to 
reference.  "d" is generally derived the same as f and a, but we apply 
one last filter of the return type, so if you had:

q = p->f->a->d;

Then d's type should be similar to q.

If you are skipping parts of the completion engine, you may miss some of 
the extra filters, or some of the optimizations.

I hope that helps, though I'm pretty sure it won't impact your current 
experiment much since these are all thoughts of optional optimizations.

Eric

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems

Re: CEDET completion-at-point-function

[Stefan Monnier]

> to better comment.  There is an expense expanding from some symbol p (which
> could be port) past the ->, and to the next symbol.  If there happen to be
> 10 possible "p" expansions that are all also some sort of struct, then p->f,
> the f could be a rather large  number of possible things, and thus be
> expensive if not handled carefully.

Indeed.

> For example if the code had 4 variables "port1" "port2", etc that were
> all the same type.  When expanding the ->f, that should happen once
> instead of 4 times because we know all the types are the same.

A bit of caching should go a long way, but in any case, as you mention:
we'll cross that bridge when we get there.

> reference.  "d" is generally derived the same as f and a, but we apply one
> last filter of the return type, so if you had:
> q = p->f->a->d;
> Then d's type should be similar to q.

That's odd.  Does that mean that the user might have been unable to use
completion on the "p->f->a" part (before typing "->d") because the "a"
field was not of a similar type as q?
Or do you first use this "type similar to context" filter an if the
filter doesn't let anything through, then you drop the filter (kind of
like we filter out files that match completion-ignored-extensions,
except when that results in "no match")?

> I hope that helps, though I'm pretty sure it won't impact your current
> experiment much since these are all thoughts of optional optimizations.

Indeed.


        Stefan

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems

Re: CEDET completion-at-point-function

[Eric M. Ludlam]

On 06/18/2014 09:47 PM, Stefan Monnier wrote:
>> reference.  "d" is generally derived the same as f and a, but we apply one
>> >  last filter of the return type, so if you had:
>> >  q = p->f->a->d;
>> >  Then d's type should be similar to q.
> That's odd.  Does that mean that the user might have been unable to use
> completion on the "p->f->a" part (before typing "->d") because the "a"
> field was not of a similar type as q?
> Or do you first use this "type similar to context" filter an if the
> filter doesn't let anything through, then you drop the filter (kind of
> like we filter out files that match completion-ignored-extensions,
> except when that results in "no match")?

Good call.  The list of all possible completions from withing a 
struct/class contains any terminal type that matches (ie - int, char, 
etc) and any type that is a compound type (structs, classes) so it is both.

Basically, if you are completing to an int, it will toss chars, floats, 
etc but keep user types which have fields.

You can see it in semantic/analyze/complete around line 230.

Eric


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems