ido slow for outline path completion

ido slow for outline path completion

[Samuel Wales]

ido for refiling works very well and makes refiling easier
by orders of magnitude.  Thank you Eric.  Thank you Carsten.

In org 6.14, I find that full path completion (which is the
only way that org-goto can use ido) can take 22-45 seconds
before it shows you the prompt.  This is on a 1ghz mac and
about a 300k org file with lots of nesting.

Is there a way to speed this up?  Offhand I imagine:

  - allow org-goto to limit the depth of the tree.  i do
    this for org-refile to depth 5.  this takes 3-5 seconds.
    - to achieve this i show only the basename.  is it
      possible with the full path in org-refile?
  - of course, it would be better not to have limits and do
    profiling/caching/..., but i don't know what the issues
    could be.  is it ido or org?  is it sorting?

Here are my settings for ido, goto, refile, and remember
refiling:

(ido-mode 1)
(ido-everywhere 1)
(setf ido-confirm-unique-completion t)

(setq org-completion-use-ido t)
;;slow (45 seconds before prompt shows, a week or two ago
;;2008-12-12).  but this is what i want.  would be even
;;better with uniquify type disambiguation.
(setf org-outline-path-complete-in-steps nil)

;;slow with ido 2008-12-12.  takes 22 seconds on about 300k
;;org file on 1ghz mac.  use 'outline for searching and
;;org-occur and navigation.  i prefer olpath.
(setf org-goto-interface 'outline-path-completion)
;;t does not work with making help stuff one window.  i think this
;;is moot unless for some reason you want outline.
(setf org-goto-auto-isearch nil)

;;for speed, set level to 5 and set this to nil.  for full
;;path completion, set level to be unlimited and/or set this
;;to t.
(setf org-refile-use-outline-path nil)
;;this is as slow as setting org-goto-interface to
;;olpath completion.
;;(setf org-refile-use-outline-path t)
;;i would use this functionality for both goto and refile if there
;;were a uniquify type disambiguation scheme.
;;(setf org-refile-use-outline-path 'file)
;;this is the most candidates i can handle the slowness of.  it
;;takes 5 seconds or so on about 300k org file on a 1ghz mac.
(setq org-refile-targets '((org-agenda-files . (:maxlevel . 5))))

;;can be re also.  this is for refile and remember.
(setf org-reverse-note-order t)

;;default is to use refile interface, which is what i want
;;(setf org-remember-interactive-interface ...)

Thanks.

-- 
Myalgic encephalomyelitis denialists are knowingly causing further
suffering and death by opposing biomedical research on this serious
infectious disease.  Do you care about the world?
http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm

Re: ido slow for outline path completion

[Carsten Dominik]

Hi Samuel,

I am sure that both org and ido will contribute to a slow start
of this command when the lists are long.  ido does have a noticeable
on performance, but that is more than outweighed by the speedup of using
its commands.

However, I believe the more significant problem may
be in Org in this case.  The refile interface was
written with the idea that refile targets are a limited
set of entries.  So when Org constructs the list of
refile targets, it first finds each target according
to its criteria, and then it adds all necessary information.
When using the outline path, it scans back from the entry to top
level and records all the parents and grand parents.

If this assumption that the possible targets are not a
limited subset of all entries in a file fails, then this
implementation is a bad one because it requires
potentially N^2 operations where N is the number
of entries in the file.

There are several ways that could be used to improve
this situation:

1. Compile Org!  Are you using byte-compiled files?
    In a case like this, the differences may be significant.

2. Improve the way Org finds information about the outline tree.
    Currently it uses `outline-up-heading', which was written
    with arbitrary outline settings in mind, I could
    probably write an org-mode specific one that could
    be a lot faster.
     ... hack hack ....  OK, I just did that, seems to
    speed up the Org part of it by a factor to 3 or so...
    Better, but still not quite acceptable, I guess.....

3. Cacheing: We could cache the refile targets - in fact
    the first implementation of this feature did that, but I
    stepped away from this later because it requires the
    user to decide when an update of the cache should happen,
    and it relies on searches for the headline text, which may
    fail if there are several identical headings in a file.

4. Create the list of headings with full outline path
    in a single path of the file, similar to the tags matcher.
    This would avoid the N^2 operation I discussed above.


......

OK, I did some profiling, implemented the single-path construction
of outline paths for the case where this is possible (the :maxlevel
case wich is also used by org-goto), and also removed a stupid file
name comparison wich unnecessarily constructed a file-truename for
each and every hit in the list.

There is also a new variable `org-goto-max-level' with
a default of 5.

This should be *a lot* faster now.  Thanks for the report!

- Carsten



On Dec 12, 2008, at 9:04 PM, Samuel Wales wrote:

> ido for refiling works very well and makes refiling easier
> by orders of magnitude.  Thank you Eric.  Thank you Carsten.
>
> In org 6.14, I find that full path completion (which is the
> only way that org-goto can use ido) can take 22-45 seconds
> before it shows you the prompt.  This is on a 1ghz mac and
> about a 300k org file with lots of nesting.
>
> Is there a way to speed this up?  Offhand I imagine:
>
> - allow org-goto to limit the depth of the tree.  i do
>   this for org-refile to depth 5.  this takes 3-5 seconds.
>   - to achieve this i show only the basename.  is it
>     possible with the full path in org-refile?
> - of course, it would be better not to have limits and do
>   profiling/caching/..., but i don't know what the issues
>   could be.  is it ido or org?  is it sorting?
>
> Here are my settings for ido, goto, refile, and remember
> refiling:
>
> (ido-mode 1)
> (ido-everywhere 1)
> (setf ido-confirm-unique-completion t)
>
> (setq org-completion-use-ido t)
> ;;slow (45 seconds before prompt shows, a week or two ago
> ;;2008-12-12).  but this is what i want.  would be even
> ;;better with uniquify type disambiguation.
> (setf org-outline-path-complete-in-steps nil)
>
> ;;slow with ido 2008-12-12.  takes 22 seconds on about 300k
> ;;org file on 1ghz mac.  use 'outline for searching and
> ;;org-occur and navigation.  i prefer olpath.
> (setf org-goto-interface 'outline-path-completion)
> ;;t does not work with making help stuff one window.  i think this
> ;;is moot unless for some reason you want outline.
> (setf org-goto-auto-isearch nil)
>
> ;;for speed, set level to 5 and set this to nil.  for full
> ;;path completion, set level to be unlimited and/or set this
> ;;to t.
> (setf org-refile-use-outline-path nil)
> ;;this is as slow as setting org-goto-interface to
> ;;olpath completion.
> ;;(setf org-refile-use-outline-path t)
> ;;i would use this functionality for both goto and refile if there
> ;;were a uniquify type disambiguation scheme.
> ;;(setf org-refile-use-outline-path 'file)
> ;;this is the most candidates i can handle the slowness of.  it
> ;;takes 5 seconds or so on about 300k org file on a 1ghz mac.
> (setq org-refile-targets '((org-agenda-files . (:maxlevel . 5))))
>
> ;;can be re also.  this is for refile and remember.
> (setf org-reverse-note-order t)
>
> ;;default is to use refile interface, which is what i want
> ;;(setf org-remember-interactive-interface ...)
>
> Thanks.
>
> -- 
> Myalgic encephalomyelitis denialists are knowingly causing further
> suffering and death by opposing biomedical research on this serious
> infectious disease.  Do you care about the world?
> http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: ido slow for outline path completion

[Samuel Wales]

Hi Carsten,

On Mon, Dec 15, 2008 at 02:36, Carsten Dominik <dominik@science.uva.nl> wrote:
> 1. Compile Org!  Are you using byte-compiled files?
>    In a case like this, the differences may be significant.

Always!  :)

> 4. Create the list of headings with full outline path
>    in a single path of the file, similar to the tags matcher.
>    This would avoid the N^2 operation I discussed above.

Single pass?  Very nice.

> There is also a new variable `org-goto-max-level' with
> a default of 5.

Excellent.  Might also be useful to unify the syntax for
org-goto, org-refile, remember.

> This should be *a lot* faster now.  Thanks for the report!

Almost instantaneous in many cases.

However, when I do org-refile as full path (not basenames),
it sometimes takes 45 seconds.

I haven't narrowed it down yet.  Emacs just freezes.

Every character I type from that point on takes another 45
seconds or so.  Deletes also take that long until I get back
to the place where it was fast.  At that point it is fast
again.

The difference from before is that before, it occurred
before the minibuffer showed the candidates, while now it
occurs in the middle of entering the selection string.

Discontiguity might something to do with it -- sometimes it
happens when the character typed is different from the next
char in the target.  Maybe the candidate list needs lots of
searching in that case.  Maybe it is starting from the
beginning to match against the path?

I have not noticed this happening when I use basenames in
org-refile ido.  This leads me to conjecture that it might
be the size of the full paths.  Maybe too much stuff for the
ido flex matcher to consider.  uniquify-style disambiguation
might help by reducing the number of characters.

It might or might not have something to do with typing a
space that exists next in the target -- or not typing the
same.  Another possible hint is that sometimes, typing a
space inserts a slash.  Not sure why.

As I said, I have not narrowed down the exact cases.  If you
have a priori ideas, please let me know and I will try them.

Thank you very much.

-- 
Myalgic encephalomyelitis denialists are knowingly causing further
suffering and death by opposing biomedical research on this serious
infectious disease.  Do you care about the world?
http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm

Re: ido slow for outline path completion

[Carsten Dominik]

[-- Attachment #1.1: Type: text/plain, Size: 1828 bytes --]

Hi Samuel,
>
> However, when I do org-refile as full path (not basenames),
> it sometimes takes 45 seconds.

Could you sent a test file for this?  I cannot reproduce it.


- Carsten

>
>
> I haven't narrowed it down yet.  Emacs just freezes.
>
> Every character I type from that point on takes another 45
> seconds or so.  Deletes also take that long until I get back
> to the place where it was fast.  At that point it is fast
> again.
>
> The difference from before is that before, it occurred
> before the minibuffer showed the candidates, while now it
> occurs in the middle of entering the selection string.
>
> Discontiguity might something to do with it -- sometimes it
> happens when the character typed is different from the next
> char in the target.  Maybe the candidate list needs lots of
> searching in that case.  Maybe it is starting from the
> beginning to match against the path?
>
> I have not noticed this happening when I use basenames in
> org-refile ido.  This leads me to conjecture that it might
> be the size of the full paths.  Maybe too much stuff for the
> ido flex matcher to consider.  uniquify-style disambiguation
> might help by reducing the number of characters.
>
> It might or might not have something to do with typing a
> space that exists next in the target -- or not typing the
> same.  Another possible hint is that sometimes, typing a
> space inserts a slash.  Not sure why.
>
> As I said, I have not narrowed down the exact cases.  If you
> have a priori ideas, please let me know and I will try them.
>
> Thank you very much.
>
> -- 
> Myalgic encephalomyelitis denialists are knowingly causing further
> suffering and death by opposing biomedical research on this serious
> infectious disease.  Do you care about the world?
> http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm


[-- Attachment #1.2: Type: text/html, Size: 2321 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: ido slow for outline path completion

[Samuel Wales]

Hi Carsten,

On Fri, Dec 19, 2008 at 01:39, Carsten Dominik <dominik@science.uva.nl> wrote:
> Could you sent a test file for this?  I cannot reproduce it.

My org agenda files take up 500kB (I think completion uses those?) and
I don't know what about them is causing this in order to create a test
case.

I wish there were a better way.  I guess I just have to let you know
when I find out something more.

I presume that you tried large, nested org files.  If your computer is
10x faster (which might be possible), then you have to notice a 4.5
second freeze.  You would have noticed that, however.

Thanks for trying it.

Re: ido slow for outline path completion

[Samuel Wales]

*** ido
Hi Carsten,

Thanks for looking at the speed issues.

This time an error showed up.  The error from the messages
buffer is:

  Getting targets...done
  eq: Wrong type argument: arrayp, (#("footodo/...

The trigger is:

org-goto, same settings as before, searching for a headline called
"parsing risk",
which does exist but never shows up as the main candidate.
I did load-file on ido.el and org.el for a better backtrace.

I don't know the meanings of the different color patterns in
ido, and this might be a hint for speed, or this error, or both.

p --
  shows an entry in body text face with a ^ with a red
  background, rest of candidates are in red, fast.
pa -- same but with different entry
par -- same but with different entry
pars -- only one candidate shows and it is in green, fast
parsi --
  back to the first color pattern, different entry, slightly
  slower
parsin -- same entry as parsi, slightly slower
parsing --
  same but with different entry, slower still (but only a
  few seconds for me; maybe not noticeable on your computer)
parsing  (space after g) --
  causes error, space not inserted, fast

I append the backtrace.  It isn't perfect.  I deleted a
small number of lines that had the entire completion list in
them (the whole backtrace is 700k) and had to minorly
obfuscate a line for privacy, but did so in a consistent
way.

ido-auto-merge-delay-time might or might not be relevant.  I
have it set to the default of 0.7.

Thanks.

=====

The backtrace is:

  aref((#("footodo/mumble/issues/is froz bar foo something risking/" 0
6 (face org-level-1 org-category "todo-foo" fontified t) 6 7 nil 7 13
(face org-level-1 org-category #("mumble" 0 6 ...) fontified t) 13 14
nil 14 20 (face org-level-1 org-category #("mumble" 0 6 ...) fontified
t) 20 21 nil 21 64 (org-category #("mumble" 0 6 ...) fontified nil) 64
65 nil) "/Users/foo/Desktop/a/quux/todo-foo.org"
"^\\*\\*\\*\\*\\*\\*\\* TODO is froz bar foo something risking[ 	]*$"
17576) 3)
  (eq (aref s l) 47)
  (and (> l 0) (eq (aref s l) 47))
  (if (and (> l 0) (eq ... 47)) (substring s 0 l) s)
  (let ((l ...)) (if (and ... ...) (substring s 0 l) s))
  ido-no-final-slash((#("...
  (string-lessp (ido-no-final-slash a) (ido-no-final-slash b))
  ido-file-lessp((#("...
  sort(((#("...
  (let ((completion-list ...)) (if (featurep ...) (let ... ...)
(display-completion-list completion-list)))
  (with-output-to-temp-buffer ido-completion-buffer (let (...) (if ...
... ...)))
  (if display-it (with-output-to-temp-buffer ido-completion-buffer
(let ... ...)))
  (let ((temp-buf ...) display-it full-list) (if (and ... temp-buf)
(let ... ... ... ... ... ...) (setq display-it t)) (if display-it
(with-output-to-temp-buffer ido-completion-buffer ...)))
  ido-completion-help()
  call-interactively(ido-completion-help)
  (if (and (not ...) (not ...)) (progn (setq ido-rescan ...)
(delete-region ... ...) (insert res)) (call-interactively (setq
this-command ido-cannot-complete-command)))
  (cond (ido-incomplete-regexp) ((and ... ...) (let ... ...))
(ido-directory-too-big (setq ido-directory-too-big nil) (setq
ido-text-init ido-text) (setq ido-exit ...) (exit-minibuffer)) ((not
ido-matches) (when ido-completion-buffer ...)) ((and ... ... ...) (if
... ... ... ... ...)) (t (setq res ido-common-match-string) (if ...
... ...)))
  (let (res) (cond (ido-incomplete-regexp) (... ...)
(ido-directory-too-big ... ... ... ...) (... ...) (... ...) (t ...
...)))
  ido-complete()
  (if (and (stringp ido-common-match-string) (stringp ido-text) (cond
... ... ...)) (insert " ") (ido-complete))
  ido-complete-space()
  call-interactively(ido-complete-space)
  completing-read("Goto: " (("dummy" . 1)) nil nil nil org-refile-history)
  (catch (quote ido) (completing-read (ido-make-prompt item prompt)
(quote ...) nil nil (prog1 ido-text-init ...) history))
  (setq ido-final-text (catch (quote ido) (completing-read ... ... nil
nil ... history)))
  (let ((minibuffer-local-completion-map ...)
(minibuffer-local-filename-completion-map ...) (max-mini-window-height
...) (ido-completing-read t) (ido-require-match require-match)
(ido-use-mycompletion-depth ...) (show-paren-mode nil)) (setq ido-exit
nil) (setq ido-final-text (catch ... ...)))
  (while (not done) (ido-trace "
_LOOP_" ido-text-init) (setq ido-exit nil) (setq ido-rescan t) (setq
ido-rotate nil) (setq ido-text "") (when ido-set-default-item (setq
ido-default-item ...) (if ... ...) (ido-trace "new default"
ido-default-item) (if ido-default-item ...) (setq ido-set-default-item
nil)) (if ido-process-ignore-lists-inhibit (setq
ido-process-ignore-lists nil)) (if (and ido-use-merged-list ... ...)
(let ... ... ...)) (cond (ido-keep-item-list ...) (... ...) (... ...)
(... ...) (... ...) (t nil)) (setq ido-rotate-temp nil) (if
ido-process-ignore-lists-inhibit (setq ido-process-ignore-lists t
ido-process-ignore-lists-inhibit nil)) (ido-set-matches) (if (and
ido-matches ...) (setq ido-try-merged-list t)) (let (... ... ... ...
... ... ...) (setq ido-exit nil) (setq ido-final-text ...)) (ido-trace
"completing-read" ido-final-text) (if (get-buffer
ido-completion-buffer) (kill-buffer ido-completion-buffer)) (ido-trace
"
_EXIT_" ido-exit) (cond (... ... nil) (... ...) (... ... nil) (...
...) (... ...) (... ...) (... ... ...) (... ... ...) (... ...) (t ...
...)))
  (let ((ido-cur-item item) (ido-entry-buffer ...)
(ido-process-ignore-lists t) (ido-process-ignore-lists-inhibit nil)
(ido-set-default-item t) ido-default-item ido-selected ido-final-text
(done nil) (icomplete-mode nil) ido-cur-list ido-ignored-list
(ido-rotate-temp nil) (ido-keep-item-list nil) (ido-use-merged-list
nil) (ido-try-merged-list t) (ido-pre-merge-state nil) (ido-case-fold
ido-case-fold) (ido-enable-prefix ido-enable-prefix)
(ido-enable-regexp ido-enable-regexp)) (ido-setup-completion-map)
(setq ido-text-init initial) (setq ido-input-stack nil) (run-hooks
(quote ido-setup-hook)) (while (not done) (ido-trace "
_LOOP_" ido-text-init) (setq ido-exit nil) (setq ido-rescan t) (setq
ido-rotate nil) (setq ido-text "") (when ido-set-default-item ... ...
... ... ...) (if ido-process-ignore-lists-inhibit ...) (if ... ...)
(cond ... ... ... ... ... ...) (setq ido-rotate-temp nil) (if
ido-process-ignore-lists-inhibit ...) (ido-set-matches) (if ... ...)
(let ... ... ...) (ido-trace "completing-read" ido-final-text) (if ...
...) (ido-trace "
_EXIT_" ido-exit) (cond ... ... ... ... ... ... ... ... ... ...)) ido-selected)
  ido-read-internal(list "Goto: " org-refile-history nil t nil)
  (let ((ido-current-directory nil) (ido-directory-nonreadable nil)
(ido-directory-too-big nil) (ido-context-switch-command ...)
(ido-choice-list choices)) (ido-read-internal (quote list) prompt hist
def require-match initial-input))

...

  (if (and org-completion-use-ido (fboundp ...) (boundp ...) ido-mode
(listp ...)) (apply (quote ido-completing-read) (concat ...) (cdr
args)) (apply (quote completing-read) args))

...

  (let* ((cbuf ...) (cfunc ...) (extra ...) (filename ...) (tbl ...)
(completion-ignore-case t)) (assoc (funcall cfunc prompt tbl nil t nil
...) tbl))
  org-refile-get-location("Goto: ")
  (nth 3 (org-refile-get-location "Goto: "))
  (if (eq interface (quote outline)) (car (org-get-location ...
org-goto-help)) (nth 3 (org-refile-get-location "Goto: ")))
  (let* ((org-refile-targets ...) (org-refile-use-outline-path t)
(interface ...) (org-goto-start-pos ...) (selected-point ...)) (if
selected-point (progn ... ... ...) (message "Quit")))
  org-goto(nil)
  call-interactively(org-goto)

Re: ido slow for outline path completion

[Carsten Dominik]

[-- Attachment #1.1: Type: text/plain, Size: 8337 bytes --]

Hi Samuel,

this is deep inside the ido routines, and I do not have enough
insight to see what is going on.  Maybe send a bug report to Kim?
Maybe you can create a test case for him by capturing the table
before it is passed to the ido-complete command?


If it is really the length of the strings, a uniquify-like solution
might be workable, but constructing the uniquify list may
be slow as well.....
I am putting this on my list, but not for the current year :-)

- Carsten

On Dec 19, 2008, at 11:32 PM, Samuel Wales wrote:

> *** ido
> Hi Carsten,
>
> Thanks for looking at the speed issues.
>
> This time an error showed up.  The error from the messages
> buffer is:
>
>  Getting targets...done
>  eq: Wrong type argument: arrayp, (#("footodo/...
>
> The trigger is:
>
> org-goto, same settings as before, searching for a headline called
> "parsing risk",
> which does exist but never shows up as the main candidate.
> I did load-file on ido.el and org.el for a better backtrace.
>
> I don't know the meanings of the different color patterns in
> ido, and this might be a hint for speed, or this error, or both.
>
> p --
>  shows an entry in body text face with a ^ with a red
>  background, rest of candidates are in red, fast.
> pa -- same but with different entry
> par -- same but with different entry
> pars -- only one candidate shows and it is in green, fast
> parsi --
>  back to the first color pattern, different entry, slightly
>  slower
> parsin -- same entry as parsi, slightly slower
> parsing --
>  same but with different entry, slower still (but only a
>  few seconds for me; maybe not noticeable on your computer)
> parsing  (space after g) --
>  causes error, space not inserted, fast
>
> I append the backtrace.  It isn't perfect.  I deleted a
> small number of lines that had the entire completion list in
> them (the whole backtrace is 700k) and had to minorly
> obfuscate a line for privacy, but did so in a consistent
> way.
>
> ido-auto-merge-delay-time might or might not be relevant.  I
> have it set to the default of 0.7.
>
> Thanks.
>
> =====
>
> The backtrace is:
>
>  aref((#("footodo/mumble/issues/is froz bar foo something risking/" 0
> 6 (face org-level-1 org-category "todo-foo" fontified t) 6 7 nil 7 13
> (face org-level-1 org-category #("mumble" 0 6 ...) fontified t) 13 14
> nil 14 20 (face org-level-1 org-category #("mumble" 0 6 ...) fontified
> t) 20 21 nil 21 64 (org-category #("mumble" 0 6 ...) fontified nil) 64
> 65 nil) "/Users/foo/Desktop/a/quux/todo-foo.org"
> "^\\*\\*\\*\\*\\*\\*\\* TODO is froz bar foo something risking[ 	]*$"
> 17576) 3)
>  (eq (aref s l) 47)
>  (and (> l 0) (eq (aref s l) 47))
>  (if (and (> l 0) (eq ... 47)) (substring s 0 l) s)
>  (let ((l ...)) (if (and ... ...) (substring s 0 l) s))
>  ido-no-final-slash((#("...
>  (string-lessp (ido-no-final-slash a) (ido-no-final-slash b))
>  ido-file-lessp((#("...
>  sort(((#("...
>  (let ((completion-list ...)) (if (featurep ...) (let ... ...)
> (display-completion-list completion-list)))
>  (with-output-to-temp-buffer ido-completion-buffer (let (...) (if ...
> ... ...)))
>  (if display-it (with-output-to-temp-buffer ido-completion-buffer
> (let ... ...)))
>  (let ((temp-buf ...) display-it full-list) (if (and ... temp-buf)
> (let ... ... ... ... ... ...) (setq display-it t)) (if display-it
> (with-output-to-temp-buffer ido-completion-buffer ...)))
>  ido-completion-help()
>  call-interactively(ido-completion-help)
>  (if (and (not ...) (not ...)) (progn (setq ido-rescan ...)
> (delete-region ... ...) (insert res)) (call-interactively (setq
> this-command ido-cannot-complete-command)))
>  (cond (ido-incomplete-regexp) ((and ... ...) (let ... ...))
> (ido-directory-too-big (setq ido-directory-too-big nil) (setq
> ido-text-init ido-text) (setq ido-exit ...) (exit-minibuffer)) ((not
> ido-matches) (when ido-completion-buffer ...)) ((and ... ... ...) (if
> ... ... ... ... ...)) (t (setq res ido-common-match-string) (if ...
> ... ...)))
>  (let (res) (cond (ido-incomplete-regexp) (... ...)
> (ido-directory-too-big ... ... ... ...) (... ...) (... ...) (t ...
> ...)))
>  ido-complete()
>  (if (and (stringp ido-common-match-string) (stringp ido-text) (cond
> ... ... ...)) (insert " ") (ido-complete))
>  ido-complete-space()
>  call-interactively(ido-complete-space)
>  completing-read("Goto: " (("dummy" . 1)) nil nil nil org-refile- 
> history)
>  (catch (quote ido) (completing-read (ido-make-prompt item prompt)
> (quote ...) nil nil (prog1 ido-text-init ...) history))
>  (setq ido-final-text (catch (quote ido) (completing-read ... ... nil
> nil ... history)))
>  (let ((minibuffer-local-completion-map ...)
> (minibuffer-local-filename-completion-map ...) (max-mini-window-height
> ...) (ido-completing-read t) (ido-require-match require-match)
> (ido-use-mycompletion-depth ...) (show-paren-mode nil)) (setq ido-exit
> nil) (setq ido-final-text (catch ... ...)))
>  (while (not done) (ido-trace "
> _LOOP_" ido-text-init) (setq ido-exit nil) (setq ido-rescan t) (setq
> ido-rotate nil) (setq ido-text "") (when ido-set-default-item (setq
> ido-default-item ...) (if ... ...) (ido-trace "new default"
> ido-default-item) (if ido-default-item ...) (setq ido-set-default-item
> nil)) (if ido-process-ignore-lists-inhibit (setq
> ido-process-ignore-lists nil)) (if (and ido-use-merged-list ... ...)
> (let ... ... ...)) (cond (ido-keep-item-list ...) (... ...) (... ...)
> (... ...) (... ...) (t nil)) (setq ido-rotate-temp nil) (if
> ido-process-ignore-lists-inhibit (setq ido-process-ignore-lists t
> ido-process-ignore-lists-inhibit nil)) (ido-set-matches) (if (and
> ido-matches ...) (setq ido-try-merged-list t)) (let (... ... ... ...
> ... ... ...) (setq ido-exit nil) (setq ido-final-text ...)) (ido-trace
> "completing-read" ido-final-text) (if (get-buffer
> ido-completion-buffer) (kill-buffer ido-completion-buffer)) (ido-trace
> "
> _EXIT_" ido-exit) (cond (... ... nil) (... ...) (... ... nil) (...
> ...) (... ...) (... ...) (... ... ...) (... ... ...) (... ...) (t ...
> ...)))
>  (let ((ido-cur-item item) (ido-entry-buffer ...)
> (ido-process-ignore-lists t) (ido-process-ignore-lists-inhibit nil)
> (ido-set-default-item t) ido-default-item ido-selected ido-final-text
> (done nil) (icomplete-mode nil) ido-cur-list ido-ignored-list
> (ido-rotate-temp nil) (ido-keep-item-list nil) (ido-use-merged-list
> nil) (ido-try-merged-list t) (ido-pre-merge-state nil) (ido-case-fold
> ido-case-fold) (ido-enable-prefix ido-enable-prefix)
> (ido-enable-regexp ido-enable-regexp)) (ido-setup-completion-map)
> (setq ido-text-init initial) (setq ido-input-stack nil) (run-hooks
> (quote ido-setup-hook)) (while (not done) (ido-trace "
> _LOOP_" ido-text-init) (setq ido-exit nil) (setq ido-rescan t) (setq
> ido-rotate nil) (setq ido-text "") (when ido-set-default-item ... ...
> ... ... ...) (if ido-process-ignore-lists-inhibit ...) (if ... ...)
> (cond ... ... ... ... ... ...) (setq ido-rotate-temp nil) (if
> ido-process-ignore-lists-inhibit ...) (ido-set-matches) (if ... ...)
> (let ... ... ...) (ido-trace "completing-read" ido-final-text) (if ...
> ...) (ido-trace "
> _EXIT_" ido-exit) (cond ... ... ... ... ... ... ... ... ... ...))  
> ido-selected)
>  ido-read-internal(list "Goto: " org-refile-history nil t nil)
>  (let ((ido-current-directory nil) (ido-directory-nonreadable nil)
> (ido-directory-too-big nil) (ido-context-switch-command ...)
> (ido-choice-list choices)) (ido-read-internal (quote list) prompt hist
> def require-match initial-input))
>
> ...
>
>  (if (and org-completion-use-ido (fboundp ...) (boundp ...) ido-mode
> (listp ...)) (apply (quote ido-completing-read) (concat ...) (cdr
> args)) (apply (quote completing-read) args))
>
> ...
>
>  (let* ((cbuf ...) (cfunc ...) (extra ...) (filename ...) (tbl ...)
> (completion-ignore-case t)) (assoc (funcall cfunc prompt tbl nil t nil
> ...) tbl))
>  org-refile-get-location("Goto: ")
>  (nth 3 (org-refile-get-location "Goto: "))
>  (if (eq interface (quote outline)) (car (org-get-location ...
> org-goto-help)) (nth 3 (org-refile-get-location "Goto: ")))
>  (let* ((org-refile-targets ...) (org-refile-use-outline-path t)
> (interface ...) (org-goto-start-pos ...) (selected-point ...)) (if
> selected-point (progn ... ... ...) (message "Quit")))
>  org-goto(nil)
>  call-interactively(org-goto)


[-- Attachment #1.2: Type: text/html, Size: 9283 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: ido slow for outline path completion

[Carsten Dominik]

[-- Attachment #1.1: Type: text/plain, Size: 2590 bytes --]


On Dec 19, 2008, at 12:57 AM, Samuel Wales wrote:

> Hi Carsten,
>
> On Mon, Dec 15, 2008 at 02:36, Carsten Dominik  
> <dominik@science.uva.nl> wrote:
>> 1. Compile Org!  Are you using byte-compiled files?
>>   In a case like this, the differences may be significant.
>
> Always!  :)
>
>> 4. Create the list of headings with full outline path
>>   in a single path of the file, similar to the tags matcher.
>>   This would avoid the N^2 operation I discussed above.
>
> Single pass?  Very nice.
>
>> There is also a new variable `org-goto-max-level' with
>> a default of 5.
>
> Excellent.  Might also be useful to unify the syntax for
> org-goto, org-refile, remember.


Hi Samuel,

could you expand a bit on what you meant here?

- Carsten

>
>
>> This should be *a lot* faster now.  Thanks for the report!
>
> Almost instantaneous in many cases.
>
> However, when I do org-refile as full path (not basenames),
> it sometimes takes 45 seconds.
>
> I haven't narrowed it down yet.  Emacs just freezes.
>
> Every character I type from that point on takes another 45
> seconds or so.  Deletes also take that long until I get back
> to the place where it was fast.  At that point it is fast
> again.
>
> The difference from before is that before, it occurred
> before the minibuffer showed the candidates, while now it
> occurs in the middle of entering the selection string.
>
> Discontiguity might something to do with it -- sometimes it
> happens when the character typed is different from the next
> char in the target.  Maybe the candidate list needs lots of
> searching in that case.  Maybe it is starting from the
> beginning to match against the path?
>
> I have not noticed this happening when I use basenames in
> org-refile ido.  This leads me to conjecture that it might
> be the size of the full paths.  Maybe too much stuff for the
> ido flex matcher to consider.  uniquify-style disambiguation
> might help by reducing the number of characters.
>
> It might or might not have something to do with typing a
> space that exists next in the target -- or not typing the
> same.  Another possible hint is that sometimes, typing a
> space inserts a slash.  Not sure why.
>
> As I said, I have not narrowed down the exact cases.  If you
> have a priori ideas, please let me know and I will try them.
>
> Thank you very much.
>
> -- 
> Myalgic encephalomyelitis denialists are knowingly causing further
> suffering and death by opposing biomedical research on this serious
> infectious disease.  Do you care about the world?
> http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm


[-- Attachment #1.2: Type: text/html, Size: 3562 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: ido slow for outline path completion

[Samuel Wales]

Hi Carsten,

I just meant that there are 3 path completion entry points
(remember, which can also use refile; refile; and goto), and
they are similar, but are specified differently
(e.g. maxlevel variable in goto and alist in refile).  So,
maybe there is a way to make the syntax for specifying them
consistent for new users.

Perhaps goto can use refile's path selection subsystem.
That allows basename flex matching (by turning off
org-refile-use-outline-path, useful for me as an interim workaround for ido
bugs) and other stuff like specifying the list of files to
search.

By the way, if you type "?" a few times using ido, it will
show a completions buffer.  But here's the interesting part:
that showed one path, which was "mytodo//" plus the full
path name of the file -- not the outline path.  M might
be another clue?

Re: ido slow for outline path completion

[Carsten Dominik]

On Dec 21, 2008, at 10:24 PM, Samuel Wales wrote:

> Hi Carsten,
>
> I just meant that there are 3 path completion entry points
> (remember, which can also use refile; refile; and goto), and
> they are similar, but are specified differently
> (e.g. maxlevel variable in goto and alist in refile).  So,
> maybe there is a way to make the syntax for specifying them
> consistent for new users.

> Perhaps goto can use refile's path selection subsystem.
> That allows basename flex matching (by turning off
> org-refile-use-outline-path, useful for me as an interim workaround  
> for ido
> bugs) and other stuff like specifying the list of files to
> search.

Hi Samuel,

org-goto is different because it only goes to the
current file.  Refile was originally meant to
go to a selected class of headings.  You can use the refile
command itself including its setup to *go* some place
by calling org-refile with a prefix argument.  A double prefix
will make the command jump directly to the location where
the last refile event delivered its content.

- Carsten

> By the way, if you type "?" a few times using ido, it will
> show a completions buffer.  But here's the interesting part:
> that showed one path, which was "mytodo//" plus the full
> path name of the file -- not the outline path.  M might
> be another clue?

I have no time to debug ido, I am afraid, unless you can
show that the table I pass to ido is invalid.

Re: ido slow for outline path completion

[Samuel Wales]

On Mon, Dec 22, 2008 at 01:30, Carsten Dominik <dominik@science.uva.nl> wrote:
>> By the way, if you type "?" a few times using ido, it will
>> show a completions buffer.  But here's the interesting part:
>> that showed one path, which was "mytodo//" plus the full
>> path name of the file -- not the outline path.  M might
>> be another clue?
>
> I have no time to debug ido, I am afraid, unless you can
> show that the table I pass to ido is invalid.

Very understandable :).

But I just meant that this might be a clue that would ring a bell with
you immediately as indicating that somehow files rather than olpaths
were involved -- in the call to ido.  Unlikely, I now realize.

In any case, I reported the bugs (except for the last one) to Kim
Storm and will post here if there is anything relevant.

-- 
Myalgic encephalomyelitis denialists are knowingly causing further
suffering and death by opposing biomedical research on this serious
infectious disease.  Do you care about the world?
http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm