`start-process' awfully slow

`start-process' awfully slow

[York Zhao]

Hi Everyone,

Clang completion is awesome for completion in C++. I'm using
auto-complete-clang.el, but changed a lot to suit my personal favour, of course.
The biggest problem though is the speed, it is painfully slow for completion in
C++ code that uses std library. On Intel i7 16G RAM, Windows 7 it takes 2
seconds for 15,000 lines of completion results to transfer into emacs buffer.
Here is the code snippet:

(defun ac-clang-candidate ()

  ...

  (apply (if ac-clang-asynchronous
             'ac-clang-start-process
           'ac-clang-call-process)
         ac-prefix
         (ac-clang-build-complete-args (- (point) (length ac-prefix)))))

(defsubst ac-clang-build-complete-args (pos)
  (append '("-cc1" "-fsyntax-only")
          (unless ac-clang-auto-save
            (list "-x" (ac-clang-lang-option)))
          ac-clang-flags
          (when (stringp ac-clang-prefix-header)
            (list "-include-pch" (expand-file-name ac-clang-prefix-header)))
          '("-code-completion-at")
          (list (ac-clang-build-location pos))
          (list (if ac-clang-auto-save buffer-file-name "-"))))

So, when `ac-clang-asynchronous' is NOT set, eventually `call-process' will be
called to call Clang executable (clang.exe). Otherwise, `start-process' will
eventually be called. As mentioned above, when `call-process' is used, it takes
2 seconds for Clang to generate 15,000 lines of completion results and then the
results gets received by Emacs in a temporary buffer. 2 seconds is painful but
still OK. Because of the slowness, I tried to add the functionality to be able
to call Clang asynchronously so that the keyboard input will not be blocked
while awaiting the completion results. What had driven me crazy is that it takes
more than 15 seconds for Emacs to receive the 15,000 lines of completion results
if `start-process' is is being used to call the Clang executable. During this 15
seconds I was not typing anything so Emacs is "idling". The question is why
`start-process' takes 15 seconds while `call-process' takes only 2 seconds, and
what can I do about it?

Any help will be greatly appreciated.

Thanks in advance

York

Re: `start-process' awfully slow

[Stefan Monnier]

> As mentioned above, when `call-process' is used, it takes 2 seconds
> for Clang to generate 15,000 lines of completion results and then the
> results gets received by Emacs in a temporary buffer. 2 seconds is
> painful but still OK.

Which part of those 2 seconds is due to clang, and which part to Emacs?

> Because of the slowness, I tried to add the functionality to be able
> to call Clang asynchronously so that the keyboard input will not be
> blocked while awaiting the completion results. What had driven me
> crazy is that it takes more than 15 seconds for Emacs to receive the
> 15,000 lines of completion results if `start-process' is is being used
> to call the Clang executable. During this 15 seconds I was not typing
> anything so Emacs is "idling". The question is why `start-process'
> takes 15 seconds while `call-process' takes only 2 seconds, and what
> can I do about it?

There's no doubt that start-process has to work harder than
call-process, but 13s to transfer 15K lines (I assume those lines aren't
terribly long, so we might be talking about less than 1MB of data) is
too long.  I suggest you report it as a bug (via M-x report-emacs-bug),
trying to provide as much info as possible to make it reproducible
(e.g. replacing clang with a "cat" that simply outputs those 15K lines).


        Stefan

Re: `start-process' awfully slow

[York Zhao]

>> As mentioned above, when `call-process' is used, it takes 2 seconds
>> for Clang to generate 15,000 lines of completion results and then the
>> results gets received by Emacs in a temporary buffer. 2 seconds is
>> painful but still OK.
>
> Which part of those 2 seconds is due to clang, and which part to Emacs?

I don't know exactly the proportion of time being consumed, but I think most of
the 2 seconds are consumed by the clang program. Is there an easy way to know
exactly how long haven been consumed by Emacs to receive the output from clang?

>> Because of the slowness, I tried to add the functionality to be able
>> to call Clang asynchronously so that the keyboard input will not be
>> blocked while awaiting the completion results. What had driven me
>> crazy is that it takes more than 15 seconds for Emacs to receive the
>> 15,000 lines of completion results if `start-process' is is being used
>> to call the Clang executable. During this 15 seconds I was not typing
>> anything so Emacs is "idling". The question is why `start-process'
>> takes 15 seconds while `call-process' takes only 2 seconds, and what
>> can I do about it?

> I assume those lines aren't terribly long.

The longest line had exceeded 200, I noticed this because there had been a time
I had to set `line-number-display-limit-width' to over 200 (default being 200)
in order for `column-number-mode' to be able to display the column number if the
maximum length of lines in a buffer exceeds 200.

> so we might be talking about less than 1MB of data

The size is actually 1.4MB.

> I suggest you report it as a bug (via M-x report-emacs-bug), trying to provide
> as much info as possible to make it reproducible (e.g. replacing clang with a
> "cat" that simply outputs those 15K lines).

Replacing "clang" with "cat" is a good idea to reproduce the problem. I might
have to report the bug, thanks for your suggestion.

Thanks,

York

On Sun, Apr 14, 2013 at 2:24 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> As mentioned above, when `call-process' is used, it takes 2 seconds
>> for Clang to generate 15,000 lines of completion results and then the
>> results gets received by Emacs in a temporary buffer. 2 seconds is
>> painful but still OK.
>
> Which part of those 2 seconds is due to clang, and which part to Emacs?
>
>> Because of the slowness, I tried to add the functionality to be able
>> to call Clang asynchronously so that the keyboard input will not be
>> blocked while awaiting the completion results. What had driven me
>> crazy is that it takes more than 15 seconds for Emacs to receive the
>> 15,000 lines of completion results if `start-process' is is being used
>> to call the Clang executable. During this 15 seconds I was not typing
>> anything so Emacs is "idling". The question is why `start-process'
>> takes 15 seconds while `call-process' takes only 2 seconds, and what
>> can I do about it?
>
> There's no doubt that start-process has to work harder than
> call-process, but 13s to transfer 15K lines (I assume those lines aren't
> terribly long, so we might be talking about less than 1MB of data) is
> too long.  I suggest you report it as a bug (via M-x report-emacs-bug),
> trying to provide as much info as possible to make it reproducible
> (e.g. replacing clang with a "cat" that simply outputs those 15K lines).
>
>
>         Stefan