Accelerating Emacs?

Accelerating Emacs?

[Herbert Euler]

Hello everyone,

Emacs is undoubtly the most powerful editor because of
its architecture, Elisp causes lots of problems solvable within
Emacs. But Emacs seems to be a bit slower for the same
reason.

I wrote a Lisp program to generate a huge amount of
random data for testing. This created a file whose size
was 192MB. Then I wanted to edit it and Emacs behaved
like a snail even when deleting a character. Finally I edited
the file with vim and it was not so slow as Emacs.

It might be because Lisp is an interpreting language,
and any operation in Emacs is firstly translated into calling
of Lisp functions and then executed. But common operations
in vim is written in C and linked in the binary executable
file, so vim is fast.

So perhaps I cannot get the power (as well as conveniency)
plus speed at the same time, in another word, it is not
possible to accelerate Emacs. Is this really true?

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: Accelerating Emacs?

[Eli Zaretskii]

> From: "Herbert Euler" <herberteuler@hotmail.com>
> Date: Fri, 28 Oct 2005 10:25:41 +0800
> 
> Emacs is undoubtly the most powerful editor because of
> its architecture, Elisp causes lots of problems solvable within
> Emacs. But Emacs seems to be a bit slower for the same
> reason.

Primitive operations that are used a lot and need to be fast are
written in C, not in Lisp.

FWIW, on a reasonably fast machine, I find Emacs reasonably fast for
day-to-day operation.

Re: Accelerating Emacs?

[Herbert Euler]

>From: Eli Zaretskii <eliz@gnu.org>
>To: help-gnu-emacs@gnu.org
>Subject: Re: Accelerating Emacs?
>Date: Fri, 28 Oct 2005 10:25:47 +0200
>
>Primitive operations that are used a lot and need to be fast are
>written in C, not in Lisp.

But at least the searching, syntax highlighting, tags processing,
and some other modules are written in Lisp, although it might
be byte-compiled.

>FWIW, on a reasonably fast machine, I find Emacs reasonably fast for
>day-to-day operation.

I believe this is true. But hey, lots of algorithms that are very effective
in general cases have to face some really bad situation, just like I
mentioned here. Is it possible to improve the behaviour of Emacs in
such situations?

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: Accelerating Emacs?

[Eli Zaretskii]

> From: "Herbert Euler" <herberteuler@hotmail.com>
> Date: Fri, 28 Oct 2005 19:38:26 +0800
> 
> >Primitive operations that are used a lot and need to be fast are
> >written in C, not in Lisp.
> 
> But at least the searching, syntax highlighting, tags processing,
> and some other modules are written in Lisp

They are written in Lisp, but they make heavy use of search and syntax
primitives that are written in C.

> >FWIW, on a reasonably fast machine, I find Emacs reasonably fast for
> >day-to-day operation.
> 
> I believe this is true. But hey, lots of algorithms that are very effective
> in general cases have to face some really bad situation, just like I
> mentioned here.

Sure, but making decisions about the tools you use based on such rare
bad situations is not a very wise thing to do, is it?

> Is it possible to improve the behaviour of Emacs in such situations?

It might be.  Feel free to report such cases as bugs, and the
developers will see if there's something that should be done.

Re: Accelerating Emacs?

[Christopher C. Stacy]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Herbert Euler" <herberteuler@hotmail.com>
>> Date: Fri, 28 Oct 2005 10:25:41 +0800
>> 
>> Emacs is undoubtly the most powerful editor because of
>> its architecture, Elisp causes lots of problems solvable within
>> Emacs. But Emacs seems to be a bit slower for the same
>> reason.
>
> Primitive operations that are used a lot and need to be
> fast are written in C, not in Lisp.

Just for the record: the above optimization strategy is an artifact 
of this Emacs-Lisp interpreter, not the Lisp language in general.

Re: Accelerating Emacs?

[Eli Zaretskii]

> From: cstacy@news.dtpq.com (Christopher C. Stacy)
> Date: Mon, 28 Nov 2005 01:09:36 GMT
> 
> > Primitive operations that are used a lot and need to be
> > fast are written in C, not in Lisp.
> 
> Just for the record: the above optimization strategy is an artifact 
> of this Emacs-Lisp interpreter, not the Lisp language in general.

Just for the record: this is gnu.emacs.help, so this Emacs-Lisp
interpreter is the only one that matters here.

Re: Accelerating Emacs?

[Pascal Bourguignon]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: cstacy@news.dtpq.com (Christopher C. Stacy)
>> Date: Mon, 28 Nov 2005 01:09:36 GMT
>> 
>> > Primitive operations that are used a lot and need to be
>> > fast are written in C, not in Lisp.
>> 
>> Just for the record: the above optimization strategy is an artifact 
>> of this Emacs-Lisp interpreter, not the Lisp language in general.
>
> Just for the record: this is gnu.emacs.help, so this Emacs-Lisp
> interpreter is the only one that matters here.

One could write an emacs-lisp compiler to native code and write even
the optimized parts in emacs-lisp.  The only trouble would be that
you'd need a compiler back-end for each platform.  Perhaps gcc should
include an emacs-lisp front-end?

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Kitty like plastic.
Confuses for litter box.
Don't leave tarp around.

Re: Accelerating Emacs?

[Thien-Thi Nguyen]

Pascal Bourguignon <spam@mouse-potato.com> writes:

> Perhaps gcc should
> include an emacs-lisp front-end?

probably someone is working on it.

thi

Re: Accelerating Emacs?

[Stefan Monnier]

> One could write an emacs-lisp compiler to native code and write even
> the optimized parts in emacs-lisp.

Problem is: with dynamic scoping and buffer-local variables, you're in for
a big disappointment if you think compiling to native code will give you
anywhere close to the performance of C code.  To get any kind of real
performance, you'll need to either switch to lexical scoping, or add
a sophisticated (whole program) analysis to figure out where lexical scope
can be used without affecting the end result.


        Stefan

Re: Accelerating Emacs?

[Flying Grass]

try fundamental-mode

Re: Accelerating Emacs?

[Herbert Euler]

>From: "Flying Grass" <wcy123@gmail.com>
>To: help-gnu-emacs@gnu.org
>Subject: Re: Accelerating Emacs?
>Date: 27 Oct 2005 19:59:30 -0700
>
>try fundamental-mode

I tried that, but it is only effective when making small
changes i.e. deleting a character. On my test on a
8612 KB size file, Emacs is not quick enough yet. The
sed command

    sed 's/[A-Z]/0/g' test

needs 17s to finish its job, and vim command

    :1,$:s/[A-Z]/0/g

needs 1min 7s, but even 20min is still not enough for
the Emacs command (in fundamental-mode)

    (replace-regexp "[A-Z]" "0" nil nil nil)

Besides, Emacs uses about 128MB memory. And when
I tried to mark all text, Emacs complains about the
memory.

So I think it is better of using tools such as sed and
vim to edit file bigger than 5MB instead of Emacs,
since Emacs is not good at this job.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: Accelerating Emacs?

[Eli Zaretskii]

> From: "Herbert Euler" <herberteuler@hotmail.com>
> Date: Fri, 28 Oct 2005 13:53:08 +0800
> 
> >try fundamental-mode
> 
> I tried that, but it is only effective when making small
> changes i.e. deleting a character.

Well, your original complaint _was_ about deleting a single
character.  Are you on the quest to prove at all costs that Emacs is
slow?

> On my test on a 8612 KB size file, Emacs is not quick enough
> yet. The sed command
> 
>     sed 's/[A-Z]/0/g' test
> 
> needs 17s to finish its job, and vim command
> 
>     :1,$:s/[A-Z]/0/g
> 
> needs 1min 7s, but even 20min is still not enough for
> the Emacs command (in fundamental-mode)
> 
>     (replace-regexp "[A-Z]" "0" nil nil nil)

What kind of machine do you have there?  I tried this on a 3MB file
(my email inbox), and it took less than 1 minute, even though I needed
to answer the question about discarding undo info several times during
that time.  This is on a 3GHz Pentium 4 running Windows XP.  I then
tried the same with a 19MB email box on a 700MHz Pentium III running
Debian GNU/Linux, and it took 13 minutes there (vim did it in 30
seconds).  Perhaps you should upgrade your hardware?

Anyway, `replace-regexp' does much more than just replace its first
argument with the second, and those other things make it run slower.
The doc string for `replace-regexp' says (note the last part,
especially):

    This function is usually the wrong thing to use in a Lisp program.
    What you probably want is a loop like this:
      (while (re-search-forward regexp nil t)
	(replace-match to-string nil nil))
    which will run faster and will not set the mark or print anything.

> Besides, Emacs uses about 128MB memory.

??? Not unless you visit many large files, it isn't.  My Emacs session
where I'm typing this runs for many days, has gobs of files and
buffers in it, and still uses only 22MB of memory.

> And when I tried to mark all text, Emacs complains about the memory.

How much memory do you have on that machine (and what OS is that)?
Also, please tell what command you used ``to mark all text'', and what
was the exact language of the Emacs complaint about memory.

> So I think it is better of using tools such as sed and
> vim to edit file bigger than 5MB instead of Emacs,
> since Emacs is not good at this job.

You are entitled to decide whatever you wish, but this would be not a
very wise decision, I think.  It is based on an unrealistic example
(perform unrealistic replacement in an unrealistic file).  I'd advise
against such a decision.

Re: Accelerating Emacs?

[Herbert Euler]

>From: Eli Zaretskii <eliz@gnu.org>
>To: help-gnu-emacs@gnu.org
>Subject: Re: Accelerating Emacs?
>Date: Fri, 28 Oct 2005 10:23:46 +0200

>Well, your original complaint _was_ about deleting a single
>character.  Are you on the quest to prove at all costs that Emacs is
>slow?

I am _not_ trying to prove something, I just want to know
whether it is possible to accelerate Emacs in some way,
especially in poor situation, and how.

>What kind of machine do you have there?  I tried this on a 3MB file
>(my email inbox), and it took less than 1 minute, even though I needed
>to answer the question about discarding undo info several times during
>that time.  This is on a 3GHz Pentium 4 running Windows XP.  I then
>tried the same with a 19MB email box on a 700MHz Pentium III running
>Debian GNU/Linux, and it took 13 minutes there (vim did it in 30
>seconds).  Perhaps you should upgrade your hardware?

The CPU of this machine is Pentium(R) 4 CPU 2.80 GHz, and there is
504 MB physical memory, running Windows XP and Emacs 21.3. Actually
Emacs is much more faster than vim in my first try, just replace the
beginning of each line by 4 spaces. Vim spent more than 1min finishing
this, and Emacs spent 7s only. I was so surprised while getting this
result, so I did the second, which discourages me.

Now I must show what I did. First, I wrote a Lisp program to generate
random data:

(let ((i 0))
  (while (< i 20000)
    (let ((j 0) s)
      (while (< j 10)
	(setq s (concat s (char-to-string (+ 50 (random 50))))
	      j (1+ j)))
      (insert s "\n"))
    (setq i (1+ i))))

This resulted in a 235KB size file. I named it 'test', and created a bigger
file with the following command sequence:

    $ cat test >>test1
    $ cat test1 >>test
    $ cat test >>test1
    $ cat test1 >>test
    ... ...

Finally I got a file with about 8MB. I then did what I described.

>Anyway, `replace-regexp' does much more than just replace its first
>argument with the second, and those other things make it run slower.
>The doc string for `replace-regexp' says (note the last part,
>especially):
>
>     This function is usually the wrong thing to use in a Lisp program.
>     What you probably want is a loop like this:
>       (while (re-search-forward regexp nil t)
>	(replace-match to-string nil nil))
>     which will run faster and will not set the mark or print anything.

I tried this, Task Manager said Emacs were using 229 636K memory
by the time I wrote this sentence. And it spent about 5min replacing
about half (more than a half) upper case letter in the generated file.

>??? Not unless you visit many large files, it isn't.  My Emacs session
>where I'm typing this runs for many days, has gobs of files and
>buffers in it, and still uses only 22MB of memory.

I found Emacs used more and more memory when generating random
data, so did when it replacing. These memory is released after Emacs
finishes its job. Is this because Emacs operating buffer residing in memory?

>How much memory do you have on that machine (and what OS is that)?
>Also, please tell what command you used ``to mark all text'', and what
>was the exact language of the Emacs complaint about memory.

This happens when I am testing a 100MB size file. I go to the beginning
of the file, press C-SPACE, then go to the end of the file, press M-w.
Emacs told me:

Warning: past 95% of memory limit
Killing some buffers may delay running out of memory.
However, certainly by the time you receive the 95% warning,
you should clean up, kill this Emacs, and start a new one.

If the place of pressing M-w is a bit backward, only the percentage
will be modified:

Warning: past 85% of memory limit
Killing some buffers may delay running out of memory.
However, certainly by the time you receive the 85% warning,
you should clean up, kill this Emacs, and start a new one.

And in some other case (I do not know what it is yet) it told me in the
Mini-buffer:

Memory exhausted--use M-x save-some-buffers RET

>You are entitled to decide whatever you wish, but this would be not a
>very wise decision, I think.  It is based on an unrealistic example
>(perform unrealistic replacement in an unrealistic file).  I'd advise
>against such a decision.

So perhaps I can restate it like this: if Emacs starts to be slow in
some condition, it is better of using some other tools instead of
it. And I find it very cool of running vim in Eshell.

Regards,
Guanpeng Xu

_________________________________________________________________
Is your PC infected? Get a FREE online computer virus scan from McAfee® 
Security. http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963

Re: Accelerating Emacs?

[Eli Zaretskii]

> From: "Herbert Euler" <herberteuler@hotmail.com>
> Date: Fri, 28 Oct 2005 19:31:35 +0800
> 
> >What kind of machine do you have there?  I tried this on a 3MB file
> >(my email inbox), and it took less than 1 minute, even though I needed
> >to answer the question about discarding undo info several times during
> >that time.  This is on a 3GHz Pentium 4 running Windows XP.  I then
> >tried the same with a 19MB email box on a 700MHz Pentium III running
> >Debian GNU/Linux, and it took 13 minutes there (vim did it in 30
> >seconds).  Perhaps you should upgrade your hardware?
> 
> The CPU of this machine is Pentium(R) 4 CPU 2.80 GHz, and there is
> 504 MB physical memory, running Windows XP and Emacs 21.3.

And on this machine, it took Emacs more than 20 minutes to do the
replacement in a 8.8 MB file?  That's amazingly slow, and contradicts
my testing (although I tested on a real file, not one with random
text)!

> Task Manager said Emacs were using 229 636K memory

Probably because the program that you used to generate the random file
conses a lot of strings.  Forcing garbage collection could have
returned some of the memory to the OS.

> I found Emacs used more and more memory when generating random
> data, so did when it replacing. These memory is released after Emacs
> finishes its job.

It is also released during garbage collection.

> This happens when I am testing a 100MB size file. I go to the beginning
> of the file, press C-SPACE, then go to the end of the file, press M-w.
> Emacs told me:
> 
> Warning: past 95% of memory limit

Well, M-w causes Emacs to copy the entire 100MB region into the
Windows clipboard.  To do that, Emacs allocates a temporary buffer
copies those 100MB into the buffer, then encodes the text in the
temporary buffer as appropriate for the Windows clipboard.  As your
system has only 512MB of installed physical memory, using up 100MB
could easily exhaust what your machine has.

If you really need to copy/paste such large regions, you should
disable the automatic copying into the clipboard.

> So perhaps I can restate it like this: if Emacs starts to be slow in
> some condition, it is better of using some other tools instead of
> it.

That is always true.  As Per wrote elsewhere in this thread, Sed is
ideally suited for the substitutions that you tried, especially on
very large files.

Re: Accelerating Emacs?

[Per Abrahamsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Herbert Euler" <herberteuler@hotmail.com>
>> Date: Fri, 28 Oct 2005 13:53:08 +0800
>> 
>> Besides, Emacs uses about 128MB memory.
>
> ??? Not unless you visit many large files, it isn't.  My Emacs session
> where I'm typing this runs for many days, has gobs of files and
> buffers in it, and still uses only 22MB of memory.

He started by writing that he created a 192MB file of random data,
which means Emacs uses at least that amount.  

If that is typical of the kind of data Herbert work with, he may very
well be best served by using another tool.  'sed' is actually
optimized for the kind of task he have used as examples, so that would
be a good choice.

Re: Accelerating Emacs?

[Alan Mackenzie]

Herbert Euler <herberteuler@hotmail.com> wrote on Fri, 28 Oct 2005
13:53:08 +0800:
>>From: "Flying Grass" <wcy123@gmail.com> To: help-gnu-emacs@gnu.org
>>Subject: Re: Accelerating Emacs?  Date: 27 Oct 2005 19:59:30 -0700

>>try fundamental-mode

> I tried that, but it is only effective when making small changes i.e.
> deleting a character. On my test on a 8612 KB size file, Emacs is not
> quick enough yet. The sed command

>     sed 's/[A-Z]/0/g' test

> needs 17s to finish its job, and vim command

>     :1,$:s/[A-Z]/0/g

> needs 1min 7s, but even 20min is still not enough for the Emacs command
> (in fundamental-mode)

>     (replace-regexp "[A-Z]" "0" nil nil nil)

Why are you using this lisp form rather than using the normal Emacs
replacement command C-M-% (query-replace-regexp)?  Could you try this
command in your large file, entering the arguments as follows:

C-M-%  [A-Z] <ret> 0 <ret>  (Use M-x query-replace-regexp if you haven't
got the keybinding.)

, then start your stop watch as you type "!" (which means replace all
further occurances).


The lisp form you quoted merely replaces one occurrence, so presumably
you embedded this in a loop to do the timing.  The doc-string for
replace-regexp actually recommends against doing this, suggesting a
better way, as follows:

    This function [replace-regexp] is usually the wrong thing to use in
    a Lisp program.  What you probably want is a loop like this:
        (while (re-search-forward REGEXP nil t)
          (replace-match TO-STRING nil nil))
    which will run faster and will not set the mark or print anything.

Could you perhaps time this on your 8612 KB file?  I doubt it'll be quite
as fast as vim's 67 seconds, but I'd be surprised if it takes anything
like 20 minutes.


> Besides, Emacs uses about 128MB memory. And when I tried to mark all
> text, Emacs complains about the memory.

What are you doing to have Emacs use 128MB memory?  I frequently run two
copies of Emacs simultaneously (occasionally even 3), and I've only got
64MB of RAM.  :-)

> So I think it is better of using tools such as sed and vim to edit file
> bigger than 5MB instead of Emacs, since Emacs is not good at this job.

You're surely right here, up to a point.  If by "editing" you mean simple
mechanical changes repeated a vast number of times, Emacs is definitely
not the right tool.  This is what sed is for, as you know.  Emacs is
optimised for interactive editing of human-readable text files, which
tend to be smaller than 1MB.  For tasks in between (say, extracting
information from log files), you'd want something like an AWK, Perl, or
Python script.

But I think the nature of the task rather than the size of the file is
the prime criterion for deciding on the tool.  I sometimes massage data
in ~40 Mb files (for example leafnode (the newsreader)'s .overview files)
and find Emacs fast enough (just a little sluggish), even on my 166 MHz
PC.

> Regards,
> Guanpeng Xu

-- 
Alan Mackenzie (Munich, Germany)
Email: aacm@muuc.dee; to decode, wherever there is a repeated letter
(like "aa"), remove half of them (leaving, say, "a").

Re: Accelerating Emacs?

[Herbert Euler]

And are there some configuration steps before using Emacs
in Windows? I typed

    M-x texinfo-format-buffer

in the buffer associated with 'emacs-lisp-intro.texi'. On the same
machine, Emacs in Windows didn't finish it in 10min, while Emacs
in Debian finished it very quickly in no more than 1min. Am I having
some config problems?

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: Accelerating Emacs?

[Eli Zaretskii]

> From: "Herbert Euler" <herberteuler@hotmail.com>
> Date: Sat, 29 Oct 2005 02:28:30 +0800
> 
> And are there some configuration steps before using Emacs
> in Windows?

Not as far as performance is concerned, no (to the best of my
knowledge).

> I typed
> 
>     M-x texinfo-format-buffer
> 
> in the buffer associated with 'emacs-lisp-intro.texi'. On the same
> machine, Emacs in Windows didn't finish it in 10min, while Emacs
> in Debian finished it very quickly in no more than 1min.

That's very strange.  On my 3GHz box, the above takes 6 seconds in the
Windows port of the CVS Emacs.  What version of Emacs are you using
(what does "M-x emacs-version RET" display)?

> Am I having some config problems?

There's a problem, allright, but I'm unsure what kind, exactly.  When
you invoke "M-x texinfo-format-buffer", do you see in the echo area
messages like the following:

    Formatting: Passing the argument ... 
    Formatting: Passing a negative argument ... 
    Formatting: @code{yank-pop} ... 
    Formatting: The @file{ring.el} File ... 
    Formatting: A Graph with Labelled Axes ... 
    Formatting: Labelled Example Graph ... 
    Formatting: The @code{print-graph} Varlist ... 
    Formatting: The @code{print-Y-axis} Function ... 
    Formatting: What height should the label be? ... 
    Formatting: Side Trip: Compute a Remainder ... 
    Formatting: Construct a Y Axis Element ... 
    Formatting: Create a Y Axis Column ... 
    Formatting: The Not Quite Final Version of @code{print-Y-axis} ... 
    Formatting: The @code{print-X-axis} Function ... 
    Formatting: Similarities and differences ... 

Re: Accelerating Emacs?

[Herbert Euler]

>From: Eli Zaretskii <eliz@gnu.org>
>To: help-gnu-emacs@gnu.org
>Subject: Re: Accelerating Emacs?
>Date: Sat, 29 Oct 2005 12:18:27 +0200
>
>That's very strange.  On my 3GHz box, the above takes 6 seconds in the
>Windows port of the CVS Emacs.  What version of Emacs are you using
>(what does "M-x emacs-version RET" display)?

It's 21.3 in Windows and 21.4 in Debian.

>There's a problem, allright, but I'm unsure what kind, exactly.  When
>you invoke "M-x texinfo-format-buffer", do you see in the echo area
>messages like the following:
>
>     Formatting: Passing the argument ...
>     Formatting: Passing a negative argument ...
>     Formatting: @code{yank-pop} ...
>     Formatting: The @file{ring.el} File ...
>     Formatting: A Graph with Labelled Axes ...
>     Formatting: Labelled Example Graph ...
>     Formatting: The @code{print-graph} Varlist ...
>     Formatting: The @code{print-Y-axis} Function ...
>     Formatting: What height should the label be? ...
>     Formatting: Side Trip: Compute a Remainder ...
>     Formatting: Construct a Y Axis Element ...
>     Formatting: Create a Y Axis Column ...
>     Formatting: The Not Quite Final Version of @code{print-Y-axis} ...
>     Formatting: The @code{print-X-axis} Function ...
>     Formatting: Similarities and differences ...

Also strange to me now. I tried that for more times, and sometimes
Emacs is so slow that I'm tired and then typed C-g to abort the generating
process, while it's OK in other trials. The job is half-finished when I
interrupt it. The signal of taking long time is Emacs prints one of the
messages listed above and stops there from continuing printing. If that
happens, the only way seems to be press C-g to abort it.

Is it because of some bugs in garbage collection? I don't know how to find
out what the problem is, so can't provide any constructive information.

Regards,
Guanpeng Xu

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

Re: Accelerating Emacs?

[Eli Zaretskii]

> From: "Herbert Euler" <herberteuler@hotmail.com>
> Date: Sat, 29 Oct 2005 19:44:56 +0800
> 
> Also strange to me now. I tried that for more times, and sometimes
> Emacs is so slow that I'm tired and then typed C-g to abort the generating
> process, while it's OK in other trials. The job is half-finished when I
> interrupt it. The signal of taking long time is Emacs prints one of the
> messages listed above and stops there from continuing printing. If that
> happens, the only way seems to be press C-g to abort it.
> 
> Is it because of some bugs in garbage collection? I don't know how to find
> out what the problem is, so can't provide any constructive information.

You can enable garbage-collect messages by setting the variable
garbage-collection-messages to non-nil.  Then you will see a message
in the echo area when Emacs is in GC.

Other things to consider include:

  . Perhaps your system is running out of free RAM and Emacs starts
    paging?  You can use the Windows Task Manager to see how much
    physical memory is used, how many page faults Emacs generates, and
    other pertinent info.

  . How many other programs are using up CPU cycles?  The Task Manager
    will help you see that as well.

  . When Emacs freezes, do other programs on this system become
    unresponsive as well, or do they work normally?  If the former,
    perhaps some network-related problem ties up the system in a
    busy-wait loop.

  . Does Emacs run any subprocesses concurrently with
    texinfo-format-buffer?  If so, perhaps it's one of those
    subprocesses that causes Emacs to hang.

Re: Accelerating Emacs?

[Herbert Euler]

>From: Eli Zaretskii <eliz@gnu.org>
>To: help-gnu-emacs@gnu.org
>Subject: Re: Accelerating Emacs?
>Date: Sat, 29 Oct 2005 17:43:29 +0200

I'm sorry I was a bit busy yesterday.

The following description is based on my Emacs 21.3 in Windows.

>You can enable garbage-collect messages by setting the variable
>garbage-collection-messages to non-nil.  Then you will see a message
>in the echo area when Emacs is in GC.

I turned it on, and when Emacs hangs the two messages

    "Garbage collecting..."

and

    "???? (a long word I can't see it clearly) @code{print-elements-of-list} 
..."

are displayed alternately in the echo area. But I can only conclude
that the exchanging speed of displaying is slower and slower by time.
Task manager shows that Emacs uses about 80MB memory and
the number increases even if I abort the formatting process, although
the speed is becoming slow. A few seconds later, this number stops
increasing, but doesn't fall down as well. Emacs occupies 50% of
CPU cycle and this percentage is not changed during these periods.
C-x C-b shows no buffer contains further information about the details
of garbage collection. It seems that there is a died repetition
somewhere although I can't address it.

>Other things to consider include:
>
>   . Perhaps your system is running out of free RAM and Emacs starts
>     paging?  You can use the Windows Task Manager to see how much
>     physical memory is used, how many page faults Emacs generates, and
>     other pertinent info.

The bottom line of Task Manager contains a string like '362M / 1230M'.
The information about Emacs is

Name            User             CPU            Memory
emacs.exe      xuguanpeng        50          82,520K

I don't know what these mean exactly, but I'm sure the physical memory
is not used out.

>   . How many other programs are using up CPU cycles?  The Task Manager
>     will help you see that as well.

The total of CPU cycles used by other programs (except System Idle
Process) is about 10%.

>   . When Emacs freezes, do other programs on this system become
>     unresponsive as well, or do they work normally?  If the former,
>     perhaps some network-related problem ties up the system in a
>     busy-wait loop.

And I'm sure there is nothing wrong with network since I can open url
with Opera.

>   . Does Emacs run any subprocesses concurrently with
>     texinfo-format-buffer?  If so, perhaps it's one of those
>     subprocesses that causes Emacs to hang.

No, I ran only M-x texinfo-format-buffer in a fresh Emacs process.

I wonder if I can get the details about garbage collection so I can figure
out how much Emacs goes through at least.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: Accelerating Emacs?

[David Kastrup]

"Herbert Euler" <herberteuler@hotmail.com> writes:

>>From: Eli Zaretskii <eliz@gnu.org>
>>To: help-gnu-emacs@gnu.org
>>Subject: Re: Accelerating Emacs?
>>Date: Fri, 28 Oct 2005 10:23:46 +0200
>
> Now I must show what I did. First, I wrote a Lisp program to generate
> random data:
>
> (let ((i 0))
>   (while (< i 20000)
>     (let ((j 0) s)
>       (while (< j 10)
> 	(setq s (concat s (char-to-string (+ 50 (random 50))))
> 	      j (1+ j)))
>       (insert s "\n"))
>     (setq i (1+ i))))

This is not good in any programming language, as it has quadratic time
behavior.

_Buffers_ are the data structures for inserting text, not strings.  So
you better write:

(dotimes (i 20000)
  (dotimes (j 10)
    (insert-char (+ 50 (random 50))))
  (insert-char ?\C-j))

> I found Emacs used more and more memory when generating random data,
> so did when it replacing. These memory is released after Emacs
> finishes its job. Is this because Emacs operating buffer residing in
> memory?

No, it is because of nonsensical accumulation of strings which are
only garbage-collected from time to time.

> This happens when I am testing a 100MB size file. I go to the beginning
> of the file, press C-SPACE, then go to the end of the file, press M-w.

Don't do that, then.  Use delete-region instead of kill-region, or the
region will end up in the kill ring, where it still occupies memory.
And you might want to disable the undo history as well.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Accelerating Emacs?

[Herbert Euler]

Suddenly I've got a perfect idea of improving speed in this
condition. I wrote another Lisp program:

(call-process-region
(point-min) (point-max)
"sed"
t
(current-buffer)
nil
"s/[0-0]/0/g")

So I can pick up both the speed of sed and the power of Emacs.
I will figure out whether it's possible to generalize this method.

Regards,
Guanpeng Xu

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: Accelerating Emacs?

[Thien-Thi Nguyen]

"Herbert Euler" <herberteuler@hotmail.com> writes:

> (let ((i 0))
>   (while (< i 20000)
>     (let ((j 0) s)
>       (while (< j 10)
> 	(setq s (concat s (char-to-string (+ 50 (random 50))))
> 	      j (1+ j)))
>       (insert s "\n"))
>     (setq i (1+ i))))

using repeated `concat' followed by `insert' generates a lot of garbage
to be collected (primarily temporary strings).  here's a variation that
goes faster (how much faster depends on many factors!):

(funcall
 (byte-compile
  (lambda ()
    (insert (with-temp-buffer
              (set-buffer-multibyte nil)
              (let ((s (concat (make-string 10 0) "\n")))
                (dotimes (i 20000)
                  (dotimes (j 10)
                    (aset s j (+ 50 (random 50))))
                  (insert s)))
              (buffer-string))))))

it may be unsuitable for other data domains, but that's by design...

> So perhaps I can restate it like this: if Emacs starts to be slow in
> some condition, it is better of using some other tools instead of
> it. And I find it very cool of running vim in Eshell.

the best tool is your brain, which is nice because it is portable, and
subject to "instead" only by its own admission.  if you can use it to
exploit other tools (including emacs) in various configurations, what
more is there to ask?

thi

Re: Accelerating Emacs?

[Stefan Monnier]

> The CPU of this machine is Pentium(R) 4 CPU 2.80 GHz, and there is
> 504 MB physical memory, running Windows XP and Emacs 21.3.

That sounds quite reasonable.

> I tried this, Task Manager said Emacs were using 229 636K memory
> by the time I wrote this sentence.

My guess is that most of that memory is in the undo-log.  The undo-log works
well in general, but there are indeed "corner cases" where it ends up using
a lot of memory.

> I found Emacs used more and more memory when generating random data, so
> did when it replacing.  These memory is released after Emacs finishes its
> job.  Is this because Emacs operating buffer residing in memory?

No.  E.g. the undo-log cannot be shrunk before some undo-boundary is
inserted, which only happens later.  You could report your problem via
M-x report-emacs-bug.

> This happens when I am testing a 100MB size file. I go to the beginning
> of the file, press C-SPACE, then go to the end of the file, press M-w.

M-w copies the marked region, so you end up with a 100MB buffer plus
a 100MB string.  100MB thingies is definitely pushing it in Emacs nowadays.
On 64bit machines it may work better, but it's still likely that using Emacs
to edit 100MB files won't work well unless you're careful to use a special
"stripped-down" mode.

> So perhaps I can restate it like this: if Emacs starts to be slow in
> some condition, it is better of using some other tools instead of
> it. And I find it very cool of running vim in Eshell.

Sounds right.  I think it's good practice to complain about those situations
(via M-x report-emacs-bug) so that Emacs maintainers keep it in mind, but
it's clear that Emacs is not designed to edit very large files.  I tend to
use `sed' for that in most cases.


        Stefan