Poor Performance w/ Long Files

Poor Performance w/ Long Files

[Tim Visher]

Hello Everyone,

I work with many large files (+10,000,000 ASCII characters) at work
and I've noticed that Emacs does extremely poorly with those files.  I
have taken to opening them up in something like Notepad++ to look at
them and editing the programs that operate on them in Emacs.  I'd love
to be able to just stick with Emacs.

Is there something that I just haven't set yet or does Emacs have
trouble with large files?

-- 

In Christ,

Timmy V.

http://burningones.com/
http://five.sentenc.es/ - Spend less time on e-mail

Re: Poor Performance w/ Long Files

[Eli Zaretskii]

> From: Tim Visher <tim.visher@gmail.com>
> Date: Wed, 19 Aug 2009 16:12:54 -0400
> 
> I work with many large files (+10,000,000 ASCII characters) at work
> and I've noticed that Emacs does extremely poorly with those files.  I
> have taken to opening them up in something like Notepad++ to look at
> them and editing the programs that operate on them in Emacs.  I'd love
> to be able to just stick with Emacs.
> 
> Is there something that I just haven't set yet or does Emacs have
> trouble with large files?

What file? is there a way to get hold of an example?

And what version of Emacs?

FWIW, I habitually need to edit 20-60MB files, and I didn't notice any
significant slowdown.  That was with Emacs 23.1.

Re: Poor Performance w/ Long Files

[Pascal J. Bourguignon]

Tim Visher <tim.visher@gmail.com> writes:

> Hello Everyone,
>
> I work with many large files (+10,000,000 ASCII characters) at work
> and I've noticed that Emacs does extremely poorly with those files.  I
> have taken to opening them up in something like Notepad++ to look at
> them and editing the programs that operate on them in Emacs.  I'd love
> to be able to just stick with Emacs.
>
> Is there something that I just haven't set yet or does Emacs have
> trouble with large files?

Perhaps you could disable some emacs options that might take time,
like font-locking.  Basically, if you edit this files in fundamental-mode,
with truncate-line turned off with C-u 1 M-x toggle-truncate-lines RET and
with font-locking turned off with C-u -1 M-x font-lock-mode RET,
it should go faster.

Otherwise, have a computer with a lot of RAM.  Arg!  You're
mentionning Notepad++, perhaps you are using MS-Windows?
Unfortunately, I know nothing about MS-Windows, much less about its
optimization...

-- 
__Pascal Bourguignon__

Re: Poor Performance w/ Long Files

[Lennart Borgman]

On Thu, Aug 20, 2009 at 2:09 AM, Pascal J.
Bourguignon<pjb@informatimago.com> wrote:
> Tim Visher <tim.visher@gmail.com> writes:
>
>> Hello Everyone,
>>
>> I work with many large files (+10,000,000 ASCII characters) at work
>> and I've noticed that Emacs does extremely poorly with those files.  I
>> have taken to opening them up in something like Notepad++ to look at
>> them and editing the programs that operate on them in Emacs.  I'd love
>> to be able to just stick with Emacs.
>>
>> Is there something that I just haven't set yet or does Emacs have
>> trouble with large files?
>
> Perhaps you could disable some emacs options that might take time,
> like font-locking.  Basically, if you edit this files in fundamental-mode,
> with truncate-line turned off with C-u 1 M-x toggle-truncate-lines RET and
> with font-locking turned off with C-u -1 M-x font-lock-mode RET,
> it should go faster.


I think we recently got a bug report where the user was using
fundamental-mode to edit a file with very long lines. That was very
slow.

If I remember correctly that depends on how the display engine in
Emacs is organized. It searches for line ends. If I understand Stefan
Monnier correctly this is something that can be cured and have a
rather high priority. So if someone want to jump in and have enough
knowledge to understand this it is probably welcome. (It might be a
difficult problem.)

Re: Poor Performance w/ Long Files

[Tim Visher]

On Wed, Aug 19, 2009 at 8:09 PM, Pascal J.
Bourguignon<pjb@informatimago.com> wrote:
> Tim Visher <tim.visher@gmail.com> writes:
>
>> Hello Everyone,
>>
>> I work with many large files (+10,000,000 ASCII characters) at work
>> and I've noticed that Emacs does extremely poorly with those files.  I
>> have taken to opening them up in something like Notepad++ to look at
>> them and editing the programs that operate on them in Emacs.  I'd love
>> to be able to just stick with Emacs.
>>
>> Is there something that I just haven't set yet or does Emacs have
>> trouble with large files?
>
> Perhaps you could disable some emacs options that might take time,
> like font-locking.  Basically, if you edit this files in fundamental-mode,
> with truncate-line turned off with C-u 1 M-x toggle-truncate-lines RET and
> with font-locking turned off with C-u -1 M-x font-lock-mode RET,
> it should go faster.

This fixed things.  I hadn't even thought of it but the file I was
opening tried to open in tcl and pabbrev modes and that's what seems
to have caused the slowdown.  Putting thing back into Fundamental with
pabbrev off quickened things right up.

Is there a way to manually force a file to open in a particular mode
without setting an auto-mode in .emacs?

On Wed, Aug 19, 2009 at 4:18 PM, Eli Zaretskii<eliz@gnu.org> wrote:
>> From: Tim Visher <tim.visher@gmail.com>
>> Date: Wed, 19 Aug 2009 16:12:54 -0400
>>
>> I work with many large files (+10,000,000 ASCII characters) at work
>> and I've noticed that Emacs does extremely poorly with those files.  I
>> have taken to opening them up in something like Notepad++ to look at
>> them and editing the programs that operate on them in Emacs.  I'd love
>> to be able to just stick with Emacs.
>>
>> Is there something that I just haven't set yet or does Emacs have
>> trouble with large files?
>
> What file? is there a way to get hold of an example?

Unfortunately I can't give a sample out.  However it doesn't seem to
be germane anymore anyway.

> And what version of Emacs?

22.3

Thanks everyone.

-- 

In Christ,

Timmy V.

http://burningones.com/
http://five.sentenc.es/ - Spend less time on e-mail

Re: Poor Performance w/ Long Files

[Pascal J. Bourguignon]

Tim Visher <tim.visher@gmail.com> writes:

> On Wed, Aug 19, 2009 at 8:09 PM, Pascal J.
> Bourguignon<pjb@informatimago.com> wrote:
>> Tim Visher <tim.visher@gmail.com> writes:
>>
>>> Hello Everyone,
>>>
>>> I work with many large files (+10,000,000 ASCII characters) at work
>>> and I've noticed that Emacs does extremely poorly with those files.  I
>>> have taken to opening them up in something like Notepad++ to look at
>>> them and editing the programs that operate on them in Emacs.  I'd love
>>> to be able to just stick with Emacs.
>>>
>>> Is there something that I just haven't set yet or does Emacs have
>>> trouble with large files?
>>
>> Perhaps you could disable some emacs options that might take time,
>> like font-locking.  Basically, if you edit this files in fundamental-mode,
>> with truncate-line turned off with C-u 1 M-x toggle-truncate-lines RET and
>> with font-locking turned off with C-u -1 M-x font-lock-mode RET,
>> it should go faster.
>
> This fixed things.  I hadn't even thought of it but the file I was
> opening tried to open in tcl and pabbrev modes and that's what seems
> to have caused the slowdown.  Putting thing back into Fundamental with
> pabbrev off quickened things right up.
>
> Is there a way to manually force a file to open in a particular mode
> without setting an auto-mode in .emacs?

M-x find-file-literally RET

It's written right down at the bottom of the help page for find-file ;-)

    To visit a file without any kind of conversion and without
    automatically choosing a major mode, use M-x find-file-literally.



-- 
__Pascal Bourguignon__

Re: Poor Performance w/ Long Files

[Tim Visher]

On Thu, Aug 20, 2009 at 9:50 AM, Pascal J.
Bourguignon<pjb@informatimago.com> wrote:
> Tim Visher <tim.visher@gmail.com> writes:
>
>> On Wed, Aug 19, 2009 at 8:09 PM, Pascal J.
>> Bourguignon<pjb@informatimago.com> wrote:
>>> Tim Visher <tim.visher@gmail.com> writes:
>>>
>>>> Hello Everyone,
>>>>
>>>> I work with many large files (+10,000,000 ASCII characters) at work
>>>> and I've noticed that Emacs does extremely poorly with those files.  I
>>>> have taken to opening them up in something like Notepad++ to look at
>>>> them and editing the programs that operate on them in Emacs.  I'd love
>>>> to be able to just stick with Emacs.
>>>>
>>>> Is there something that I just haven't set yet or does Emacs have
>>>> trouble with large files?
>>>
>>> Perhaps you could disable some emacs options that might take time,
>>> like font-locking.  Basically, if you edit this files in fundamental-mode,
>>> with truncate-line turned off with C-u 1 M-x toggle-truncate-lines RET and
>>> with font-locking turned off with C-u -1 M-x font-lock-mode RET,
>>> it should go faster.
>>
>> This fixed things.  I hadn't even thought of it but the file I was
>> opening tried to open in tcl and pabbrev modes and that's what seems
>> to have caused the slowdown.  Putting thing back into Fundamental with
>> pabbrev off quickened things right up.
>>
>> Is there a way to manually force a file to open in a particular mode
>> without setting an auto-mode in .emacs?
>
> M-x find-file-literally RET
>
> It's written right down at the bottom of the help page for find-file ;-)
>
>    To visit a file without any kind of conversion and without
>    automatically choosing a major mode, use M-x find-file-literally.

Thanks!  I should've checked the manual myself first. :)

-- 

In Christ,

Timmy V.

http://burningones.com/
http://five.sentenc.es/ - Spend less time on e-mail

Re: Poor Performance w/ Long Files

[Stefan Vollmar]

Dear Tim,

On 20.08.2009, at 18:24, Tim Visher wrote:

>> M-x find-file-literally RET


Emacs 23.1.1 (Aquamacs 2.0pr2)
I just tried that with a 300 MB file (admittedly, in this particular  
case more out of curiosity) and only got a "Maximum buffer size  
exceeded" message. Is there a way to load files of that size (300 MB  
does not seem daunting with modern desktops featuring 4 GB RAM), and I  
would be disappointed if my favourite editor insists on having all  
data in memory all the time - is that really the case?

Warm regards,
  Stefan
-- 
Dr. Stefan Vollmar, Dipl.-Phys.
Max-Planck-Institut für neurologische Forschung
Gleuelerstr. 50, 50931 Köln, Germany
Tel.: +49-221-4726-213  FAX +49-221-4726-298
Tel.: +49-221-478-5713  Mobile: 0160-93874279
Email: vollmar@nf.mpg.de   http://www.nf.mpg.de