Large files

Large files

[Rathnaprabhu Rajendran]

Emacs 21.3.1 did not open a 150Mb text file in windows XP. Is there
are way to make emacs open larger files ?
Any help would be greatly appreciated.
Thanks in advance
Prabhu

Re: Large files

[Stefan Monnier]

> Emacs 21.3.1 did not open a 150Mb text file in windows XP. Is there
> are way to make emacs open larger files ?

On 32bit systems, the maximum file size in Emacs-21.3 is 128MB.
In Emacs-CVS, it's been pushed to 256MB.
It can be fairly easily be pushed further to 512MB, tho the corresponding
patch is not in Emacs-CVS.

If that's not good enough:
1 - use a 64bit system (with an Emacs compiled accordingly).
2 - split your file into smaller chunks.
3 - use XEmacs whose max is 1GB.


        Stefan

Re: Large files

[Carsten Weinberg]

Stefan Monnier wrote:

>On 32bit systems, the maximum file size in Emacs-21.3 is 128MB.
>  
>
how about disk file editing?

Windows Ultraedit, and SlickEdit, a emacs derivate, can do that!

Re: Large files

[Eli Zaretskii]

> Date: Tue, 11 Jan 2005 20:22:44 +0100
> From: Carsten Weinberg <cweinberg@freenet.de>
> 
> how about disk file editing?

What do you mean by that, exactly?  When I visit a file in Emacs, I'm
editing a disk file, don't I?

Re: Large files

[cweinberg]

Hi Eli,

"Disk based text editing - supports files in excess of 4GB, minimum RAM
used even for multi-megabyte files" (from Ultraedit feature list)

UE doesn't read a file as an whole into an memory buffer, only
parts. When scrolling down to a certain degree, UE reads the next
chunk from disk, and so on. Therefore the 4GB without 4GB memory.

-- 
Regards
-Carsten

Re: Large files

[Peter Dyballa]

Am 11.01.2005 um 20:22 schrieb Carsten Weinberg:

> how about disk file editing?

IMO this is a so-called "feature" that an underprivileged OS might 
need. When you don't have tools as on UNIX or Linux that can deal 
efficiently with data you might develop the idea of reading some GB 
from disk to retrieve a few bytes. Since mature file systems have a 
journal it's not recommended to bypass this system and "edit" inodes or 
the journal on disk directly.

Besides, isn't there already tar-mode? Doesn't it allow to manipulate 
an even compressed set of archived files? If you feel better with that 
power at your finger prints ...


Sometimes I do have to deal with enormous archives, wanting to 'look 
inside' before doing anything. I get often asked then whether I really 
want to do it -- and than sometimes it fails to open and view the 
archive because of limitations. OK! From the command line I can have a 
look into too, not that comfortable, but at least with the help of UNIX 
I am a bit as flexible as Flexi Girl ...

--
Greetings

   Pete

If my theory of relativity is proven successful, Germany will claim me
as a German, and France will declare that I am a citizen of the world.
Should my theory prove untrue, France will say that I am a German, and
Germany will declare that I am a Jew.          --Albert Einstein, 1929

Re: Large files

[Pascal Bourguignon]

Carsten Weinberg <cweinberg@freenet.de> writes:

> Stefan Monnier wrote:
> 
> >On 32bit systems, the maximum file size in Emacs-21.3 is 128MB.
> >
> how about disk file editing?

emacs is not a disk file editor.  It's a text editor.
 
> Windows Ultraedit, and SlickEdit, a emacs derivate, can do that!

Good for it.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Wanna go outside.
Oh, no! Help! I got outside!
Let me back inside!

Re: Large files

[Stefan Monnier]

>> On 32bit systems, the maximum file size in Emacs-21.3 is 128MB.
> how about disk file editing?

Emacs does not know how to do that.
Someone could try to write an elisp package that does it somewhat
transparently, but clearly, the needed hasn't ben strong enough for anyone
to sit down and actually do it.


        Stefan

Re: Large files

[cweinberg]

All the professional text editors known to me are disk oriented. I
always wondered that emacs isn't.

Anyway, emacs can't, and today in days large memory systems are less
expensive, disk oriented editing appears to be less important - but
only at the first glance, because modern emacs features like
auto-revert-tail mode, making viewing (not editing) logfiles easy,
which really can be extremely large. Or loading a large number of java
classes may also cause a memory consumption which is significant
larger than that of Windows text editors like UE.

Therefore I support Stefans proposal that someone could try to write
an elisp package that does disk oriented editing.

-- 
Regards
-Carsten

Re: Large files

[Stefan Monnier]

> All the professional text editors known to me are disk oriented.

I'm not sure what the "professional" above refers to.

> I always wondered that Emacs isn't.

Probably because it never seemed necessary.

Why would anybody edit enormously large files?  Such large files are not
written by hand (otherwise, they'd be smaller), so you'd only want to edit
them by hand in some rare circumstances.
Such circumstances are sufficiently particular that people are happy using
some other solution instead.

> Anyway, Emacs can't, and today in days large memory systems are less
> expensive, disk oriented editing appears to be less important - but
> only at the first glance, because modern Emacs features like
> auto-revert-tail mode, making viewing (not editing) logfiles easy,
> which really can be extremely large.

`tail -f' in a shell buffer works as well, and with comint-truncate-buffer,
it'll work even if your logfile grows to several GB.

> Or loading a large number of java classes may also cause a memory
> consumption which is significant larger than that of Windows text editors
> like UE.

It may but I've never seen people complain about it, so it looks like either
they don't suffer from it, or they silently switch to something else.
Emacs is not famous for being frugal, but Eclipse seem to "eclipse" even
Emacs on this front and that doesn't seem to prevent it from being used for
large Java projects.

> Therefore I support Stefan's proposal that someone could try to write
> an elisp package that does disk oriented editing.

Supporting it won't help.  You need to go and write it.


        Stefan