"maximum buffer size exceeded" in 64-bit emacs 22.1.1 (64G RAM)

"maximum buffer size exceeded" in 64-bit emacs 22.1.1 (64G RAM)

[Mike]

I recently tried opening a 5GB file on a 64GB RAM machine and was a
little surprised that it didn't work.  This is with emacs 22.1.1,
compiled 64-bit (part of a Red Hat distribution).

Is this supposed to work?  Is there any reason why it shouldn't?

Re: "maximum buffer size exceeded" in 64-bit emacs 22.1.1 (64G RAM)

[Evans Winner]

Mike <tutufan@gmail.com> writes:


    I recently tried opening a 5GB file

It had to happen sometime.  someone opened the complete
works of... everybody in history concatenated into a single
file. 

    on a 64GB RAM machine

So, uh, is that standard these days?  What kind of machine
is it?  Where can I get one cheap?

    Is this supposed to work?  Is there any reason why it
    shouldn't?

I'm sorry this is a totally content-less reply.  I just
couldn't help myself.  

But there is an emacswiki page that might be useful[1].
Perhaps the OS is not really prepared to grant all 64Gb of
memory to that one process.  Even with 8 bits used for lispy
things that I didn't really pause to read carefully, I would
think 64 bit addressing gives you over 7Gb, right?

Footnotes: 
[1]  http://www.emacswiki.org/emacs/EmacsFileSizeLimit

Re: "maximum buffer size exceeded" in 64-bit emacs 22.1.1 (64G RAM)

[Xah Lee]

On Mar 25, 4:37 pm, Evans Winner <tho...@timbral.net> wrote:
> Mike <tutu...@gmail.com> writes:
>
>     I recently tried opening a 5GB file
> ...
> [1]  http://www.emacswiki.org/emacs/EmacsFileSizeLimit

yeah, this is a frequently asked problem. It was discussed by emacs
developers here now and there, but as far as i know it's not something
active for fix.

i think it might be intuitive to look at what's the max file size
other editor supports, so we can get a sense how emacs does in
comparison.

i don't have much data on this... but from experience in 2000, i think
i was able to open large files in vi, but not in emacs. (the file size
was prob a hundred megabyte or more) From my experience of using
BBEdit/TextWrangler, i doubt it can do better than emacs, and same for
MicrosoftWord. It is also my guess that Eclipse, JEdit, Xcode, are
probably all worse than emacs in this regard...

  Xah
∑ http://xahlee.org/

☄

Re: "maximum buffer size exceeded" in 64-bit emacs 22.1.1 (64G RAM)

[Mike]

On Mar 25, 6:37 pm, Evans Winner <tho...@timbral.net> wrote:
>     I recently tried opening a 5GB file
>
> It had to happen sometime.  someone opened the complete
> works of... everybody in history concatenated into a single
> file.

Yeah, bioinformatics is like that.  :-)  I imagine there are a lot of
scientific disciplines that have voluminous experimental output.  Not
to mention the CIA...

>     on a 64GB RAM machine
>
> So, uh, is that standard these days?  What kind of machine
> is it?  Where can I get one cheap?

At Wal-Mart, about eight years from now.  Seriously, though--yes, I'm
spoiled, but trying to get things fixed for the poor chaps that will
be following after me.

> But there is an emacswiki page that might be useful[1].
> Perhaps the OS is not really prepared to grant all 64Gb of
> memory to that one process.  Even with 8 bits used for lispy
> things that I didn't really pause to read carefully, I would
> think 64 bit addressing gives you over 7Gb, right?
>
> Footnotes:
> [1]  http://www.emacswiki.org/emacs/EmacsFileSizeLimit

I looked at the page, and like everything else that google turned up,
it seems to be dated.  It would seem that there ought to be a very
straightforward way to get huge buffers on a 64-bit, 64G machine.
It's not an OS issue--I can create 32GB+ strings in Python on this
machine.

I'll send in a bug report and see what that draws.

(Thanks also to Xah for your reply.)

Mike

Re: "maximum buffer size exceeded" in 64-bit emacs 22.1.1 (64G RAM)

[Eli Zaretskii]

> From: Mike <tutufan@gmail.com>
> Date: Wed, 25 Mar 2009 11:00:45 -0700 (PDT)
> 
> I recently tried opening a 5GB file on a 64GB RAM machine and was a
> little surprised that it didn't work.  This is with emacs 22.1.1,
> compiled 64-bit (part of a Red Hat distribution).

Are you sure that Emacs is a 64-bit binary?  What does "file emacs"
say?