From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.help Subject: Re: opening large files (few hundred meg) Date: Wed, 30 Jan 2008 15:01:44 -0500 Organization: UseNetServer.com Message-ID: References: <1f94fef6-a335-4ce5-8d4b-7e87025a28dc@e32g2000prn.googlegroups.com> <87r6g1esga.fsf@gmx.de> <2adae7bb-c775-4a6e-bf83-66a8618b326d@s12g2000prg.googlegroups.com> <87ejc1m21x.fsf@lion.rapttech.com.au> <36295bce-c40b-4faf-ae7d-04eb90796da8@q39g2000hsf.googlegroups.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1201725745 1502 80.91.229.12 (30 Jan 2008 20:42:25 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 30 Jan 2008 20:42:25 +0000 (UTC) To: help-gnu-emacs@gnu.org Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Wed Jan 30 21:42:45 2008 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1JKJlB-00035F-I8 for geh-help-gnu-emacs@m.gmane.org; Wed, 30 Jan 2008 21:42:17 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1JKJkk-0004Ey-Dv for geh-help-gnu-emacs@m.gmane.org; Wed, 30 Jan 2008 15:41:50 -0500 Original-Path: shelby.stanford.edu!newsfeed.stanford.edu!postnews.google.com!news1.google.com!news.glorb.com!uns-out.usenetserver.com!news.usenetserver.com!pc03.usenetserver.com!TEKSAVVY.COM-Free-a2kHrUvQQWlmc!not-for-mail Original-Newsgroups: gnu.emacs.help User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.50 (gnu/linux) Cancel-Lock: sha1:y+tfsFdTxIcHJD1qU3UlFKWR6Ec= Original-X-Complaints-To: abuse@usenetserver.com Original-Lines: 28 Original-X-Trace: 28f7547a0d76b6b5d3adc19650 Original-Xref: shelby.stanford.edu gnu.emacs.help:155718 X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.help:51101 Archived-At: >> Perhaps you could process the file in chunks, using the optional args >> to insert-file-contents to put subsets of the file into a buffer. >> I haven't tried this myself, so I am not even sure it would work. > No need to try: it won't work. As I wrote earlier in this thread, the > problem is that Emacs cannot address offsets into the buffer larger > than 0.5 gig, and this problem will cause the arguments to > insert-file-contents to overflow exactly like when you read the entire > file. You don't have to use the built in limits of insert-file-contents: you can extract parts of the file using `dd' first (using Elisp floats to represent the larger integers). Also it'd be easy enough to extend insert-file-contents (at the C level) to accept float values for BEG and END (or pairs of integers) so as to be able to represent larger values. It's quite doable. The way I see it, a large-text-buffer would generally have 3 chunks of N megabytes each, point being in the middle one. The 1st and 3rd chunks would be covered with a `point-entered' property that would automatically slide the window forward or backward to bring point back into the middle chunk. That wouldn't be sufficient to make it all work, but it's probably a good starting point. Stefan