From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.help Subject: Re: Reading huge files Date: Thu, 11 Jan 2007 00:54:57 -0500 Message-ID: References: <8642ba650701081417n3c658d33v6dbf743a9bd30c7c@mail.gmail.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1168497671 22879 80.91.229.12 (11 Jan 2007 06:41:11 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Thu, 11 Jan 2007 06:41:11 +0000 (UTC) Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Thu Jan 11 07:41:09 2007 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1H4tcW-0000YG-SM for geh-help-gnu-emacs@m.gmane.org; Thu, 11 Jan 2007 07:41:05 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1H4tcW-0005Al-H1 for geh-help-gnu-emacs@m.gmane.org; Thu, 11 Jan 2007 01:41:04 -0500 Original-Path: shelby.stanford.edu!newsfeed.stanford.edu!postnews.google.com!news3.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!bcklog2.nntp.dca.giganews.com!nntp.umontreal.ca!news.umontreal.ca.POSTED!not-for-mail Original-NNTP-Posting-Date: Wed, 10 Jan 2007 23:55:10 -0600 Original-Newsgroups: gnu.emacs.help User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.91 (gnu/linux) Cancel-Lock: sha1:PHWK6TvVQynwidmLngf8QRPRAyU= Original-Lines: 53 Original-NNTP-Posting-Host: 132.204.27.213 Original-X-Trace: sv3-4hWZ7oeyDtWObybIvwMnZiazzXGkZexzyOCLdzlPO9Xkj8DjfUv3SIfebgjgSPGUb3qtRbRBjdszoRo!JNYyAQYGY0zpXcAQGtPTnfvF3rMiTFS6n0i1Pmk0kJ59rMeb5ra3UCHKy+REWBwmDcoNk+sXW9Mm!X6O5HCqZHGdj/G3byg== Original-X-Complaints-To: abuse@umontreal.ca X-DMCA-Complaints-To: abuse@umontreal.ca X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly X-Postfilter: 1.3.32 Original-Xref: shelby.stanford.edu gnu.emacs.help:144622 Original-To: help-gnu-emacs@gnu.org X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.help:40227 Archived-At: > ;;; vlf.el --- View Large Files I like that. > ;; (defun vlf-extract-part-of-file (file from to) > ;; "Returns bytes in FILE from FROM to TO." > ;; (let ((size (vlf-file-size file))) > ;; (if (or (> from size) > ;; (> to size)) > ;; (error "From or to is larger that the file size")) > ;; (with-temp-buffer > ;; (shell-command > ;; (format "head --bytes %d %s | tail --bytes %d" > ;; to file (+ (- to from) 1)) t) > ;; (buffer-substring (point-min) (point-max))))) You'll indeed need something like that. I'd recommend to use `dd' rather than `split' or `head&tail'. But note that the problem is bigger than you think: the above will need to work with floating-point arguments (since fixnums suffer from the 2^28 limit) and the %d format also suffers from the 2^28 limit (or 2^31 in Emacs-CVS). That's another good reason to use `dd' since you can use a 1048576 block size and thus divide your large floats by 1048576 before turning them into fixnums. > (defvar vlf-mode-map (make-sparse-keymap) > "Keymap for `vlf-mode'.") > (defun vlf-define-keymap () > "Define keymap for `vlf-mode'." > (define-key vlf-mode-map [next] 'vlf-next) > (define-key vlf-mode-map [prior] 'vlf-prev) > (define-key vlf-mode-map "q" 'vlf-quit)) Please just use (defvar vlf-mode-map (let ((map (make-sparse-keymap))) (define-key map [next] 'vlf-next) (define-key map [prior] 'vlf-prev) (define-key map "q" 'vlf-quit) map) "Keymap for `vlf-mode'.") Also, wrt features, it'd probably be good to "slide more smoothly": e.g. always keep 2 or 3 blocks in the buffer at the same time so you can comfortably look at the text at the boundaries between blocks. Next step: use window-scroll-functions or jit-lock to detect when reaching one of the ends so that we can automatically load in the next consecutive block. Stefan