From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.devel Subject: Re: map-file-lines Date: Mon, 02 Feb 2009 14:48:25 -0600 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <86k5887eg6.fsf@lifelogs.com> References: <86wsc87o3c.fsf@lifelogs.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1233607639 9515 80.91.229.12 (2 Feb 2009 20:47:19 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 2 Feb 2009 20:47:19 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Feb 02 21:48:34 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1LU5iU-0002is-2V for ged-emacs-devel@m.gmane.org; Mon, 02 Feb 2009 21:48:26 +0100 Original-Received: from localhost ([127.0.0.1]:58094 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LU5hB-0005cm-Dh for ged-emacs-devel@m.gmane.org; Mon, 02 Feb 2009 15:47:05 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LU5h6-0005cX-Bo for emacs-devel@gnu.org; Mon, 02 Feb 2009 15:47:00 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LU5h2-0005c6-F4 for emacs-devel@gnu.org; Mon, 02 Feb 2009 15:46:59 -0500 Original-Received: from [199.232.76.173] (port=42148 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LU5h2-0005c3-DB for emacs-devel@gnu.org; Mon, 02 Feb 2009 15:46:56 -0500 Original-Received: from main.gmane.org ([80.91.229.2]:54181 helo=ciao.gmane.org) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1LU5h1-0004kz-U6 for emacs-devel@gnu.org; Mon, 02 Feb 2009 15:46:56 -0500 Original-Received: from list by ciao.gmane.org with local (Exim 4.43) id 1LU5gz-0007TS-U9 for emacs-devel@gnu.org; Mon, 02 Feb 2009 20:46:53 +0000 Original-Received: from 38.98.147.130 ([38.98.147.130]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 02 Feb 2009 20:46:53 +0000 Original-Received: from tzz by 38.98.147.130 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 02 Feb 2009 20:46:53 +0000 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 60 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 38.98.147.130 X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.0.60 (gnu/linux) Cancel-Lock: sha1:bnkAHN2iE/XBbIRtQvAOszvJ9us= X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:108635 Archived-At: On Mon, 02 Feb 2009 11:20:07 -0600 Ted Zlatanov wrote: TZ> Emacs Lisp lacks a good way to iterate over all the lines of a file, TZ> especially for a large file. The following code tries to provide a TZ> solution, concentrating on reading a block of data in one shot and then TZ> processing it line by line. It may be more efficient to write this in TZ> C. Also, it does not deal with cases where the first line read is TZ> bigger than the buffer size, and may have other bugs, but it works for TZ> me so I thought I'd post it for comments and criticism. Updated: - line count 0-based now, logic is cleaner - buffer size 128K by default - accept start line and count - abort when the lambda func returns nil - renamed endline to line-end for clarity Thanks Ted (defun map-file-lines (file func &optional startline count bufsize) (let ((filepos 0) (linenum 0) (bufsize (or bufsize (* 128 1024)))) (with-temp-buffer (while (let* ((inserted (insert-file-contents file nil filepos (+ filepos bufsize) t)) (numlines (count-lines (point-min) (point-max))) (read (nth 1 inserted)) (done (< 1 read)) result line-end) (dotimes (n (count-lines (point-min) (point-max))) (goto-char (point-min)) (setq line-end (line-end-position) result (if (and startline (< linenum startline)) () (if (and count (>= (- linenum startline) count)) (return) (funcall func (buffer-substring (line-beginning-position) line-end) linenum))) done (and done result)) (incf filepos line-end) (forward-line) (incf linenum)) done))) linenum)) ;;(map-file-lines "/tmp/test" (lambda (line num) (message "%d: %s" num line))) ;;(map-file-lines "/tmp/test" (lambda (line num) (message "%d: %s" num line)) 100) ;;(map-file-lines "/tmp/test" (lambda (line num) (message "%d: %s" num line)) 100 10)