From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.devel Subject: map-file-lines Date: Mon, 02 Feb 2009 11:20:07 -0600 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <86wsc87o3c.fsf@lifelogs.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1233596039 30224 80.91.229.12 (2 Feb 2009 17:33:59 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 2 Feb 2009 17:33:59 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Feb 02 18:35:13 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1LU2hI-0003dl-Cl for ged-emacs-devel@m.gmane.org; Mon, 02 Feb 2009 18:35:00 +0100 Original-Received: from localhost ([127.0.0.1]:56810 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LU2fz-0001Y5-Py for ged-emacs-devel@m.gmane.org; Mon, 02 Feb 2009 12:33:39 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LU2RZ-0003Yf-AN for emacs-devel@gnu.org; Mon, 02 Feb 2009 12:18:45 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LU2RW-0003Wo-Pu for emacs-devel@gnu.org; Mon, 02 Feb 2009 12:18:44 -0500 Original-Received: from [199.232.76.173] (port=43901 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LU2RW-0003Wf-FO for emacs-devel@gnu.org; Mon, 02 Feb 2009 12:18:42 -0500 Original-Received: from main.gmane.org ([80.91.229.2]:33839 helo=ciao.gmane.org) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1LU2RW-00026O-7B for emacs-devel@gnu.org; Mon, 02 Feb 2009 12:18:42 -0500 Original-Received: from list by ciao.gmane.org with local (Exim 4.43) id 1LU2RR-0007YE-E0 for emacs-devel@gnu.org; Mon, 02 Feb 2009 17:18:37 +0000 Original-Received: from 38.98.147.130 ([38.98.147.130]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 02 Feb 2009 17:18:37 +0000 Original-Received: from tzz by 38.98.147.130 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 02 Feb 2009 17:18:37 +0000 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 38 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 38.98.147.130 X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.0.60 (gnu/linux) Cancel-Lock: sha1:CygFj6zSuJXDe6U6yYaFa2xWGYY= X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:108604 Archived-At: Emacs Lisp lacks a good way to iterate over all the lines of a file, especially for a large file. The following code tries to provide a solution, concentrating on reading a block of data in one shot and then processing it line by line. It may be more efficient to write this in C. Also, it does not deal with cases where the first line read is bigger than the buffer size, and may have other bugs, but it works for me so I thought I'd post it for comments and criticism. Thanks Ted (defun map-file-lines (file func &optional bufsize) (let ((filepos 0) (linenum 0) (bufsize (or bufsize 4096))) (with-temp-buffer (while (let* ((inserted (insert-file-contents file nil filepos (+ filepos bufsize) t)) (numlines (count-lines (point-min) (point-max))) (read (nth 1 inserted)) (done (< 1 read))) (dotimes (n (count-lines (point-min) (point-max))) (goto-char (point-min)) (funcall func (buffer-substring (line-beginning-position) (line-end-position)) (incf linenum)) (incf filepos (line-end-position)) (forward-line)) done))) linenum)) ;;(map-file-lines "/tmp/test" (lambda (line num) (message "%d: %s" line)))