Re: map-file-lines

[Ted Zlatanov <tzz@lifelogs.com>]

On Mon, 02 Feb 2009 11:20:07 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> Emacs Lisp lacks a good way to iterate over all the lines of a file,
TZ> especially for a large file.  The following code tries to provide a
TZ> solution, concentrating on reading a block of data in one shot and then
TZ> processing it line by line.  It may be more efficient to write this in
TZ> C.  Also, it does not deal with cases where the first line read is
TZ> bigger than the buffer size, and may have other bugs, but it works for
TZ> me so I thought I'd post it for comments and criticism.

Updated:

- line count 0-based now, logic is cleaner
- buffer size 128K by default
- accept start line and count
- abort when the lambda func returns nil
- renamed endline to line-end for clarity

Thanks
Ted

(defun map-file-lines (file func &optional startline count bufsize)
  (let ((filepos 0)
	(linenum 0)
	(bufsize (or bufsize (* 128 1024))))
    (with-temp-buffer
      (while
	  (let*
	      ((inserted (insert-file-contents
			  file nil
			  filepos (+ filepos bufsize) 
			  t))
	       (numlines (count-lines (point-min) (point-max)))
	       (read (nth 1 inserted))
	       (done (< 1 read))
	       result line-end)
	    (dotimes (n (count-lines (point-min) (point-max)))
		(goto-char (point-min))
		(setq line-end (line-end-position)
		      result (if (and startline (< linenum startline))
				 ()
			       (if (and 
				    count
				    (>= (- linenum startline) count))
				   (return)
				 (funcall func 
					  (buffer-substring 
					   (line-beginning-position)
					   line-end)
					  linenum)))
		      done (and done result))
		(incf filepos line-end)
		(forward-line)
		(incf linenum))
	    done)))
    linenum))

;;(map-file-lines "/tmp/test" (lambda (line num) (message "%d: %s" num line)))
;;(map-file-lines "/tmp/test" (lambda (line num) (message "%d: %s" num line)) 100)
;;(map-file-lines "/tmp/test" (lambda (line num) (message "%d: %s" num line)) 100 10)