From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.devel Subject: Re: Possibility for a stream editor (sed) inside emacs ? Date: Wed, 25 May 2011 08:11:44 -0500 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <87tyciq60f.fsf@lifelogs.com> References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1306334629 30716 80.91.229.12 (25 May 2011 14:43:49 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 25 May 2011 14:43:49 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed May 25 16:43:44 2011 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1QPFJH-0001IZ-8z for ged-emacs-devel@m.gmane.org; Wed, 25 May 2011 16:43:43 +0200 Original-Received: from localhost ([::1]:53459 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QPFJG-0007Mv-Gq for ged-emacs-devel@m.gmane.org; Wed, 25 May 2011 10:43:42 -0400 Original-Received: from eggs.gnu.org ([140.186.70.92]:52855) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QPFJD-0007MS-2S for emacs-devel@gnu.org; Wed, 25 May 2011 10:43:39 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QPFJC-00067e-3w for emacs-devel@gnu.org; Wed, 25 May 2011 10:43:39 -0400 Original-Received: from lo.gmane.org ([80.91.229.12]:39067) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QPFJB-00067C-RM for emacs-devel@gnu.org; Wed, 25 May 2011 10:43:38 -0400 Original-Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QPDsY-0002hK-KP for emacs-devel@gnu.org; Wed, 25 May 2011 15:12:02 +0200 Original-Received: from 38.98.147.130 ([38.98.147.130]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 25 May 2011 15:12:02 +0200 Original-Received: from tzz by 38.98.147.130 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 25 May 2011 15:12:02 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 62 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 38.98.147.130 X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" User-Agent: Gnus/5.110018 (No Gnus v0.18) Emacs/24.0.50 (gnu/linux) Cancel-Lock: sha1:tdesiZKAselBN77NgVe+ajBBEBk= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 80.91.229.12 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:139693 Archived-At: On Wed, 25 May 2011 02:08:05 +0300 Alin Soare wrote: AS> My purpose was to write the machine that makes the computations, and not to AS> write a complete sed. Its parser that generates tokens for example does not AS> jump over the spaces. It's just a toy sed. ... AS> Do you consider, an internal stream editor would be good for emacs? Emacs already has good text editing functionality. Do you like sed's brevity? I think most of it can be emulated with an inline DSL based on macros instead of a full VM and parser. I do think it's worthwhile to explore a "apply a function to every line or block of a file" wrapper. The key utility would be that large files will not need to be loaded entirely into a buffer. It's trivial with data blocks and a little harder with line-oriented processing, but still not too bad. Then any stream processing functionality, including something like sed, can be passed to the wrapper as a lambda. Something like this line-oriented file processor (found in my .emacs but IIRC originally from several sources, and not well tested) is what I'm thinking of: #+begin_src lisp (defun map-file-lines (file func &optional startline count bufsize) (let ((filepos 0) (linenum 0) (bufsize (or bufsize (* 128 1024)))) (with-temp-buffer (while (let* ((inserted (insert-file-contents file nil filepos (+ filepos bufsize) t)) (numlines (count-lines (point-min) (point-max))) (read (nth 1 inserted)) (done (< 1 read)) result line-end) (while (not (zerop (decf numlines))) (goto-char (point-min)) (setq line-end (line-end-position) result (if (and startline (< linenum startline)) () (if (and count (>= (- linenum startline) count)) (return) (funcall func (buffer-substring (line-beginning-position) line-end) linenum))) done (and done result)) (incf filepos line-end) (forward-line) (incf linenum)) done))) linenum)) #+end_src Ted