Possibility for a stream editor (sed) inside emacs ?

Possibility for a stream editor (sed) inside emacs ?

[Alin Soare]

[-- Attachment #1.1: Type: text/plain, Size: 2154 bytes --]

Last 2 days I played, and I wrote a toy sed (stream-editor). I did use and
learn sed, and to convince myself if I understood it, I wanted to write an
own sed.

My purpose was to write the machine that makes the computations, and not to
write a complete sed. Its parser that generates tokens for example does not
jump over the spaces. It's just a toy sed.

The machine generates the internal structure that can be used by a virtual
machine to render every sed command.

I did not write neither the virtual machine (I wrote just a little sketch
for a vm, and a sketch of lazy-evaluator of the special forms of conditional
and non conditional branch "s label" and "t label"), and neither the action
functions, that should be called by this machine via apply.  I wrote only
the kernel of sed. For me it became clear, and I do not want any more to
work on.

The virtual machine that evaluates the data structure that I generated can
be written to use Emacs internal buffers or elisp strings, instead of files,
as the GNU/sed does.

The data structure I generated can also be used quite directly to generate
the precise indentation of sed scripts, without defining the *action
functions for every command, using the same logic I used when I wrote the
indentation of lisp code ( ignored! ):
http://permalink.gmane.org/gmane.emacs.devel/100252.

I attach the backus-naur form I wrote last days. In order to understand the
code, you have to look in the same time to bn-form. I wrote a major mode
called "backus naur mode" to browse the backus-naur forms. The most
difficult part of the implementation was to generate the graph with the
mutual recursive function Command and Command_block. This is why I used
artist-mode to draw a picture of the data structures, starting from atomic
expressions of sed, upto the creation of the branches of Blocks.

I attach the implementation of sed, and the grammar of sed in bn-form, and a
file bn-mode.el , in order for those of you who are interested to be able to
browse the grammar. I posted the bn-mode long time ago, but the post was
ignored.

Do you consider, an internal stream editor would be good for emacs?




Alin

[-- Attachment #1.2: Type: text/html, Size: 2362 bytes --]

[-- Attachment #2: sed.tar.bz2 --]
[-- Type: application/x-bzip2, Size: 9640 bytes --]

Re: Possibility for a stream editor (sed) inside emacs ?

[Ted Zlatanov]

On Wed, 25 May 2011 02:08:05 +0300 Alin Soare <as1789@gmail.com> wrote: 

AS> My purpose was to write the machine that makes the computations, and not to
AS> write a complete sed. Its parser that generates tokens for example does not
AS> jump over the spaces. It's just a toy sed.
...
AS> Do you consider, an internal stream editor would be good for emacs?

Emacs already has good text editing functionality.  Do you like sed's
brevity?  I think most of it can be emulated with an inline DSL based on
macros instead of a full VM and parser.

I do think it's worthwhile to explore a "apply a function to every line
or block of a file" wrapper.  The key utility would be that large files
will not need to be loaded entirely into a buffer.  It's trivial with
data blocks and a little harder with line-oriented processing, but still
not too bad.  Then any stream processing functionality, including
something like sed, can be passed to the wrapper as a lambda.

Something like this line-oriented file processor (found in my .emacs but
IIRC originally from several sources, and not well tested) is what I'm
thinking of:

#+begin_src lisp
(defun map-file-lines (file func &optional startline count bufsize)
  (let ((filepos 0)
        (linenum 0)
        (bufsize (or bufsize (* 128 1024))))
    (with-temp-buffer
      (while
          (let*
              ((inserted (insert-file-contents
                          file nil
                          filepos (+ filepos bufsize)
                          t))
               (numlines (count-lines (point-min) (point-max)))
               (read (nth 1 inserted))
               (done (< 1 read))
               result line-end)
            (while (not (zerop (decf numlines)))
                (goto-char (point-min))
                (setq line-end (line-end-position)
                      result (if (and startline (< linenum startline))
                                 ()
                               (if (and
                                    count
                                    (>= (- linenum startline) count))
                                   (return)
                                 (funcall func
                                          (buffer-substring
                                           (line-beginning-position)
                                           line-end)
                                          linenum)))
                      done (and done result))
                (incf filepos line-end)
                (forward-line)
                (incf linenum))
            done)))
    linenum))
#+end_src

Ted