Towards a large file mode, slowness

Towards a large file mode, slowness

[Andreas Röhler]

Hi,

dealing with large files, exceeding emacs capabilities,
idea is to use `sed' to deliver subsequent chunks.

This seems possible if the files structur is line-separated, 
if no single line exceeds max volume.

For example that way:

sed -n '111,130p' ein.txt

In the real program below the range is given by var
`ltm-start-line' and

(+ ltm-start-line ltm-line-step)

If I call `sed' from within an Emacs-Shell, its quite
fast regardless of the files volume.

Using `shell-command-to-string' or `call-process' as
the program below, it's slowed down
considerably.

Any ideas how to avoid this?

Thanks

Andreas Röhler

;;; large-text-mode.el --- Loads large line-separated files 

;; Copyright (C) 2008  Andreas Roehler

;; Author: vorname nachname <andreas.roehler@online.de>
;; Keywords: wp

;; This file is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 3, or (at your option)
;; any later version.

;; This file is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to
;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
;; Boston, MA 02110-1301, USA.

;;; Commentary:

;; 

;;; Code:

;; sed -n '111,130p' ein.txt

(require 'ffap)

(setq ltm-line-step 2)
(setq ltm-start-line 1)
(defvar ltm-file nil)

(defun ltm-find-file (&optional ltm-start-line ltm-file)
  (interactive "p")
  (let* ((file (cond (ltm-file ltm-file)
                     ((featurep 'ffap)
                      (ffap-guesser))
                     (read-from-minibuffer "File: ")))
         (buffer (file-name-nondirectory file)))
    (ltm-find-file-intern file buffer (concat "sed -n "(number-to-string 
ltm-start-line)"," (number-to-string (+ ltm-start-line ltm-line-step))"p ")))
  (setq ltm-start-line ltm-start-line)
  (setq ltm-file ltm-file))

(defun ltm-find-file-intern (file buffer sedcommand)
  (set-buffer (get-buffer-create buffer))
  (erase-buffer)
  (switch-to-buffer buffer)
 ;; Two alternatives
 ;; (call-process "/bin/bash" file buffer t "-c" sedcommand) 
  (insert (shell-command-to-string  (concat sedcommand file)))
  )

(defun ltm-next ()
  " "
  (interactive)
  (setq ltm-start-line (+ ltm-start-line ltm-line-step))
  (ltm-find-file ltm-start-line ltm-file))

(provide 'large-text-mode)
;;; large-text-mode.el ends here

Re: Towards a large file mode, slowness

[Kevin Rodgers]

Andreas Röhler wrote:
> Hi,
> 
> dealing with large files, exceeding emacs capabilities,
> idea is to use `sed' to deliver subsequent chunks.
> 
> This seems possible if the files structur is line-separated, 
> if no single line exceeds max volume.
> 
> For example that way:
> 
> sed -n '111,130p' ein.txt
> 
> In the real program below the range is given by var
> `ltm-start-line' and
> 
> (+ ltm-start-line ltm-line-step)
> 
> If I call `sed' from within an Emacs-Shell, its quite
> fast regardless of the files volume.
> 
> Using `shell-command-to-string' or `call-process' as
> the program below, it's slowed down
> considerably.
> 
> Any ideas how to avoid this?

Read the thread re: "opening large files (few hundred meg)" from
January - February for some other ideas on how to implement this.

-- 
Kevin Rodgers
Denver, Colorado, USA