Re: What's your favourite *under_publicized* editing feature of Emacs?

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Simón Ortiz]

I don't know if it is under_publicized, but I like a lot M-x hexl-mode

It's mode for editing binary files! Extremely useful when you need to
save/load data to/from binaries.

Also, since I have several windows opened at same time, I use windmove-
up, windmove-right, etc. to navigate the windows. It's more precise
than C-x o. Of course, I have these command assigned to shortcuts.

Erik: I'm adopting mouse-avoidance-mode :D thanks!

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Ken Goldman]

I use hexl-mode all the time.

I've often needed an insert, as opposed to overwrite.  Do you know if 
there's a way to insert?

On 01/28/2011 04:54 AM, Simón Ortiz wrote:
> I don't know if it is under_publicized, but I like a lot M-x hexl-mode
>
> It's mode for editing binary files! Extremely useful when you need to
> save/load data to/from binaries.

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Stefan Monnier]

> I use hexl-mode all the time.
> I've often needed an insert, as opposed to overwrite.  Do you know if
> there's a way to insert?

Sadly, there isn't.  I have a new mode I call "nhexl-mode" which does
"the same" as hexl-mode but in a completely different way (the
conversion is done in a font-lockish way, so the buffer's content is
not affected, only the display) so you can use it with buffer insertion
and pretty much anything else.  Sadly, it bumps into some severe
performance limitations in the current redisplay code, so it's largely
unusable other than on very small buffers :-(


        Stefan

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Javier Sanz]

Some that come to my mind:
- nXML mode, specially sexp navigation. I edit XML files quite a lot,
and features like navigating up and down in the tree, killing the
contents of some XML element or auto tag closing are great.
- I could not live with other some minor modes: autopair.el, guess-
style.el, browse-kill-ring.el, globalff.el
- Using define-generic-mode, I've defined a major mode for my app
logs, which highlights errors and warnings in different colors and
makes them easier to see.
- hippie-expand, rgrep, psvn-mode, global-linum-mode. These are more
common.
- Some macros that call some command line Linux tools to process the
content of a buffer have saved me a lot of time.

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Oleksandr Gavenko]

On 2011-01-29 0:59, Stefan Monnier wrote:
>> I use hexl-mode all the time.
>> I've often needed an insert, as opposed to overwrite.  Do you know if
>> there's a way to insert?
>
> Sadly, there isn't.  I have a new mode I call "nhexl-mode" which does
> "the same" as hexl-mode but in a completely different way (the
> conversion is done in a font-lockish way, so the buffer's content is
> not affected, only the display) so you can use it with buffer insertion
> and pretty much anything else.  Sadly, it bumps into some severe
> performance limitations in the current redisplay code, so it's largely
> unusable other than on very small buffers :-(
>
It is possible read only portion of file by Emacs
and don't load all file content into buffer?

Many hex viewer in read-only mode allow open large files without
performance penalty. Emacs don't.

-- 
Best regards!

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Eli Zaretskii]

> From: Oleksandr Gavenko <gavenkoa@gmail.com>
> Date: Sat, 29 Jan 2011 16:46:33 +0200
> 
> It is possible read only portion of file by Emacs
> and don't load all file content into buffer?

Yes, see the doc string of insert-file-contents.

> Many hex viewer in read-only mode allow open large files without
> performance penalty. Emacs don't.

It won't help in this case, because the performance problems Stefan
mentions make his nhexl-mode unusable even on relatively small
buffers, unfortunately.

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Perry Smith]

On Jan 29, 2011, at 8:46 AM, Oleksandr Gavenko wrote:

> On 2011-01-29 0:59, Stefan Monnier wrote:
>>> I use hexl-mode all the time.
>>> I've often needed an insert, as opposed to overwrite.  Do you know if
>>> there's a way to insert?
>> 
>> Sadly, there isn't.  I have a new mode I call "nhexl-mode" which does
>> "the same" as hexl-mode but in a completely different way (the
>> conversion is done in a font-lockish way, so the buffer's content is
>> not affected, only the display) so you can use it with buffer insertion
>> and pretty much anything else.  Sadly, it bumps into some severe
>> performance limitations in the current redisplay code, so it's largely
>> unusable other than on very small buffers :-(
>> 
> It is possible read only portion of file by Emacs
> and don't load all file content into buffer?
> 
> Many hex viewer in read-only mode allow open large files without
> performance penalty. Emacs don't.

This thread is beginning to drift a bit... but something I wish I had was a way
to view a file without loading all of it.  I look at hugh trace files and I have to
get out of emacs and start a shell so I can use more / less.

The shame of it all! ;-)

I don't know of such a critter.  If there is one, that would be super cool.

Perry

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Stefan Monnier]

> I don't know of such a critter.  If there is one, that would be super cool.

Agreed,


        Stefan

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Oleksandr Gavenko]

On 2011-01-29 14:55, Javier Sanz wrote:
> Some that come to my mind:
> - Using define-generic-mode, I've defined a major mode for my app
> logs, which highlights errors and warnings in different colors and
> makes them easier to see.
How large your log files?

I set 'grep-mode' on my log file (which follow GNU
error coding convention). On 50-100 KiB logs it take
10 sec to highlight matches. On larger files I type
C-g to stop matching for highlighting.

-- 
Best regards!

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Oleksandr Gavenko]

On 2011-02-06 22:11, Oleksandr Gavenko wrote:
> On 2011-01-29 14:55, Javier Sanz wrote:
>> Some that come to my mind:
>> - Using define-generic-mode, I've defined a major mode for my app
>> logs, which highlights errors and warnings in different colors and
>> makes them easier to see.
> How large your log files?
>
> I set 'grep-mode' on my log file (which follow GNU
> error coding convention). On 50-100 KiB logs it take
> 10 sec to highlight matches. On larger files I type
> C-g to stop matching for highlighting.
>
Performance penalty because I use grep-mode which based on compilation-mode.

In private mail Javier Sanz show me his log mode based on
'define-generic-mode'.

I check and found that font-lock mechanism is pretty fast.

This become start point to develop some code by hand.

And I realize that compilation-mode is dumb as it try
set on all matched file/line pairs text properties to
allow move to another file by RET.

Instead I highlight required patterns and make GOTO error
function which analyse only current line:


;;; my-log-mode.el --- major mode for error logs

;; Copyright (C) 2010 by Oleksandr Gavenko <gavenkoa@gmail.com>

;; You can do anything with this file without any warranty.

;; Author: Oleksandr Gavenko <gavenkoa@gmail.com>
;; Maintainer: Oleksandr Gavenko <gavenkoa@gmail.com>
;; Created: 2011-02-09
;; Version: 0.1
;; Keywords: logging

;;; Commentary:
;;
;; Very pure release.

;;; Code:

(defun my-log-goto (point)
   ""
   (interactive "d")
   (let ( start stop line fname fline (fregex 
"^\\([^:]+\\):\\([[:digit:]]+\\):") )
     (save-excursion
       (move-beginning-of-line 1)
       (setq start (point))
       (move-end-of-line 1)
       (setq stop (point))
       (setq line (filter-buffer-substring start stop))
       (string-match fregex line)
       (setq fname (match-string 1 line))
       (when fname
         (setq fline (string-to-int (match-string 2 line)))
         )
       )
     (when (and fname (file-exists-p fname))
       (find-file-other-window fname)
       (goto-line fline)
       )
     ))

(setq my-log-mode-map (make-sparse-keymap))
(define-key my-log-mode-map (kbd "RET") 'my-log-goto)

(require 'generic-x)

;;;###autoload
(define-generic-mode
   'my-log-mode
   nil
   nil
   '(
     ("^\\([^:]+\\):\\([[:digit:]]+\\):[^
]+$" (1 font-lock-keyword-face) (2 font-lock-type-face))
     ("^\\([^:]\\{1,10\\}\\):[^
]+$" (1 font-lock-function-name-face))
     )
   ;; '("\\.log$")
   nil
   (list
    (lambda nil
      ;; (setq buffer-read-only t)
      (use-local-map my-log-mode-map)
      (modify-syntax-entry ?' ".")
      (modify-syntax-entry ?\" ".")
      ))
   )

;;; my-log-mode.el ends here

-- 
Best regards!

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Stefan Monnier]

>> I set 'grep-mode' on my log file (which follow GNU
>> error coding convention). On 50-100 KiB logs it take
>> 10 sec to highlight matches. On larger files I type
>> C-g to stop matching for highlighting.

You may want to try the Emacs trunk's version of compile.el (I think
it'll work with Emacs-23 as well).  It should help out by being more
lazy (e.g. it'll still takes 10s to parse the file when you jump to the
end of it, but it should open instantaneously because it'll only need to
parse the first page for that).

> And I realize that compilation-mode is dumb as it try
> set on all matched file/line pairs text properties to
> allow move to another file by RET.

compilation-mode indeed does more work than I'd like at the "highlight"
time, and some of that work could indeed be delayed to the moment the
user hits RET on a line.  But my impression is that most of the time is
spent regexp-matching (i.e. looking for matches) anyway, so moving this
work won't make much difference if any.  IOW my impression is that if
your code is faster it's because it uses either more efficient regexps,
or fewer regexps.

>     ("^\\([^:]+\\):\\([[:digit:]]+\\):[^

E.g. this regexp is more efficient than the one used by grep.el, by
disallowing : in file names (resulting in a backtrack-free regexp).


        Stefan

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Oleksandr Gavenko]

On 2011-02-09 6:37, Stefan Monnier wrote:
>>> I set 'grep-mode' on my log file (which follow GNU
>>> error coding convention). On 50-100 KiB logs it take
>>> 10 sec to highlight matches. On larger files I type
>>> C-g to stop matching for highlighting.
>
> You may want to try the Emacs trunk's version of compile.el (I think
> it'll work with Emacs-23 as well).  It should help out by being more
> lazy (e.g. it'll still takes 10s to parse the file when you jump to the
> end of it, but it should open instantaneously because it'll only need to
> parse the first page for that).
>
Sound interesting.

But you wrote that jumping to end of buffer take whole time ((

We recently discuss about Emacs hex mode and I ask how read only
needed part of file to display (as binaries some times very large
for opening in Emacs).

This allow Emacs spend time on processing only needed (displayed)
part of information.

Same with highlighting of compilation/grep like buffers.

Usually reference for file/line placed on single line (or two...).

So highlighting can be performed only on visible part on large files.

Does currently Emacs highlight all file or this depend on mode?

How about generic-modes?

>> And I realize that compilation-mode is dumb as it try
>> set on all matched file/line pairs text properties to
>> allow move to another file by RET.
>
> compilation-mode indeed does more work than I'd like at the "highlight"
> time, and some of that work could indeed be delayed to the moment the
> user hits RET on a line.  But my impression is that most of the time is
> spent regexp-matching (i.e. looking for matches) anyway, so moving this
> work won't make much difference if any.  IOW my impression is that if
> your code is faster it's because it uses either more efficient regexps,
> or fewer regexps.
>
>>      ("^\\([^:]+\\):\\([[:digit:]]+\\):[^
>
> E.g. this regexp is more efficient than the one used by grep.el, by
> disallowing : in file names (resulting in a backtrack-free regexp).
>
I previously ask about regex performance in this list and
try define-generic-mode with different regex. Time to
highlighting do not significant changed with different regex.

Seems problem in another place and I read doc how profile.

Look at 'my.log' file, which is 289 KiB long:

-*- mode: my-log; default-directory: "~/devel/job/plugin-hg/" -*-
WM: 0x0024: WM_GETMINMAXINFO
win-gui-admin.c:140: WndProc #8
WM: 0x0024: WM_GETMINMAXINFO
win-gui-admin.c:140: WndProc #8
WM: 0x0024: WM_GETMINMAXINFO
...  (repeat 2 last lines many times)

When I eval: (benchmark-run 1 (revert-buffer))
I get: (0.172 0 0.0)

When I change mode line to

-*- mode: grep; default-directory: "~/devel/job/plugin-hg/" -*-

and eval: (benchmark-run 1 (revert-buffer))
I get: (22.406 1 0.156)

As you can see it is 100 times slower!

I perform 'M-x elp-instrument-package grep'
and 'M-x elp-instrument-package compilation'. Next I perform
'(revert-buffer)' and 'M-x elp-results' and get:

grep-mode                              1           23.906      23.906
compilation-error-properties           4742        23.047  0.0048601855
compilation-internal-error-properties  4742        0.094   1.98...e-005
compilation-get-file-structure         4742        0.016   3.37...e-006
compilation-mode                       1           0.0           0.0
compilation-mode-font-lock-keywords    1           0.0           0.0
compilation-setup                      1           0.0           0.0

As you can see 'compilation-error-properties' eat a lot of time and
it executed on all lines like 'file:line: msg' (in my example half of
lines match such pattern)!

I press C-u C-x = on green underscored char (file name for jump) and see
among text properties: 'message' with value is:

((nil 140
       (("win-gui-admin.c" nil)
        nil
        (140 #1)))
  2 nil)

I assume that 'compilation-error-properties' spend time on setting this
properties.

So because of unnecessary work to set text property to well structured
data on *all* lines we have slow down instead perform parsing of
reference on demand.

As I previously shown 'my-log-goto' func for this purpose.

I think such idea (trying match pattern on point position) can be
generalized by looping across 'compilation-error-regexp-alist'.

Do you think this need filed as a bug or request?

Or I am something missing?

-- 
Best regards!

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Oleksandr Gavenko]

On 2011-02-09 6:37, Stefan Monnier wrote:
>>> I set 'grep-mode' on my log file (which follow GNU
>>> error coding convention). On 50-100 KiB logs it take
>>> 10 sec to highlight matches. On larger files I type
>>> C-g to stop matching for highlighting.
>
> You may want to try the Emacs trunk's version of compile.el (I think
> it'll work with Emacs-23 as well).  It should help out by being more
> lazy (e.g. it'll still takes 10s to parse the file when you jump to the
> end of it, but it should open instantaneously because it'll only need to
> parse the first page for that).
>
Sound interesting.

But you wrote that jumping to end of buffer take whole time ((

We recently discuss about Emacs hex mode and I ask how read only
needed part of file to display (as binaries some times very large
for opening in Emacs).

This allow Emacs spend time on processing only needed (displayed)
part of information.

Same with highlighting of compilation/grep like buffers.

Usually reference for file/line placed on single line (or two...).

So highlighting can be performed only on visible part on large files.

Does currently Emacs highlight all file or this depend on mode?

How about generic-modes?

>> And I realize that compilation-mode is dumb as it try
>> set on all matched file/line pairs text properties to
>> allow move to another file by RET.
>
> compilation-mode indeed does more work than I'd like at the "highlight"
> time, and some of that work could indeed be delayed to the moment the
> user hits RET on a line.  But my impression is that most of the time is
> spent regexp-matching (i.e. looking for matches) anyway, so moving this
> work won't make much difference if any.  IOW my impression is that if
> your code is faster it's because it uses either more efficient regexps,
> or fewer regexps.
>
>>      ("^\\([^:]+\\):\\([[:digit:]]+\\):[^
>
> E.g. this regexp is more efficient than the one used by grep.el, by
> disallowing : in file names (resulting in a backtrack-free regexp).
>
I previously ask about regex performance in this list and
try define-generic-mode with different regex. Time to
highlighting do not significant changed with different regex.

Seems problem in another place and I read doc how profile.

Look at 'my.log' file, which is 289 KiB long:

-*- mode: my-log; default-directory: "~/devel/job/plugin-hg/" -*-
WM: 0x0024: WM_GETMINMAXINFO
win-gui-admin.c:140: WndProc #8
WM: 0x0024: WM_GETMINMAXINFO
win-gui-admin.c:140: WndProc #8
WM: 0x0024: WM_GETMINMAXINFO
...  (repeat 2 last lines many times)

When I eval: (benchmark-run 1 (revert-buffer))
I get: (0.172 0 0.0)

When I change mode line to

-*- mode: grep; default-directory: "~/devel/job/plugin-hg/" -*-

and eval: (benchmark-run 1 (revert-buffer))
I get: (22.406 1 0.156)

As you can see it is 100 times slower!

I perform 'M-x elp-instrument-package grep'
and 'M-x elp-instrument-package compilation'. Next I perform
'(revert-buffer)' and 'M-x elp-results' and get:

grep-mode                              1           23.906      23.906
compilation-error-properties           4742        23.047  0.0048601855
compilation-internal-error-properties  4742        0.094   1.98...e-005
compilation-get-file-structure         4742        0.016   3.37...e-006
compilation-mode                       1           0.0           0.0
compilation-mode-font-lock-keywords    1           0.0           0.0
compilation-setup                      1           0.0           0.0

As you can see 'compilation-error-properties' eat a lot of time and
it executed on all lines like 'file:line: msg' (in my example half of
lines match such pattern)!

I press C-u C-x = on green underscored char (file name for jump) and see
among text properties: 'message' with value is:

((nil 140
       (("win-gui-admin.c" nil)
        nil
        (140 #1)))
  2 nil)

I assume that 'compilation-error-properties' spend time on setting this
properties.

So because of unnecessary work to set text property to well structured
data on *all* lines we have slow down instead perform parsing of
reference on demand.

As I previously shown 'my-log-goto' func for this purpose.

I think such idea (trying match pattern on point position) can be
generalized by looping across 'compilation-error-regexp-alist'.

Do you think this need filed as a bug or request?

Or I am something missing?

-- 
Best regards!

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Thien-Thi Nguyen]

() Oleksandr Gavenko <gavenkoa@gmail.com>
() Wed, 09 Feb 2011 23:31:15 +0200

   So because of unnecessary work to set text property to well
   structured data on *all* lines we have slow down instead perform
   parsing of reference on demand.

   [...]

   Or I am something missing?

You have well-structured data but the compilation (likewise, grep)
font-locking tries to cope with a more general input.  You can prune
some elements from ‘compilation-error-regexp-alist’ to help focus the
font-locking.  Have you tried that?

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Stefan Monnier]

> I perform 'M-x elp-instrument-package grep'
> and 'M-x elp-instrument-package compilation'. Next I perform
> '(revert-buffer)' and 'M-x elp-results' and get:

> grep-mode                              1           23.906      23.906
> compilation-error-properties           4742        23.047  0.0048601855
> compilation-internal-error-properties  4742        0.094   1.98...e-005
> compilation-get-file-structure         4742        0.016   3.37...e-006
> compilation-mode                       1           0.0           0.0
> compilation-mode-font-lock-keywords    1           0.0           0.0
> compilation-setup                      1           0.0           0.0

Duh!  I completely forgot about it, but yes, there was a clear
performance bug in compilation-error-properties which made it take time
proportional to the buffer size in cases such as M-x grep.
This performance bug is also largely fixed in Emacs-24's compile.el.


        Stefan

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Oleksandr Gavenko]

On 2011-02-10 0:32, Thien-Thi Nguyen wrote:
> () Oleksandr Gavenko<gavenkoa@gmail.com>
> () Wed, 09 Feb 2011 23:31:15 +0200
>
>     So because of unnecessary work to set text property to well
>     structured data on *all* lines we have slow down instead perform
>     parsing of reference on demand.
>
>     [...]
>
>     Or I am something missing?
>
> You have well-structured data but the compilation (likewise, grep)
> font-locking tries to cope with a more general input.  You can prune
> some elements from ‘compilation-error-regexp-alist’ to help focus the
> font-locking.  Have you tried that?
>
Yes. I wrote:

(require 'compile)
(setq my-log2-error-regexp-alist
       '(
         ("^\\([^:]+\\):\\([[:digit:]]+\\):" 1 2)
         ))
(define-compilation-mode my-log2-mode "MyLog"
   "Log mode."
   (set (make-local-variable 'compilation-error-regexp-alist)
         my-log2-error-regexp-alist)
   )

And as I describe previously I eval on same 289 KiB long buffer:
(benchmark-run 1 (revert-buffer))
and get: (20.937 1 0.109)

In contrast to test of 'grep-mode' where I get: (22.406 1 0.156)

As I use same regex for derived from compilation mode as for
custom highlighting mode I decide that bottle neck lies
in setting properties to text.

When set only 'face' property - this is fast. But when set 'message'
property it is slow as 'message' is complex structure which must
recalculated on every matching lines.

-- 
Best regards!

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Giorgos Keramidas]

On Sat, 29 Jan 2011 09:02:14 -0600, Perry Smith <pedzsan@gmail.com> wrote:
> This thread is beginning to drift a bit... but something I wish I had
> was a way to view a file without loading all of it.  I look at hugh
> trace files and I have to get out of emacs and start a shell so I can
> use more / less.
>
> The shame of it all! ;-)
>
> I don't know of such a critter.  If there is one, that would be super
> cool.

If there was some way to lazy-load files, I'd be happy too.

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Perry Smith]

[-- Attachment #1: Type: text/plain, Size: 2228 bytes --]


On Feb 17, 2011, at 4:53 AM, Giorgos Keramidas wrote:

> On Sat, 29 Jan 2011 09:02:14 -0600, Perry Smith <pedzsan@gmail.com> wrote:
>> This thread is beginning to drift a bit... but something I wish I had
>> was a way to view a file without loading all of it.  I look at hugh
>> trace files and I have to get out of emacs and start a shell so I can
>> use more / less.
>> 
>> The shame of it all! ;-)
>> 
>> I don't know of such a critter.  If there is one, that would be super
>> cool.
> 
> If there was some way to lazy-load files, I'd be happy too.

I looked at this.  The approach I was going to take was to put into the C code
an "open", "close", "read", and "lseek" concept and that would need a new
"type" for a file descriptor.  You might as well add "write" to complete the set.
There is nothing currently like that in the C code.  I'm not sure if you want
to put this on top of open and its siblings or on top of fopen and let the C
I/O help some with the buffering.

From there, it becomes a matter of writing lisp to load in a "page" into a
buffer so it can be displayed and then a key map so the user can page
forward and backward.  For searches I couldn't come up with a nice
solution so I was just going to loop loading a page and searching the page
until the match was found.

The second generation of this could have a fixed sized cache of pages so that
small movements (like back a page) would be instant and you could even
get fancier and pre-fetch the next page.

This could grow into a hugh beast.  You would want to hook up to the
character set things so that it would detect and process all the different
characters sets (like UTF-8, etc) properly.  And then, what the hell, you
might as well plop the nifty coloring modes on top as well so that
as you page through C text, it would be colored.  In my case, I want to
plop a wireshark mode on top so that I can view network packets.

But, for my own personal case, I have way too many projects going already
so I didn't even post this to the list until just now.  I think its a concept
whose time has come but I don't have the time right now.  I'd love to
contribute to it if anyone wants to start.

Perry


[-- Attachment #2: Type: text/html, Size: 3307 bytes --]

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Thien-Thi Nguyen]

() Perry Smith <pedzsan@gmail.com>
() Thu, 17 Feb 2011 07:43:54 -0600

   I looked at this.  The approach I was going to take was to put
   into the C code an "open", "close", "read", and "lseek" concept
   and that would need a new "type" for a file descriptor.  You
   might as well add "write" to complete the set.  There is
   nothing currently like that in the C code.

Perhaps ‘insert-file-contents’ can be used to implement this
feature in Lisp.

view/edit large files (was: What's your favourite *under_publicized* editing feature of Emacs?)

[Ted Zlatanov]

On Thu, 17 Feb 2011 15:01:54 +0100 Thien-Thi Nguyen <ttn@gnuvola.org> wrote: 

TN> () Perry Smith <pedzsan@gmail.com>
TN> () Thu, 17 Feb 2011 07:43:54 -0600

TN>    I looked at this.  The approach I was going to take was to put
TN>    into the C code an "open", "close", "read", and "lseek" concept
TN>    and that would need a new "type" for a file descriptor.  You
TN>    might as well add "write" to complete the set.  There is
TN>    nothing currently like that in the C code.

TN> Perhaps ‘insert-file-contents’ can be used to implement this
TN> feature in Lisp.

(subject adjusted to match the thread referenced below)

Please see http://thread.gmane.org/gmane.emacs.devel/108604/focus=108836
and vlf.el on the EmacsWiki.

I'd like to implement this for large (over int size) files and with
editing, but it's low on my TODO list.  If someone else is feeling
frisky...

Ted

Re: What's your favourite *under_publicized* editing feature of Emacs?

[Tim X]

Giorgos Keramidas <keramida@ceid.upatras.gr> writes:

> On Sat, 29 Jan 2011 09:02:14 -0600, Perry Smith <pedzsan@gmail.com> wrote:
>> This thread is beginning to drift a bit... but something I wish I had
>> was a way to view a file without loading all of it.  I look at hugh
>> trace files and I have to get out of emacs and start a shell so I can
>> use more / less.
>>
>> The shame of it all! ;-)
>>
>> I don't know of such a critter.  If there is one, that would be super
>> cool.
>
> If there was some way to lazy-load files, I'd be happy too.
>

Many years ago I needed to load large trace files from Oracle. I just
looked and seem to have lost the code I used, but you may find the
approach useful. This was done on a Linux box and uses standard Linux
utilities. 

The basic approach I used was to write some elisp functions that created
a temporary directory, applied the 'split' program to the input file,
split the input file into manageable chunks (split will/can add a
prefix/suffix to sequence/number the chunks) and put them into the temp
directory and then open the first chunk in a buffer. I could then run
dired, which would list al the chunks and open any one of those chunks
for viewing further chunks of the file. 

I also used a defadvice around view-file which automated the process if
the file suffix was .trc. I always planned to clean it up and add a few
extra features (such as not splitting the file if it had already been
split before etc), but then my job changed a bit and I no longer had the
need to view such large trace files.

It worked quite well. Maybe not as clean as jus having a 'lazy file
read', but did mean I could view these large files from within emacs and
setting things up was largely automated. I do vaguely remember having to
play with split a bit before I got things just right. Later I added a
command that would use sed to replace strings in really large files,
which was useful when migrating large oracle databases and you wanted to
modify the paths in order to refresh a dev version with data form a
production system. Again, can't find that code as we now do things
differently and I've not kept it.

Tim


-- 
tcross (at) rapttech dot com dot au