`make-overlay' very slow (was: hiding lines)

[Werner LEMBERG <wl@gnu.org>]

[-- Attachment #1: Type: Text/Plain, Size: 945 bytes --]


>>> I've now come up with the code as shown in the attachment.  It
>>> works fine, however, I would like to not have empty lines; in
>>> other words, invisible text should not take any space in the
>>> buffer.  Does Emacs supports such a `compressed invisibility'?
>>
>> 
>> Make the newline invisible.
> 
> Thanks!  This perhaps deserves a notice in the documentation.

I've now tried the code with my real-world table (containing
approx. 420000 lines):

  git://repo.or.cz/srv/git/wortliste.git

and I found out that it is, alas, unusable.  It starts with about 1000
replacements per second on my GNU/Linux box (using simply `-' as the
regexp), but soon the speed decreases: After approx. 10000
replacements it only can handle 100 replacements per second -- the
attached code now contains a small counter to show the progress.  If
you comment out the call to `make-overlay', the code really flies...

Am I doing something wrong?


    Werner

[-- Attachment #2: line-invisible.el --]
[-- Type: Text/Plain, Size: 1053 bytes --]

(defun make-lines-invisible (regexp &optional arg)
  "Make all lines matching a regexp invisible and intangible.
With a prefix arg, make it visible again.  It is not necessary
that REGEXP matches the whole line; if a hit is found, the
affected line gets automatically selected.

This command affects the whole buffer."
  (interactive "MRegexp: \nP")
  (let (ov
	ovs
	count)
    (cond
     ((equal arg '(4))
      (setq ovs (overlays-in (point-min) (point-max)))
      (mapc (lambda (o)
	      (if (overlay-get o 'make-lines-invisible)
		  (delete-overlay o)))
	    ovs))
     (t
      (save-excursion
	(goto-char (point-min))
	(setq count 0)
	(while (re-search-forward regexp nil t)
	  (setq count (1+ count))
	  (if (= (% count 100) 0)
	      (message "%d" count))
	  (setq ov (make-overlay (line-beginning-position)
				 (1+ (line-end-position))))
;	  (overlay-put ov 'make-lines-invisible t)
;	  (overlay-put ov 'invisible t)
;	  (overlay-put ov 'intangible t)
	  (goto-char (line-end-position))))))))

(global-set-key "\C-cz" 'make-lines-invisible)