unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* hiding lines
@ 2009-04-02  6:52 Werner LEMBERG
  2009-04-02 19:30 ` Alan Mackenzie
  0 siblings, 1 reply; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-02  6:52 UTC (permalink / raw)
  To: emacs-devel


Folks,


I would like to hide lines in a buffer which satisfy a regexp --
either the lines get completely hidden (or perhaps indicated with
`...' as in the outline mode), or a search-and-replace function
doesn't match them.

Does Emacs have such a thing?  A lookup in the docs doesn't show
anything relevant.  This would be a very convenient function for
editing lists.


    Werner




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-02  6:52 hiding lines Werner LEMBERG
@ 2009-04-02 19:30 ` Alan Mackenzie
  2009-04-03  0:39   ` Stefan Monnier
  2009-04-03  5:44   ` hiding lines Andreas Roehler
  0 siblings, 2 replies; 28+ messages in thread
From: Alan Mackenzie @ 2009-04-02 19:30 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: emacs-devel

Hi, Werner,

On Thu, Apr 02, 2009 at 08:52:01AM +0200, Werner LEMBERG wrote:

> Folks,


> I would like to hide lines in a buffer which satisfy a regexp --
> either the lines get completely hidden (or perhaps indicated with
> `...' as in the outline mode), or a search-and-replace function
> doesn't match them.

> Does Emacs have such a thing?  A lookup in the docs doesn't show
> anything relevant.  This would be a very convenient function for
> editing lists.

The following makes the current region invisible and intangible.  It
doesn't, as yet, make it visible and tangible again very well.  It's not
quite what you're asking for, but you can likely use it as the engine
end of what you want.

    (defun make-region-invisible (beg end &optional arg)
      "Make the current region invisible and intangible.
    With a prefix arg, make it visible again.  With an arg of 0,
    make all such regions visible."
      (interactive "r\nP")
      (let (ov ovs)
        (cond
         ((eq arg 0)
          (setq ovs (overlays-in (point-min) (point-max)))
          (mapc (lambda (o)
                  (when (overlay-get o 'acm-invisible)
                    (delete-overlay o)))
                ovs))
         ((equal arg '(4))
          (setq ovs (overlays-at (point)))
          (mapc (lambda (o)
                  (when (overlay-get o 'acm-invisible)
                    (delete-overlay o)))
                ovs))
         (t
          (setq ov (make-overlay beg end))
          (overlay-put ov 'acm-invisible t)
          (overlay-put ov 'invisible t)
          (overlay-put ov 'intangible t)))))
    (global-set-key "\C-cz" 'make-region-invisible)


>     Werner

-- 
Alan Mackenzie (Nuremberg, Germany).




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-02 19:30 ` Alan Mackenzie
@ 2009-04-03  0:39   ` Stefan Monnier
  2009-04-03 12:26     ` Alan Mackenzie
  2009-04-03  5:44   ` hiding lines Andreas Roehler
  1 sibling, 1 reply; 28+ messages in thread
From: Stefan Monnier @ 2009-04-03  0:39 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: emacs-devel

> The following makes the current region invisible and intangible.  It

I'll just remind people that I strongly discourage the use of
the `intangible' property.  Your code should work well with just the
`invisible' property; if not, it might be a bug.


        Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-02 19:30 ` Alan Mackenzie
  2009-04-03  0:39   ` Stefan Monnier
@ 2009-04-03  5:44   ` Andreas Roehler
  1 sibling, 0 replies; 28+ messages in thread
From: Andreas Roehler @ 2009-04-03  5:44 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: Bastien, Stefan Monnier, Richard M Stallman, emacs-devel

Alan Mackenzie wrote:
> Hi, Werner,
> 
> On Thu, Apr 02, 2009 at 08:52:01AM +0200, Werner LEMBERG wrote:
> 
>> Folks,
> 
> 
>> I would like to hide lines in a buffer which satisfy a regexp --
>> either the lines get completely hidden (or perhaps indicated with
>> `...' as in the outline mode), or a search-and-replace function
>> doesn't match them.
> 
>> Does Emacs have such a thing?  A lookup in the docs doesn't show
>> anything relevant.  This would be a very convenient function for
>> editing lists.
> 
> The following makes the current region invisible and intangible.  It
> doesn't, as yet, make it visible and tangible again very well.  It's not
> quite what you're asking for, but you can likely use it as the engine
> end of what you want.
> 
>     (defun make-region-invisible (beg end &optional arg)
>       "Make the current region invisible and intangible.
>     With a prefix arg, make it visible again.  With an arg of 0,
>     make all such regions visible."
>       (interactive "r\nP")
>       (let (ov ovs)
>         (cond
>          ((eq arg 0)
>           (setq ovs (overlays-in (point-min) (point-max)))
>           (mapc (lambda (o)
>                   (when (overlay-get o 'acm-invisible)
>                     (delete-overlay o)))
>                 ovs))
>          ((equal arg '(4))
>           (setq ovs (overlays-at (point)))
>           (mapc (lambda (o)
>                   (when (overlay-get o 'acm-invisible)
>                     (delete-overlay o)))
>                 ovs))
>          (t
>           (setq ov (make-overlay beg end))
>           (overlay-put ov 'acm-invisible t)
>           (overlay-put ov 'invisible t)
>           (overlay-put ov 'intangible t)))))
>     (global-set-key "\C-cz" 'make-region-invisible)
> 
> 
>>     Werner
> 

Thanks to Bastien Guerry Emacs-Lisp Bill-Board is
available for just that kind of tasks.

Please have have a look at http://repo.or.cz/w/elbb.git

The declared bill-board environment allows some
playing, even with git.


Andreas Röhler




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-03  0:39   ` Stefan Monnier
@ 2009-04-03 12:26     ` Alan Mackenzie
  2009-04-03 13:22       ` Stefan Monnier
  0 siblings, 1 reply; 28+ messages in thread
From: Alan Mackenzie @ 2009-04-03 12:26 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

Hi, Stefan!

On Thu, Apr 02, 2009 at 08:39:50PM -0400, Stefan Monnier wrote:
> > The following makes the current region invisible and intangible.  It

> I'll just remind people that I strongly discourage the use of
> the `intangible' property.  Your code should work well with just the
> `invisible' property; if not, it might be a bug.

I don't know exactly what `invisible' and `intangible' text/overlay
properties mean.  I don't know when to use each of them, when to use
both of them together.  The documentation is vague and unhelpful.  It
doesn't define these properties, instead merely stating _some_ of the
things they do.

For example, the documentation implies that if I isearch for something
inside the invisible bit, the cursor should end up on the other side of
the invisibility.  Instead, it does the Right Thing.

"Invisible" means you can't see something.  "Intangible" means you can't
touch it.  However, the `invisible' property has bits of intangibility
mixed in with it.  This makes it very confusing.

It seems though, at least in C Mode, that when a user makes part of his
buffer invisible, the invisible bits are still seen by the parsing
routines in CC Mode.  I don't know why this is so.

The documentation, (Elisp manual pages "Invisible Text" and "special
properties") is of no help here.  It says, in effect, "The invisible
property sort of does this".

Are there any complete descriptions of these properties anywhere?  Are
there any guidelines for there use anywhere?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-03 12:26     ` Alan Mackenzie
@ 2009-04-03 13:22       ` Stefan Monnier
  2009-04-06  6:11         ` Werner LEMBERG
  0 siblings, 1 reply; 28+ messages in thread
From: Stefan Monnier @ 2009-04-03 13:22 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: emacs-devel

> I don't know exactly what `invisible' and `intangible' text/overlay
> properties mean.

Basically:
- `invisible' means "not displayed".
- `intangible' means "can't put point in it (if you try, point will be
  put at the beginning or the end of the intangible text).
Not being able to put point in a piece of text has far reaching consequences,
e.g. (/= (1+ (point)) (save-excursion (forward-char 1) (point)))
so it tends to introduce bugs in all kinds of places.

The read-eval loop has a built-in post-command-hook (controlled by
disable-point-adjustment and global-disable-point-adjustment) that moves
point outside of `invisible' text (as well as outside of
char-compositions, and outside of images), so `invisible' text is
treated as somewhat intangible, except this kind of intangibility is
only applied after a command is done running, which introduces much
fewer problems than the `intangible' property which is applied at a much
lower level.

> I don't know when to use each of them, when to use both of
> them together.

If you don't know, then you probably don't want to use `intangible'.

> For example, the documentation implies that if I isearch for something
> inside the invisible bit, the cursor should end up on the other side of
> the invisibility.  Instead, it does the Right Thing.

Can you point me to the place that implies this behavior?

> It seems though, at least in C Mode, that when a user makes part of his
> buffer invisible, the invisible bits are still seen by the parsing
> routines in CC Mode.  I don't know why this is so.

Because `invisible' isn't like narrowing: it only prevents the display,
but doesn't remove the text.

> The documentation, (Elisp manual pages "Invisible Text" and "special
> properties") is of no help here.  It says, in effect, "The invisible
> property sort of does this".

> Are there any complete descriptions of these properties anywhere?  Are
> there any guidelines for there use anywhere?

Looks like the doc needs some improvement.  Thanks for pointing it out.


        Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-03 13:22       ` Stefan Monnier
@ 2009-04-06  6:11         ` Werner LEMBERG
  2009-04-06 13:10           ` Stefan Monnier
  2009-04-10 19:54           ` Werner LEMBERG
  0 siblings, 2 replies; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-06  6:11 UTC (permalink / raw)
  To: monnier; +Cc: acm, emacs-devel


> Basically:
> - `invisible' means "not displayed".
> - `intangible' means "can't put point in it (if you try, point will
>   be put at the beginning or the end of the intangible text).
> Not being able to put point in a piece of text has far reaching
> consequences, e.g. (/= (1+ (point)) (save-excursion (forward-char 1)
> (point))) so it tends to introduce bugs in all kinds of places.

However, I think this is exactly what I need -- thanks, Alan, for the
first try!

Consider this table:

  foo
  bar
  foo=bar
  bar-foo
  baz
  baz=baz
  baz-foo=baz

I would like to make all lines invisable *and* intangible which don't
contain a `='.  Otherwise interactive search-and-replace would try to
modify invisible text.  At least this is my conclusion of your
explanations.


    Werner




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-06  6:11         ` Werner LEMBERG
@ 2009-04-06 13:10           ` Stefan Monnier
  2009-04-06 16:20             ` Werner LEMBERG
  2009-04-10 19:54           ` Werner LEMBERG
  1 sibling, 1 reply; 28+ messages in thread
From: Stefan Monnier @ 2009-04-06 13:10 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: acm, emacs-devel

> I would like to make all lines invisable *and* intangible which don't
> contain a `='.  Otherwise interactive search-and-replace would try to
> modify invisible text.  At least this is my conclusion of your
> explanations.

No, the conclusion of my explanations is that search&replace need to be
changed to ignore (or reveal) invisible text, like isearch does.


        Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-06 13:10           ` Stefan Monnier
@ 2009-04-06 16:20             ` Werner LEMBERG
  2009-04-07  0:50               ` Stefan Monnier
  0 siblings, 1 reply; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-06 16:20 UTC (permalink / raw)
  To: monnier; +Cc: acm, emacs-devel

>> I would like to make all lines invisable *and* intangible which
>> don't contain a `='.  Otherwise interactive search-and-replace
>> would try to modify invisible text.  At least this is my conclusion
>> of your explanations.
>
> No, the conclusion of my explanations is that search&replace need to
> be changed to ignore (or reveal) invisible text, like isearch does.

And what do you advise me to do?  How can I do what I want without
setting the `intangible' property?


    Werner




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-06 16:20             ` Werner LEMBERG
@ 2009-04-07  0:50               ` Stefan Monnier
  2009-04-07  5:26                 ` Werner LEMBERG
  0 siblings, 1 reply; 28+ messages in thread
From: Stefan Monnier @ 2009-04-07  0:50 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: acm, emacs-devel

>>> I would like to make all lines invisable *and* intangible which
>>> don't contain a `='.  Otherwise interactive search-and-replace
>>> would try to modify invisible text.  At least this is my conclusion
>>> of your explanations.
>> No, the conclusion of my explanations is that search&replace need to
>> be changed to ignore (or reveal) invisible text, like isearch does.
> And what do you advise me to do?  How can I do what I want without
> setting the `intangible' property?

Send a patch ;-)


        Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-07  0:50               ` Stefan Monnier
@ 2009-04-07  5:26                 ` Werner LEMBERG
  0 siblings, 0 replies; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-07  5:26 UTC (permalink / raw)
  To: monnier; +Cc: acm, emacs-devel


>>>> I would like to make all lines invisable *and* intangible which
>>>> don't contain a `='.  Otherwise interactive search-and-replace
>>>> would try to modify invisible text.  At least this is my
>>>> conclusion of your explanations.
>>>
>>> No, the conclusion of my explanations is that search&replace need
>>> to be changed to ignore (or reveal) invisible text, like isearch
>>> does.
>>
>> And what do you advise me to do?  How can I do what I want without
>> setting the `intangible' property?
> 
> Send a patch ;-)

In other words, with current code I have to use `intangible', right?


    Werner




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-06  6:11         ` Werner LEMBERG
  2009-04-06 13:10           ` Stefan Monnier
@ 2009-04-10 19:54           ` Werner LEMBERG
  2009-04-10 20:42             ` Andreas Schwab
  1 sibling, 1 reply; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-10 19:54 UTC (permalink / raw)
  To: monnier; +Cc: acm, emacs-devel

[-- Attachment #1: Type: Text/Plain, Size: 648 bytes --]


> Consider this table:
> 
>   foo
>   bar
>   foo=bar
>   bar-foo
>   baz
>   baz=baz
>   baz-foo=baz
> 
> I would like to make all lines invisable *and* intangible which
> don't contain a `='.  Otherwise interactive search-and-replace would
> try to modify invisible text.  At least this is my conclusion of
> your explanations.

I've now come up with the code as shown in the attachment.  It works
fine, however, I would like to not have empty lines; in other words,
invisible text should not take any space in the buffer.  Does Emacs
supports such a `compressed invisibility'?  Otherwise it's getting
quite complicated, I fear...


    Werner


[-- Attachment #2: line-invisible.el --]
[-- Type: Text/Plain, Size: 904 bytes --]

(defun make-lines-invisible (regexp &optional arg)
  "Make all lines matching a regexp invisible and intangible.
With a prefix arg, make it visible again.  It is not necessary
that REGEXP matches the whole line; if a hit is found, the
affected line gets automatically selected.

This command affects the whole buffer."
  (interactive "MRegexp: \nP")
  (let (ov
	ovs)
    (cond
     ((equal arg '(4))
      (setq ovs (overlays-in (point-min) (point-max)))
      (mapc (lambda (o)
	      (if (overlay-get o 'make-lines-invisible)
		  (delete-overlay o)))
	    ovs))
     (t
      (save-excursion
	(goto-char (point-min))
	(while (re-search-forward regexp nil t)
	  (setq ov (make-overlay (line-beginning-position)
				 (line-end-position)))
	  (overlay-put ov 'make-lines-invisible t)
	  (overlay-put ov 'invisible t)
	  (overlay-put ov 'intangible t)))))))

(global-set-key "\C-cz" 'make-lines-invisible)

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-10 19:54           ` Werner LEMBERG
@ 2009-04-10 20:42             ` Andreas Schwab
  2009-04-10 21:50               ` Werner LEMBERG
  0 siblings, 1 reply; 28+ messages in thread
From: Andreas Schwab @ 2009-04-10 20:42 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: acm, monnier, emacs-devel

Werner LEMBERG <wl@gnu.org> writes:

> I've now come up with the code as shown in the attachment.  It works
> fine, however, I would like to not have empty lines; in other words,
> invisible text should not take any space in the buffer.  Does Emacs
> supports such a `compressed invisibility'?

Make the newline invisible.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: hiding lines
  2009-04-10 20:42             ` Andreas Schwab
@ 2009-04-10 21:50               ` Werner LEMBERG
  2009-04-10 22:42                 ` `make-overlay' very slow (was: hiding lines) Werner LEMBERG
  0 siblings, 1 reply; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-10 21:50 UTC (permalink / raw)
  To: schwab; +Cc: acm, monnier, emacs-devel


>> I've now come up with the code as shown in the attachment.  It
>> works fine, however, I would like to not have empty lines; in other
>> words, invisible text should not take any space in the buffer.
>> Does Emacs supports such a `compressed invisibility'?
> 
> Make the newline invisible.

Thanks!  This perhaps deserves a notice in the documentation.


    Werner




^ permalink raw reply	[flat|nested] 28+ messages in thread

* `make-overlay' very slow (was: hiding lines)
  2009-04-10 21:50               ` Werner LEMBERG
@ 2009-04-10 22:42                 ` Werner LEMBERG
  2009-04-10 23:36                   ` `make-overlay' very slow Stefan Monnier
  0 siblings, 1 reply; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-10 22:42 UTC (permalink / raw)
  To: schwab; +Cc: acm, monnier, emacs-devel

[-- Attachment #1: Type: Text/Plain, Size: 945 bytes --]


>>> I've now come up with the code as shown in the attachment.  It
>>> works fine, however, I would like to not have empty lines; in
>>> other words, invisible text should not take any space in the
>>> buffer.  Does Emacs supports such a `compressed invisibility'?
>>
>> 
>> Make the newline invisible.
> 
> Thanks!  This perhaps deserves a notice in the documentation.

I've now tried the code with my real-world table (containing
approx. 420000 lines):

  git://repo.or.cz/srv/git/wortliste.git

and I found out that it is, alas, unusable.  It starts with about 1000
replacements per second on my GNU/Linux box (using simply `-' as the
regexp), but soon the speed decreases: After approx. 10000
replacements it only can handle 100 replacements per second -- the
attached code now contains a small counter to show the progress.  If
you comment out the call to `make-overlay', the code really flies...

Am I doing something wrong?


    Werner

[-- Attachment #2: line-invisible.el --]
[-- Type: Text/Plain, Size: 1053 bytes --]

(defun make-lines-invisible (regexp &optional arg)
  "Make all lines matching a regexp invisible and intangible.
With a prefix arg, make it visible again.  It is not necessary
that REGEXP matches the whole line; if a hit is found, the
affected line gets automatically selected.

This command affects the whole buffer."
  (interactive "MRegexp: \nP")
  (let (ov
	ovs
	count)
    (cond
     ((equal arg '(4))
      (setq ovs (overlays-in (point-min) (point-max)))
      (mapc (lambda (o)
	      (if (overlay-get o 'make-lines-invisible)
		  (delete-overlay o)))
	    ovs))
     (t
      (save-excursion
	(goto-char (point-min))
	(setq count 0)
	(while (re-search-forward regexp nil t)
	  (setq count (1+ count))
	  (if (= (% count 100) 0)
	      (message "%d" count))
	  (setq ov (make-overlay (line-beginning-position)
				 (1+ (line-end-position))))
;	  (overlay-put ov 'make-lines-invisible t)
;	  (overlay-put ov 'invisible t)
;	  (overlay-put ov 'intangible t)
	  (goto-char (line-end-position))))))))

(global-set-key "\C-cz" 'make-lines-invisible)

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-10 22:42                 ` `make-overlay' very slow (was: hiding lines) Werner LEMBERG
@ 2009-04-10 23:36                   ` Stefan Monnier
  2009-04-11  6:11                     ` Werner LEMBERG
  0 siblings, 1 reply; 28+ messages in thread
From: Stefan Monnier @ 2009-04-10 23:36 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: acm, schwab, emacs-devel

> and I found out that it is, alas, unusable.  It starts with about 1000
> replacements per second on my GNU/Linux box (using simply `-' as the
> regexp), but soon the speed decreases: After approx. 10000
> replacements it only can handle 100 replacements per second -- the
> attached code now contains a small counter to show the progress.  If
> you comment out the call to `make-overlay', the code really flies...

> Am I doing something wrong?

overlays have a poor algorithmic behavior (many operations take a time
proportional to the number of overlays in the buffer).  Better use
text-properties (which are implemented with a tree and should provide
something closer to O(log N) complexity instead).


        Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-10 23:36                   ` `make-overlay' very slow Stefan Monnier
@ 2009-04-11  6:11                     ` Werner LEMBERG
  2009-04-11  7:34                       ` Lennart Borgman
                                         ` (2 more replies)
  0 siblings, 3 replies; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-11  6:11 UTC (permalink / raw)
  To: monnier; +Cc: acm, schwab, emacs-devel

[-- Attachment #1: Type: Text/Plain, Size: 691 bytes --]

> overlays have a poor algorithmic behavior (many operations take a
> time proportional to the number of overlays in the buffer).  Better
> use text-properties (which are implemented with a tree and should
> provide something closer to O(log N) complexity instead).

I did that (see attachment for reference), and it really works at a
reasonable speed.  However, it is the completely wrong concept since
it sets the `modified' flag and stores undo information, and it fails
with modes which use the `invisible' and `intangible' properties.

So here's my item to the Emacs wishlist: Improve speed of
(make-overlay) and related functions to be as fast as setting text
properties.


    Werner

[-- Attachment #2: line-invisible-text-properties.el --]
[-- Type: Text/Plain, Size: 1516 bytes --]

(defun make-lines-invisible (regexp &optional arg)
  "Make all lines matching a regexp invisible and intangible.
With a prefix arg, make them visible again.  It is not necessary
that REGEXP matches the whole line; if a hit is found, the
affected line gets automatically selected.

This function affects the whole buffer.

Note that this function modifies the `invisible' and `intangible'
text properties; it may thus interfere with modes which use them.
Due to implementation restrictions in current Emacs versions it
is not possible to use overlays -- which would avoid text
property modifications -- without becoming unbearably slow for
large buffers with many matches."
  (interactive "MRegexp: \nP")
  (save-excursion
    (cond
     (arg
      (let ((next-pos (point-min)))
	(while (setq next-pos
		     (text-property-any next-pos
					(point-max)
					'make-lines-invisible
					t))
	  (goto-char next-pos)
	  (setq next-pos (or (next-single-property-change
			      (point)
			      'make-lines-invisible)
			     (point-max)))
	  (remove-list-of-text-properties (point)
					  next-pos
					  '(make-lines-invisible
					    invisible
					    intangible)))))
    (t
     (goto-char (point-min))
     (while (re-search-forward regexp nil t)
       (add-text-properties (line-beginning-position)
			    (1+ (line-end-position)) ; handle \n
			    '(make-lines-invisible t
			      invisible t
			      intangible t))
       (goto-char (line-end-position)))))))

(global-set-key "\C-cz" 'make-lines-invisible)

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-11  6:11                     ` Werner LEMBERG
@ 2009-04-11  7:34                       ` Lennart Borgman
  2009-04-11  7:45                         ` Werner LEMBERG
  2009-04-11  8:55                       ` Eli Zaretskii
  2009-04-11 12:51                       ` Stefan Monnier
  2 siblings, 1 reply; 28+ messages in thread
From: Lennart Borgman @ 2009-04-11  7:34 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: acm, schwab, monnier, emacs-devel

On Sat, Apr 11, 2009 at 8:11 AM, Werner LEMBERG <wl@gnu.org> wrote:
>> overlays have a poor algorithmic behavior (many operations take a
>> time proportional to the number of overlays in the buffer).  Better
>> use text-properties (which are implemented with a tree and should
>> provide something closer to O(log N) complexity instead).
>
> I did that (see attachment for reference), and it really works at a
> reasonable speed.  However, it is the completely wrong concept since
> it sets the `modified' flag and stores undo information,

There is a macro in font-lock.el that you can use to modify text
properties without storing undo information or setting the modified
flag.

> and it fails
> with modes which use the `invisible' and `intangible' properties.




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-11  7:34                       ` Lennart Borgman
@ 2009-04-11  7:45                         ` Werner LEMBERG
  2009-04-11  7:46                           ` Lennart Borgman
  0 siblings, 1 reply; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-11  7:45 UTC (permalink / raw)
  To: lennart.borgman; +Cc: acm, schwab, monnier, emacs-devel


>> I did that (see attachment for reference), and it really works at a
>> reasonable speed.  However, it is the completely wrong concept
>> since it sets the `modified' flag and stores undo information,
> 
> There is a macro in font-lock.el that you can use to modify text
> properties without storing undo information or setting the modified
> flag.

Thanks, will update my code.  However, it isn't a general solution
since my function should essentially work with all modes.


    Werner




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-11  7:45                         ` Werner LEMBERG
@ 2009-04-11  7:46                           ` Lennart Borgman
  2009-04-11  8:33                             ` Werner LEMBERG
  0 siblings, 1 reply; 28+ messages in thread
From: Lennart Borgman @ 2009-04-11  7:46 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: acm, schwab, monnier, emacs-devel

On Sat, Apr 11, 2009 at 9:45 AM, Werner LEMBERG <wl@gnu.org> wrote:
>
>>> I did that (see attachment for reference), and it really works at a
>>> reasonable speed.  However, it is the completely wrong concept
>>> since it sets the `modified' flag and stores undo information,
>>
>> There is a macro in font-lock.el that you can use to modify text
>> properties without storing undo information or setting the modified
>> flag.
>
> Thanks, will update my code.  However, it isn't a general solution
> since my function should essentially work with all modes.

The macro does not depend on font-lock and you have to copy it to your code.




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-11  7:46                           ` Lennart Borgman
@ 2009-04-11  8:33                             ` Werner LEMBERG
  0 siblings, 0 replies; 28+ messages in thread
From: Werner LEMBERG @ 2009-04-11  8:33 UTC (permalink / raw)
  To: lennart.borgman; +Cc: acm, schwab, monnier, emacs-devel


>> Thanks, will update my code.  However, it isn't a general solution
>> since my function should essentially work with all modes.
> 
> The macro does not depend on font-lock and you have to copy it to
> your code.

There's a misunderstanding: I mean the overlay vs. text properties
problem.  `save-buffer-state' works just fine.


    Werner




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-11  6:11                     ` Werner LEMBERG
  2009-04-11  7:34                       ` Lennart Borgman
@ 2009-04-11  8:55                       ` Eli Zaretskii
  2009-04-11 12:51                       ` Stefan Monnier
  2 siblings, 0 replies; 28+ messages in thread
From: Eli Zaretskii @ 2009-04-11  8:55 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: emacs-devel

> Date: Sat, 11 Apr 2009 08:11:50 +0200 (CEST)
> From: Werner LEMBERG <wl@gnu.org>
> Cc: acm@muc.de, schwab@linux-m68k.org, emacs-devel@gnu.org
> 
> So here's my item to the Emacs wishlist: Improve speed of
> (make-overlay) and related functions to be as fast as setting text
> properties.

I suggest filing a ``wishlist'' bug report about this.




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-11  6:11                     ` Werner LEMBERG
  2009-04-11  7:34                       ` Lennart Borgman
  2009-04-11  8:55                       ` Eli Zaretskii
@ 2009-04-11 12:51                       ` Stefan Monnier
  2009-04-14 12:03                         ` Kenichi Handa
  2 siblings, 1 reply; 28+ messages in thread
From: Stefan Monnier @ 2009-04-11 12:51 UTC (permalink / raw)
  To: Werner LEMBERG; +Cc: acm, schwab, emacs-devel

> I did that (see attachment for reference), and it really works at a
> reasonable speed.

Good.

> However, it is the completely wrong concept since
> it sets the `modified' flag and stores undo information,

Yes, you need to wrap the call to > add-text-properties (actually the
whole loop, preferably) in something like

   (let ((buffer-file-name nil) ;; To prevent locking/unlocking the file.
         (buffer-undo-list t)
         (modified (buffer-modified-p)))
     (unwind-protect
         (...)
       (restore-modified-p modified)))

We definitely need to add a macro for it.

> and it fails with modes which use the `invisible' and
> `intangible' properties.

Since these properties are not used very often, it's probably bearable,
but yes, there is a risk of interference.

> So here's my item to the Emacs wishlist: Improve speed of
> (make-overlay) and related functions to be as fast as setting
> text properties.

Yes, that would be great.  But note that it's not just `make-overlay':
every time we make a modification to the buffer, we have to update the
position of all the overlays (and markers) after point.  So, yes,
a better data-structure for overlays (and markers) would be very welcome.


        Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-11 12:51                       ` Stefan Monnier
@ 2009-04-14 12:03                         ` Kenichi Handa
  2009-04-14 13:03                           ` Stefan Monnier
  2009-04-16 15:02                           ` clone-indirect-buffer-hook should be make-indirect-buffer-hook klaus.berndl
  0 siblings, 2 replies; 28+ messages in thread
From: Kenichi Handa @ 2009-04-14 12:03 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: acm, schwab, emacs-devel

In article <jwvr5zz1hzq.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Yes, that would be great.  But note that it's not just `make-overlay':
> every time we make a modification to the buffer, we have to update the
> position of all the overlays (and markers) after point.  So, yes,
> a better data-structure for overlays (and markers) would be very welcome.

I have not yet thought about this idea in deep but perhaps
we can use one more interval tree for overlays.

More radical idea, not related to overlays, is to make one
interval tree for one text property; i.e. one for `face',
one for `fontified'...  I think it not only improves the
processing speed, but also reduces the memory usage
(next-property-change will get slower, but we use
next-single-property-change more often).

---
Kenichi Handa
handa@m17n.org




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-14 12:03                         ` Kenichi Handa
@ 2009-04-14 13:03                           ` Stefan Monnier
  2009-04-14 21:15                             ` Richard M Stallman
  2009-04-15  3:56                             ` Kenichi Handa
  2009-04-16 15:02                           ` clone-indirect-buffer-hook should be make-indirect-buffer-hook klaus.berndl
  1 sibling, 2 replies; 28+ messages in thread
From: Stefan Monnier @ 2009-04-14 13:03 UTC (permalink / raw)
  To: Kenichi Handa; +Cc: acm, schwab, emacs-devel

>> Yes, that would be great.  But note that it's not just
>> `make-overlay': every time we make a modification to the buffer, we
>> have to update the position of all the overlays (and markers) after
>> point.  So, yes, a better data-structure for overlays (and markers)
>> would be very welcome.

> I have not yet thought about this idea in deep but perhaps
> we can use one more interval tree for overlays.

Yes, to speed up overlays algorithmically, we'd want to implement
them with some kind of tree.  If we could share some of the code with
the text properties's interval tree that would be great, indeed.

> More radical idea, not related to overlays, is to make one
> interval tree for one text property; i.e. one for `face',
> one for `fontified'...  I think it not only improves the
> processing speed, but also reduces the memory usage
> (next-property-change will get slower, but we use
> next-single-property-change more often).

I've been tempted as well, but I'm not sure it'd be much of
an improvement.  You'd end up with many trees, so might need
a hash-table or some other tree to map text-property names to the
corresponding interval tree.


        Stefan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-14 13:03                           ` Stefan Monnier
@ 2009-04-14 21:15                             ` Richard M Stallman
  2009-04-15  3:56                             ` Kenichi Handa
  1 sibling, 0 replies; 28+ messages in thread
From: Richard M Stallman @ 2009-04-14 21:15 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: acm, emacs-devel, schwab, handa

When you access overlays in nonrandom order, the speed is very much
affected by where the "center" is.  Calling `overlay-recenter'
at suitable times might cause a big speedup.




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: `make-overlay' very slow
  2009-04-14 13:03                           ` Stefan Monnier
  2009-04-14 21:15                             ` Richard M Stallman
@ 2009-04-15  3:56                             ` Kenichi Handa
  1 sibling, 0 replies; 28+ messages in thread
From: Kenichi Handa @ 2009-04-15  3:56 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: acm, schwab, emacs-devel

In article <jwvprfftn08.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > More radical idea, not related to overlays, is to make one
> > interval tree for one text property; i.e. one for `face',
> > one for `fontified'...  I think it not only improves the
> > processing speed, but also reduces the memory usage
> > (next-property-change will get slower, but we use
> > next-single-property-change more often).

> I've been tempted as well, but I'm not sure it'd be much of
> an improvement.  You'd end up with many trees, so might need
> a hash-table or some other tree to map text-property names to the
> corresponding interval tree.

I think the number of properties is not that much in normal
use and using just `assq' is sufficient.  I seldom see more
than 8 kinds of properties.  And, we can move the lastly
accessed interval tree to the head of interval tree list if
speed up is really necessary.

---
Kenichi Handa
handa@m17n.org




^ permalink raw reply	[flat|nested] 28+ messages in thread

* clone-indirect-buffer-hook should be make-indirect-buffer-hook
  2009-04-14 12:03                         ` Kenichi Handa
  2009-04-14 13:03                           ` Stefan Monnier
@ 2009-04-16 15:02                           ` klaus.berndl
  1 sibling, 0 replies; 28+ messages in thread
From: klaus.berndl @ 2009-04-16 15:02 UTC (permalink / raw)
  To: emacs-devel

Hi all,

AFAICS the new hook `clone-indirect-buffer-hook' of Emacs 23 is not used by the command `make-indirect-buffer' or is it? If not: why not?

IMHO a bug in Emacs 23 because if there is such a hook, then it should be used by both of them - ideally only by `make-indirect-buffer' (because this command is internally called by `clone-indirect-buffer') and then the new hook should also be renamed to `make-indirect-buffer-hook'.

Thoughts?

Regards
Klaus




^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2009-04-16 15:02 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-04-02  6:52 hiding lines Werner LEMBERG
2009-04-02 19:30 ` Alan Mackenzie
2009-04-03  0:39   ` Stefan Monnier
2009-04-03 12:26     ` Alan Mackenzie
2009-04-03 13:22       ` Stefan Monnier
2009-04-06  6:11         ` Werner LEMBERG
2009-04-06 13:10           ` Stefan Monnier
2009-04-06 16:20             ` Werner LEMBERG
2009-04-07  0:50               ` Stefan Monnier
2009-04-07  5:26                 ` Werner LEMBERG
2009-04-10 19:54           ` Werner LEMBERG
2009-04-10 20:42             ` Andreas Schwab
2009-04-10 21:50               ` Werner LEMBERG
2009-04-10 22:42                 ` `make-overlay' very slow (was: hiding lines) Werner LEMBERG
2009-04-10 23:36                   ` `make-overlay' very slow Stefan Monnier
2009-04-11  6:11                     ` Werner LEMBERG
2009-04-11  7:34                       ` Lennart Borgman
2009-04-11  7:45                         ` Werner LEMBERG
2009-04-11  7:46                           ` Lennart Borgman
2009-04-11  8:33                             ` Werner LEMBERG
2009-04-11  8:55                       ` Eli Zaretskii
2009-04-11 12:51                       ` Stefan Monnier
2009-04-14 12:03                         ` Kenichi Handa
2009-04-14 13:03                           ` Stefan Monnier
2009-04-14 21:15                             ` Richard M Stallman
2009-04-15  3:56                             ` Kenichi Handa
2009-04-16 15:02                           ` clone-indirect-buffer-hook should be make-indirect-buffer-hook klaus.berndl
2009-04-03  5:44   ` hiding lines Andreas Roehler

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).