Feature proposal/request: Indentation driven by display engine

Feature proposal/request: Indentation driven by display engine

[Carsten Dominik]

Hi,

I have a proposal to make:

Recently, I have experimented with an idea, to create
indentation on the Emacs display.

For example, in an outline structure

* level 1
text under level 1
** Level 2
text under level 1


I would like to display this as:

* level 1
   text under level 1
** Level 2
    text under level 1

without actually inserting white space at the beginning of
each line.  So effectively I would like to shift the location
of the left margin on a per-line basis.

I have been trying to implement this using display properties
on each newline characters, displaying "\n   " instead of
just "\n".  This basically works, but it interacts badly
with outlining:  If I fold the text below a headline, the
first indentation is still displayed.

The next thing I tried was using overlays over each "\n",
with an after-string property carrying the indentations.
This works correctly with outline, but when I add thousands
of overlays to a buffer this becomes slow and redisplay and
editing become sluggish, probably because to the huge
amount of markers in the buffer.

It then occurred to me that it might be useful to support
such a feature directly in the display engine.
For example, if the line contains text with an `indentation'
property, the display engine would add this amount of white
space to before the beginning of the line, maybe also a vertical
line indicating the location of the margin.

Would that be something useful to add?  Anyone who would
volunteer to implement this, because I frankly have no
idea how the display engine works internally.

Or are there other comments, for example how this dynamic
indentation could be implemented in a different way?

Thanks for listening.

- Carsten

Re: Feature proposal/request: Indentation driven by display engine

[Stefan Monnier]

> I have been trying to implement this using display properties
> on each newline characters, displaying "\n   " instead of
> just "\n".  This basically works, but it interacts badly
> with outlining:  If I fold the text below a headline, the
> first indentation is still displayed.

Can't you remove the display property when you fold to avoid the problem?

> The next thing I tried was using overlays over each "\n",
> with an after-string property carrying the indentations.
> This works correctly with outline, but when I add thousands
> of overlays to a buffer this becomes slow and redisplay and
> editing become sluggish, probably because to the huge
> amount of markers in the buffer.

Yes, we need to improve the implementation of markers to eliminate the
O(N) complexity.  Ideally, there'd be some clever way to store the
markers directly inside the text-properties tree.

> It then occurred to me that it might be useful to support
> such a feature directly in the display engine.
> For example, if the line contains text with an `indentation'
> property, the display engine would add this amount of white
> space to before the beginning of the line, maybe also a vertical
> line indicating the location of the margin.

This property would still need to be added to every line.  So basically
all you're asking is a "before-string" text-property, maybe?

> Or are there other comments, for example how this dynamic
> indentation could be implemented in a different way?

How 'bout modifying the actual buffer text directly?


        Stefan

Re: Feature proposal/request: Indentation driven by display engine

[Carsten Dominik]

Hi Stefan, thanks for your reply.

On May 24, 2008, at 8:48 PM, Stefan Monnier wrote:

>> I have been trying to implement this using display properties
>> on each newline characters, displaying "\n   " instead of
>> just "\n".  This basically works, but it interacts badly
>> with outlining:  If I fold the text below a headline, the
>> first indentation is still displayed.
>
> Can't you remove the display property when you fold to avoid the  
> problem?

I could, but then I would have to add and remove text properties during
each fold/unfold, quite some unnecessary overhead.

>> The next thing I tried was using overlays over each "\n",
>> with an after-string property carrying the indentations.
>> This works correctly with outline, but when I add thousands
>> of overlays to a buffer this becomes slow and redisplay and
>> editing become sluggish, probably because to the huge
>> amount of markers in the buffer.
>
> Yes, we need to improve the implementation of markers to eliminate the
> O(N) complexity.  Ideally, there'd be some clever way to store the
> markers directly inside the text-properties tree.

Yes, it would certainly be nice to get rid of this old problem.

>
>
>> It then occurred to me that it might be useful to support
>> such a feature directly in the display engine.
>> For example, if the line contains text with an `indentation'
>> property, the display engine would add this amount of white
>> space to before the beginning of the line, maybe also a vertical
>> line indicating the location of the margin.
>
> This property would still need to be added to every line.  So  
> basically
> all you're asking is a "before-string" text-property, maybe?

Hmm, nice translation of my request.  Yes, basically what I am
asking for is a working before-string text property.  I know
that adding the property sounds like a lot of effort, but I could
use the font-lock/jit-lock setup to get this done efficiently,
only for the pieces that actually get displayed.

>> Or are there other comments, for example how this dynamic
>> indentation could be implemented in a different way?
>
> How 'bout modifying the actual buffer text directly?

Yes, this is in fact what I am doing now, I use TAB, and
wrapping/paragraph-fill support to get fill prefixes.  I also modify
indentation during promotion and demotion of entries.

However, there are some disadvantages to this.  One obvious one is
this:  Org contains a publishing part, and for publishing
we allow examples like code snippets, where indentation might be
important.  So the

#+BEGIN_EXAMPLE

#+END_EXAMPLE

construct should be flush to the left margin.

Are we going to get a before-string property?  WOuld be cool.

- Carsten













>
>
>
>        Stefan

Re: Feature proposal/request: Indentation driven by display engine

[Stephen Berman]

On Sat, 24 May 2008 14:15:59 +0200 Carsten Dominik <carsten.dominik@gmail.com> wrote:

> Hi,
>
> I have a proposal to make:
>
> Recently, I have experimented with an idea, to create
> indentation on the Emacs display.
>
> For example, in an outline structure
>
> * level 1
> text under level 1
> ** Level 2
> text under level 1
>
>
> I would like to display this as:
>
> * level 1
>   text under level 1
> ** Level 2
>    text under level 1
>
> without actually inserting white space at the beginning of
> each line.  So effectively I would like to shift the location
> of the left margin on a per-line basis.
>
> I have been trying to implement this using display properties
> on each newline characters, displaying "\n   " instead of
> just "\n".  This basically works, but it interacts badly
> with outlining:  If I fold the text below a headline, the
> first indentation is still displayed.

This sounds reminiscent of a bug I reported a while ago; see
<http://permalink.gmane.org/gmane.emacs.devel/86019>.  

> The next thing I tried was using overlays over each "\n",
> with an after-string property carrying the indentations.
> This works correctly with outline, but when I add thousands
> of overlays to a buffer this becomes slow and redisplay and
> editing become sluggish, probably because to the huge
> amount of markers in the buffer.
>
> It then occurred to me that it might be useful to support
> such a feature directly in the display engine.
> For example, if the line contains text with an `indentation'
> property, the display engine would add this amount of white
> space to before the beginning of the line, maybe also a vertical
> line indicating the location of the margin.
>
> Would that be something useful to add?  Anyone who would
> volunteer to implement this, because I frankly have no
> idea how the display engine works internally.
>
> Or are there other comments, for example how this dynamic
> indentation could be implemented in a different way?

I've attempted to develop a version of longlines-mode that implements
the same sort of left margin shifting you mention, also by means of a
similar display property (I use " \n" concatented with a customizable
string).  It works fairly well (also with adaptive filling, which does
not work with current longlines-mode) but among other problems suffers
from the display bug I referred to in the above URL, which sounds like
what you are also seeing.  In an earlier version I tried using
before-string overlays and also ran into scalability problems.  I don't
know if a before-string property would solve all issues, but it would be
nice to have the possibility.

Steve Berman

Re: Feature proposal/request: Indentation driven by display engine

[Stefan Monnier]

>> Can't you remove the display property when you fold to avoid the problem?
> I could, but then I would have to add and remove text properties during
> each fold/unfold, quite some unnecessary overhead.

Not sure how much of an overhead that would be.  Doesn't seem major.
It might be inconvenient to code, tho.  Is that what you meant?

>>> It then occurred to me that it might be useful to support
>>> such a feature directly in the display engine.
>>> For example, if the line contains text with an `indentation'
>>> property, the display engine would add this amount of white
>>> space to before the beginning of the line, maybe also a vertical
>>> line indicating the location of the margin.
>> 
>> This property would still need to be added to every line.  So basically
>> all you're asking is a "before-string" text-property, maybe?

> Hmm, nice translation of my request.  Yes, basically what I am
> asking for is a working before-string text property.  I know
> that adding the property sounds like a lot of effort, but I could
> use the font-lock/jit-lock setup to get this done efficiently,
> only for the pieces that actually get displayed.

The same has been requested for display properties (e.g. to put
something in the margin without hiding any of the current text, you
need an overlay).

A `before-string' text-property might be doable, but I couldn't tell you
how easy it'd be to add.  If someone wants to implement that, I might
accept it (e.g., as a form of `display' property), but if I were you
I wouldn't hold my breath (we have enough bugs to fix and are getting
close to feature freeze).

The best I can offer for now, is to use overlays which you add via
jit-lock and which you eagerly remove in the background (so they don't
accumulate).

>> How 'bout modifying the actual buffer text directly?

> Yes, this is in fact what I am doing now, I use TAB, and
> wrapping/paragraph-fill support to get fill prefixes.  I also modify
> indentation during promotion and demotion of entries.

> However, there are some disadvantages to this.  One obvious one is
> this:  Org contains a publishing part, and for publishing
> we allow examples like code snippets, where indentation might be
> important.  So the

> #+BEGIN_EXAMPLE

> #+END_EXAMPLE

> construct should be flush to the left margin.

I'm not sure I understand why that makes any difference.

> Are we going to get a before-string property?  WOuld be cool.

Well... patches welcome,


        Stefan

Re: Feature proposal/request: Indentation driven by display engine

[Carsten Dominik]

Hi Stafan,

On May 26, 2008, at 11:02 PM, Stefan Monnier wrote:

>>> Can't you remove the display property when you fold to avoid the  
>>> problem?
>> I could, but then I would have to add and remove text properties  
>> during
>> each fold/unfold, quite some unnecessary overhead.
>
> Not sure how much of an overhead that would be.  Doesn't seem major.
> It might be inconvenient to code, tho.  Is that what you meant?

Yes, mainly this.  There are many show/hide commands, some in outline  
mode, some in Org, I would need to make sure that each time the right  
thing happens.


>
>
>>>> It then occurred to me that it might be useful to support
>>>> such a feature directly in the display engine.
>>>> For example, if the line contains text with an `indentation'
>>>> property, the display engine would add this amount of white
>>>> space to before the beginning of the line, maybe also a vertical
>>>> line indicating the location of the margin.
>>>
>>> This property would still need to be added to every line.  So  
>>> basically
>>> all you're asking is a "before-string" text-property, maybe?
>
>> Hmm, nice translation of my request.  Yes, basically what I am
>> asking for is a working before-string text property.  I know
>> that adding the property sounds like a lot of effort, but I could
>> use the font-lock/jit-lock setup to get this done efficiently,
>> only for the pieces that actually get displayed.
>
> The same has been requested for display properties (e.g. to put
> something in the margin without hiding any of the current text, you
> need an overlay).
>
> A `before-string' text-property might be doable, but I couldn't tell  
> you
> how easy it'd be to add.  If someone wants to implement that, I might
> accept it (e.g., as a form of `display' property), but if I were you
> I wouldn't hold my breath (we have enough bugs to fix and are getting
> close to feature freeze).

Sure, I appreciate that.

>
>
> The best I can offer for now, is to use overlays which you add via
> jit-lock and which you eagerly remove in the background (so they don't
> accumulate).


Yes, one possible path.
>
> Well... patches welcome,

In this case, not from me, not clue where to even start.

Thanks.

- Carsten