Re: Scrollbar thumbs

Re: Scrollbar thumbs

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>         I also find the scrollbar somewhat confusing: When the end of
>>  buffer is visible in the window, I would expect the scrollbar to reach
>>  all the way to the bottom, but it does not.
>
> It does with some GUI and not with others.  In principle it should.
> The only reason for it not to is to try and work around problems when
> dragging the scrollbar thumb, where many GUI toolkits are absurdly
> dogmatic and make it pretty much impossible for Emacs to do what
> it should.

Is it only GTK which is brain-damaged in this area?  Do other toolkits
do the same thing?

-Miles

-- 
Zeal, n. A certain nervous disorder afflicting the young and inexperienced.

Re: Scrollbar thumbs

[Stefan Monnier]

> Is it only GTK which is brain-damaged in this area?  Do other toolkits
> do the same thing?

Yes, IIUC w32 does the same (and Xaw3d as well, tho to a lesser extent).


        Stefan

Re: Scrollbar thumbs

[Jason Rumney]

Stefan Monnier wrote:
>> Is it only GTK which is brain-damaged in this area?  Do other toolkits
>> do the same thing?
>>     
>
> Yes, IIUC w32 does the same (and Xaw3d as well, tho to a lesser extent).
>   

Since 23.1, Emacs on w32 allows the scroll thumb to shrink while being 
dragged at the bottom of the buffer.  Previous versions fixed the size 
of the thumb while dragging, which I think is the brain damage which GTK 
forces on you that Miles is refering to, as you are unable to reach the 
bottom of the buffer by dragging the scrollbar in some circumstances 
because of assumptions Emacs makes to relate characters to pages of 
scrolling without wasting time doing a full calculation over the buffer.

Re: Scrollbar thumbs

[Miles Bader]

"grischka" <grishka@gmx.de> writes:
> I guess it is just that these GUI toolkit people usually are trained to
> deliver good looking and pixel-precise artwork and in consequence might
> have some difficulties to imagine how it can be "pretty much impossibe"
> for some editor to know how many lines it is actually displaying (or
> anyway that such editor would still want to use their scrollbars).

Emacs _does_ know how many lines it's displaying.

What it doesn't know is how many lines are in the entire file, or more
precisely, the "height" of the entire file as rendered, which
essentially requires rendering the _entire thing_ (it can't be
calculated from logical number of lines because it depends on the exact
characters rendered).  That's a pretty nasty requirement for a non-toy
editor...

-Miles

-- 
Consult, v.i. To seek another's disapproval of a course already decided on.

Re: Scrollbar thumbs

[Stefan Monnier]

> Since 23.1, Emacs on w32 allows the scroll thumb to shrink while being
> dragged at the bottom of the buffer.  Previous versions fixed the size of
> the thumb while dragging, which I think is the brain damage which GTK forces
> on you that Miles is refering to, as you are unable to reach the bottom of
> the buffer by dragging the scrollbar in some circumstances because of
> assumptions Emacs makes to relate characters to pages of scrolling without
> wasting time doing a full calculation over the buffer.

Not sure how it works on w32, but yes on Gtk IIRC the thumb can't be
resized during dragging.  In Xaw3d the problem was halfway: the code
was/is able to resize the thumb during a drag, but the thumb may hit the
bottom too soon, in case we haven't resized it quickly enough.

My favorite solution to those problem nowadays is to set the thumb-size
to its minimum (typically 0, which results in a small but no invisible
thumb) during dragging: makes for smooth scrolling (without the need to
resize the thumb during scrolling) while still allowing the thumb size
to correctly reflect the begin&end of the viewed area when
not scrolling.


        Stefan

Re: Scrollbar thumbs

[Stefan Monnier]

> I guess it is just that these GUI toolkit people usually are trained to
> deliver good looking and pixel-precise artwork and in consequence might have
> some difficulties to imagine how it can be "pretty much impossibe" for some
> editor to know how many lines it is actually displaying (or anyway that such
> editor would still want to use their scrollbars).

No, sadly, it is really due to dogma: your above explanation would make
perfect sense, but in practice, even after showing them our problem,
they still insist that it would be wrong for them to remove the chunks
of code that enforce the "scrollbar metaphor" by preventing the client
from getting the events that indicate that the user wants to move the
thumb's bottom further down than the scrollbar's own bottom.


        Stefan

Re: Scrollbar thumbs

[grischka]

>> I guess it is just that these GUI toolkit people usually are trained to
>> deliver good looking and pixel-precise artwork and in consequence might have
>> some difficulties to imagine how it can be "pretty much impossibe" for some
>> editor to know how many lines it is actually displaying (or anyway that such
>> editor would still want to use their scrollbars).
> 
> No, sadly, it is really due to dogma: your above explanation would make
> perfect sense, but in practice, even after showing them our problem,
> they still insist that it would be wrong for them to remove the chunks
> of code that enforce the "scrollbar metaphor" by preventing the client
> from getting the events that indicate that the user wants to move the
> thumb's bottom further down than the scrollbar's own bottom.

I don't see any dogma beyond the dogma of consistent design which is not 
something a toolkit designer can be blamed for.

Also by saying "our problem" you already realized that is your problem 
and not theirs.  In a free world, why should they cripple their code 
just to save you from having to find the right solution within your own
program.  That would be two times wrong.

--- grischka

Re: Scrollbar thumbs

[grischka]

>> I guess it is just that these GUI toolkit people usually are trained to
>> deliver good looking and pixel-precise artwork and in consequence might
>> have some difficulties to imagine how it can be "pretty much impossibe"
>> for some editor to know how many lines it is actually displaying (or
>> anyway that such editor would still want to use their scrollbars).
> 
> Emacs _does_ know how many lines it's displaying.
> 
> What it doesn't know is how many lines are in the entire file, or more
> precisely, the "height" of the entire file as rendered, which
> essentially requires rendering the _entire thing_ (it can't be
> calculated from logical number of lines because it depends on the exact
> characters rendered).  That's a pretty nasty requirement for a non-toy
> editor...

The simple rule is:  First do it right, then do it fast.  Try to make
it work for small files and then you can still think about how to get
equally good results with larger ones.

--- grischka

Re: Scrollbar thumbs

[Stefan Monnier]

> I don't see any dogma beyond the dogma of consistent design which is not
> something a toolkit designer can be blamed for.

It has nothing to do with consistent design.

> Also by saying "our problem" you already realized that is your problem and
> not theirs.  In a free world, why should they cripple their code just to
> save you from having to find the right solution within your own
> program.  That would be two times wrong.

We're not talking about crippling anybody's code other than ours.
They specifically add code which has no effect for 99% of the clients
and which prevents Emacs from working correctly.  Removing that code
wouldn't cripple anyone.


        Stefan

Re: Scrollbar thumbs

[grischka]

>> I don't see any dogma beyond the dogma of consistent design which is not
>> something a toolkit designer can be blamed for.
> 
> It has nothing to do with consistent design.

Surely it has.

>> Also by saying "our problem" you already realized that is your problem and
>> not theirs.  In a free world, why should they cripple their code just to
>> save you from having to find the right solution within your own
>> program.  That would be two times wrong.
> 
> We're not talking about crippling anybody's code other than ours.
> They specifically add code which has no effect for 99% of the clients
> and which prevents Emacs from working correctly.  Removing that code
> wouldn't cripple anyone.

Well, extra code that prevents clients from doing unwanted things exists 
all over the place in APIs and the point of such code is that it has no 
effect under normal usage.

And the consistency aspect of "normal usage" being defined is simply 
that widgets are there to provide a consistent look & feel across 
applications.  It seems inconsequent if on one hand you want to use 
a widget but on the other hand want to use it differently to 99% of 
the other clients.  Then you'd better invent your own.

--- grischka

Re: Scrollbar thumbs

[David Kastrup]

"grischka" <grishka@gmx.de> writes:

>>> I guess it is just that these GUI toolkit people usually are trained
>>> to deliver good looking and pixel-precise artwork and in consequence
>>> might have some difficulties to imagine how it can be "pretty much
>>> impossibe" for some editor to know how many lines it is actually
>>> displaying (or anyway that such editor would still want to use their
>>> scrollbars).
>>
>> Emacs _does_ know how many lines it's displaying.
>>
>> What it doesn't know is how many lines are in the entire file, or
>> more precisely, the "height" of the entire file as rendered, which
>> essentially requires rendering the _entire thing_ (it can't be
>> calculated from logical number of lines because it depends on the
>> exact characters rendered).  That's a pretty nasty requirement for a
>> non-toy editor...
>
> The simple rule is: First do it right, then do it fast.  Try to make
> it work for small files and then you can still think about how to get
> equally good results with larger ones.

It is not an option to make Emacs unusable on large files for the sake
of the scrollbar idiosyncrasies of some toolkit.

-- 
David Kastrup

Re: Scrollbar thumbs

[joakim]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I guess it is just that these GUI toolkit people usually are trained to
>> deliver good looking and pixel-precise artwork and in consequence might have
>> some difficulties to imagine how it can be "pretty much impossibe" for some
>> editor to know how many lines it is actually displaying (or anyway that such
>> editor would still want to use their scrollbars).
>
> No, sadly, it is really due to dogma: your above explanation would make
> perfect sense, but in practice, even after showing them our problem,
> they still insist that it would be wrong for them to remove the chunks
> of code that enforce the "scrollbar metaphor" by preventing the client
> from getting the events that indicate that the user wants to move the
> thumb's bottom further down than the scrollbar's own bottom.
>
>
>         Stefan
>

I dont use scrollbars, so I dont really know, but would it be possible
to subclass the gtk scrollbars to achive our desired behaviour?
-- 
Joakim Verona

Re: Scrollbar thumbs

[Stefan Monnier]

> And the consistency aspect of "normal usage" being defined is simply that
> widgets are there to provide a consistent look & feel across applications.
> It seems inconsequent if on one hand you want to use a widget but on the
> other hand want to use it differently to 99% of the other clients.

I don't want to use it differently.  I only need to implement the check
in my own code rather than in theirs because doing it in theirs makes it
impossible for me to provide the behavior they want to see.


        Stefan

Re: Scrollbar thumbs

[grischka]

> > The simple rule is: First do it right, then do it fast.  Try to make
> > it work for small files and then you can still think about how to get
> > equally good results with larger ones.
> 
> It is not an option to make Emacs unusable on large files for the sake
> of the scrollbar idiosyncrasies of some toolkit.

In practice, if the file is large, scrollbar operations are less 
accurate and less smooth anyway, so you can get away with rough 
estimations very well without further compromising the scroll quality.

However the smaller the file the better a scrollbar can work and that 
is why accurate size calculations are especially beneficial for small 
files rsp. well choosen small parts of a file, such as at the top and 
the bottom.

--- grischka

Re: Scrollbar thumbs

[David Kastrup]

"grischka" <grishka@gmx.de> writes:

>> > The simple rule is: First do it right, then do it fast.  Try to make
>> > it work for small files and then you can still think about how to get
>> > equally good results with larger ones.
>>
>> It is not an option to make Emacs unusable on large files for the sake
>> of the scrollbar idiosyncrasies of some toolkit.
>
> In practice, if the file is large, scrollbar operations are less
> accurate and less smooth anyway, so you can get away with rough
> estimations very well without further compromising the scroll quality.

Scrollbars are not just updated when scrolling.

-- 
David Kastrup

Re: Scrollbar thumbs

[grischka]

>> And the consistency aspect of "normal usage" being defined is simply that
>> widgets are there to provide a consistent look & feel across applications.
>> It seems inconsequent if on one hand you want to use a widget but on the
>> other hand want to use it differently to 99% of the other clients.
> 
> I don't want to use it differently.  I only need to implement the check 
> in my own code rather than in theirs because doing it in theirs makes it 
> impossible for me to provide the behavior they want to see.

It cannot be impossible to get standard behavior (no matter what it is) 
with the standard API, because otherwise the behavior would hardly spread 
enough to become standard :)

--- grischka

Re: Scrollbar thumbs

[Stefan Monnier]

> It cannot be impossible to get standard behavior (no matter what it is) with
> the standard API, because otherwise the behavior would hardly spread enough
> to become standard :)

I'll be waiting for your patch that proves it,


        Stefan

Re: Scrollbar thumbs

[Miles Bader]

"grischka" <grishka@gmx.de> writes:
> Try to make it work for small files and then you can still think about
> how to get equally good results with larger ones.

This is very naive...

-Miles

-- 
Alliance, n. In international politics, the union of two thieves who have
their hands so deeply inserted in each other's pockets that they cannot
separately plunder a third.

Re: Scrollbar thumbs

[grischka]

>> Try to make it work for small files and then you can still think about
>> how to get equally good results with larger ones.
> 
> This is very naive...
> 
> -Miles
> 

Naive is what you currently have as scroll support in your program.

--- grischka

Re: Scrollbar thumbs

[joakim]

"grischka" <grishka@gmx.de> writes:

>>> Try to make it work for small files and then you can still think about
>>> how to get equally good results with larger ones.
>>
>> This is very naive...
>>
>> -Miles
>>
>
> Naive is what you currently have as scroll support in your program.
>
> --- grischka

Heres an attempt at bootstrapping this into a productive thread:

 - Is there a bugnumber describing the desired Emacs behaviour?
 - Can we locate a thread with the discussion of scrolling with the gtk developers?

Since I dont use scrollbars I'm not so knowledgeable about the technical
problem.
 
>
>
-- 
Joakim Verona

Re: Scrollbar thumbs

[Jan Djärv]

joakim@verona.se skrev 2009-11-04 12.18:
> "grischka"<grishka@gmx.de>  writes:
>
>>>> Try to make it work for small files and then you can still think about
>>>> how to get equally good results with larger ones.
>>>
>>> This is very naive...
>>>
>>> -Miles
>>>
>>
>> Naive is what you currently have as scroll support in your program.
>>
>> --- grischka
>
> Heres an attempt at bootstrapping this into a productive thread:
>
>   - Is there a bugnumber describing the desired Emacs behaviour?

A quick search didn't find any.

>   - Can we locate a thread with the discussion of scrolling with the gtk developers?
>

Started here AFAIK:
http://lists.gnu.org/archive/html/emacs-devel/2003-03/msg00555.html

	Jan D.

Re: Scrollbar thumbs

[Stefan Monnier]

>> - Can we locate a thread with the discussion of scrolling with the
>>   gtk developers?
> Started here AFAIK:
> http://lists.gnu.org/archive/html/emacs-devel/2003-03/msg00555.html

Note that the problem is more common than just Gtk.
So for example some of the discussion around Xaw3d (before the release
of Emacs-21) is probably relevant as well.

In any case, given how widespread the problem is, it's clear that we
can't expect the toolkit to adapt to Emacs.  And since nobody has found
a way yet to make Emacs DTRT, all we can do is provide various
workarounds, which is what we've done until now.  But what I suggest is
that instead of each toolkit backend providing its own workaround, we
should implement the workarounds in the generic code and even implement
several of them and let the user choose which one suits him best.


        Stefan

Re: Scrollbar thumbs

[YAMAMOTO Mitsuharu]

>>>>> On Wed, 04 Nov 2009 15:57:06 -0500, Stefan Monnier <monnier@iro.umontreal.ca> said:

> In any case, given how widespread the problem is, it's clear that we
> can't expect the toolkit to adapt to Emacs.  And since nobody has
> found a way yet to make Emacs DTRT, all we can do is provide various
> workarounds, which is what we've done until now.

The Carbon port and its descendants don't let the toolkit scrollbar
process the mouse events: we can't do that (without nontrivial changes
for multithreading of the GUI part) in the first place because the
toolkit scrollbar on Mac processes them in its own modal loop (i.e.,
we can't go back to the Lisp event loop during the mouse tracking, and
that is not compatible with the current scrollbar handling in Emacs.).

What the Carbon port (and its descendants) does is to process the
whole mouse events at the Emacs side.  It includes timer processing
for repeated scrolling (e.g, pressing an arrow button for a while.)
The toolkit is used only for displaying the scrollbar with
highlighting, detecting which part of the bar is pressed, and getting
the rectangle for a specified part of the bar.

The result is like a sort of mixture of native and toolkit scrollbars
on X11: it generates Lisp events and looks like the toolkit one, but
the size of the thumb usually changes during dragging and also
overscrolling at the bottom works as in the native one.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Scrollbar thumbs

[Jan D.]

On 2009-11-03 11:02, joakim@verona.se wrote:
> Stefan Monnier<monnier@iro.umontreal.ca>  writes:
>
>>> I guess it is just that these GUI toolkit people usually are trained to
>>> deliver good looking and pixel-precise artwork and in consequence might have
>>> some difficulties to imagine how it can be "pretty much impossibe" for some
>>> editor to know how many lines it is actually displaying (or anyway that such
>>> editor would still want to use their scrollbars).
>>
>> No, sadly, it is really due to dogma: your above explanation would make
>> perfect sense, but in practice, even after showing them our problem,
>> they still insist that it would be wrong for them to remove the chunks
>> of code that enforce the "scrollbar metaphor" by preventing the client
>> from getting the events that indicate that the user wants to move the
>> thumb's bottom further down than the scrollbar's own bottom.
>>
>>
>>          Stefan
>>
>
> I dont use scrollbars, so I dont really know, but would it be possible
> to subclass the gtk scrollbars to achive our desired behaviour?

I looked at it once, but no.  You can't override the basic event 
handling in scrollbars, so you still won't get any events when the thumb 
hit the bottom.

	Jan D.

Re: Scrollbar thumbs

[Jan D.]

On 2009-11-04 21:57, Stefan Monnier wrote:
>>> - Can we locate a thread with the discussion of scrolling with the
>>>    gtk developers?
>> Started here AFAIK:
>> http://lists.gnu.org/archive/html/emacs-devel/2003-03/msg00555.html
>
> Note that the problem is more common than just Gtk.
> So for example some of the discussion around Xaw3d (before the release
> of Emacs-21) is probably relevant as well.
>
> In any case, given how widespread the problem is, it's clear that we
> can't expect the toolkit to adapt to Emacs.  And since nobody has found
> a way yet to make Emacs DTRT, all we can do is provide various
> workarounds, which is what we've done until now.  But what I suggest is
> that instead of each toolkit backend providing its own workaround, we
> should implement the workarounds in the generic code and even implement
> several of them and let the user choose which one suits him best.
>

Wouldn't that imply using just the features that the least able toolkit 
provides?  The native scrollbar does things that can't be done in Gtk+ 
for example (not just overscroll, keybindings as well).

	Jan D.

Re: Scrollbar thumbs

[Stefan Monnier]

>> In any case, given how widespread the problem is, it's clear that we
>> can't expect the toolkit to adapt to Emacs.  And since nobody has found
>> a way yet to make Emacs DTRT, all we can do is provide various
>> workarounds, which is what we've done until now.  But what I suggest is
>> that instead of each toolkit backend providing its own workaround, we
>> should implement the workarounds in the generic code and even implement
>> several of them and let the user choose which one suits him best.

> Wouldn't that imply using just the features that the least able toolkit
> provides?  The native scrollbar does things that can't be done in Gtk+ for
> example (not just overscroll, keybindings as well).

I don't think so.  It just means to centralize the workarounds (which
mostly have to do with how the thumb size and position are computed and
where/when they're updated) without impacting anything else of
the functionality.


        Stefan

Re: Scrollbar thumbs

[grischka]

> In any case, given how widespread the problem is, it's clear that we
> can't expect the toolkit to adapt to Emacs.  And since nobody has found
> a way yet to make Emacs DTRT, all we can do is provide various
> workarounds, which is what we've done until now.  But what I suggest is
> that instead of each toolkit backend providing its own workaround, we
> should implement the workarounds in the generic code and even implement
> several of them and let the user choose which one suits him best.
> 
> 
>        Stefan

I guess it is a good idea. If you want to get rid of stuff, throw it 
all in one place, first.

Seen that nowadays the gui-dyslexics tend to become a more rare species 
and the generation simply tends to loose the ability to understand 
that a non-linear, moreover non-contiguous and even non-complementary 
geometrical relation between scrollbar and scrollable content can be 
anything else than an accumulation of design flaws.

--- grischka

Re: Scrollbar thumbs

[Lennart Borgman]

On Thu, Nov 5, 2009 at 4:16 PM, grischka <grishka@gmx.de> wrote:
>> In any case, given how widespread the problem is, it's clear that we
>> can't expect the toolkit to adapt to Emacs.  And since nobody has found
>> a way yet to make Emacs DTRT, all we can do is provide various
>> workarounds, which is what we've done until now.  But what I suggest is
>> that instead of each toolkit backend providing its own workaround, we
>> should implement the workarounds in the generic code and even implement
>> several of them and let the user choose which one suits him best.
>>
>>
>>       Stefan
>
> I guess it is a good idea. If you want to get rid of stuff, throw it all in
> one place, first.
>
> Seen that nowadays the gui-dyslexics tend to become a more rare species and
> the generation simply tends to loose the ability to understand that a
> non-linear, moreover non-contiguous and even non-complementary geometrical
> relation between scrollbar and scrollable content can be anything else than
> an accumulation of design flaws.


Maybe the gui-dyslexics are getting fewer, but did not Stefan in the
discussion from some years ago point to that the gui part is broken
due to the incapability of some of the scroll bar api:s?

Re: Scrollbar thumbs

[Óscar Fuentes]

"grischka" <grishka@gmx.de> writes:

> Seen that nowadays the gui-dyslexics tend to become a more rare
> species and the generation simply tends to loose the ability to
> understand that a non-linear, moreover non-contiguous and even
> non-complementary geometrical relation between scrollbar and
> scrollable content can be anything else than an accumulation of design
> flaws.

Sure, the Real World is slowly adapting to the rules stated by some
illuminated GUI toolkit designers.

:-/

-- 
Óscar

Re: Scrollbar thumbs

[David Kastrup]

"grischka" <grishka@gmx.de> writes:

> Seen that nowadays the gui-dyslexics

Could you look up complex words in the dictionary before you use them?
It might help to make your postings less inflammatory.

> tend to become a more rare species

dyslexia is not coupled to species.

> and the generation simply tends to loose the ability to understand

While you have the dictionary at hand, look up "loose".

> that a non-linear, moreover non-contiguous

And "contiguous".

> and even non-complementary

And "complementary".

> geometrical relation between scrollbar and scrollable content can be
> anything else than an accumulation of design flaws.

The "design flaw" is that Emacs has a variable line height.  An editor
is primarily supposed to deal with lines of text, not with continuous
graphical panes.  And previous to Emacs 21, Emacs only dealt with lines
of text of constant height.  Previous to Emacs 19, the lines
corresponded 1:1 to the screen lines.

None of those bear any relation to the fact that an editor can show
areas that are not part of the content (most trivially for an empty
file), and that should reflect on the scrollbar appearance.  A web
browser will just remove the scrollbar altogether.  But that is just
feasible for static content, and when the changing of the layout when a
scrollbar is added and removed is not disconcerting.

Also, defining some model of correctness and then adhering to it needs
to take interactive responsiveness into consideration.  The typical web
document does not contain millions of lines, but Emacs must be able to
deal with navigating and changing stuff in documents of that size.

Anyway: if you want to achieve something, chances are that choosing a
manner of correspondence that sports a less condescending appearance
might be beneficial.  Chances are that people working decades in a
particular area of expertise have all that time just been missing the
advice of a complete newcomer.

And if you are convinced of the opposite, common sense should tell you
that such a severe case should need rather more than less diplomacy to
deal with.

-- 
David Kastrup

Re: Scrollbar thumbs

[Jason Rumney]

David Kastrup wrote:

> The "design flaw" is that Emacs has a variable line height.  An editor
> is primarily supposed to deal with lines of text, not with continuous
> graphical panes.  And previous to Emacs 21, Emacs only dealt with lines
> of text of constant height.  Previous to Emacs 19, the lines
> corresponded 1:1 to the screen lines.
>   
Even Emacs 19 had line wrapping, so counting CR characters as the file 
is read (as some simple text editors do) has never been the solution to 
this problem.  The only solution seems to be to render the entire buffer 
offscreen in background, adjusting the scrollbar as we get more detailed 
information about the real contents of the buffer.  This is what word 
processors, web browsers and other complex applications do - you see the 
first page quite quickly, then you see the scroll bar growing as it 
renders the buffer in background.  Most applications do not even make an 
initial guess, so you cannot scroll to the bottom of the buffer until 
the background rendering is finished. The application then needs to use 
extra memory to cache the metrics it has calculated to avoid having to 
rerender the entire document on every change.

Re: Scrollbar thumbs

[Miles Bader]

Jason Rumney <jasonr@gnu.org> writes:
> Even Emacs 19 had line wrapping, so counting CR characters as the file
> is read (as some simple text editors do) has never been the solution to
> this problem.  The only solution seems to be to render the entire buffer
> offscreen in background

Yes.  In other words, a complete (and _massive_ in terms of code) change
in Emacs' internal model, to one which is more complex (lots of caching)
and hugely impractical for very large files.

The toolkit change, by contrast, is apparently quite small, and would
simply make the toolkit allow more application flexibility.  No existing
GTK apps would be affected.  AFAICT, the only reason GTK _hasn't_
changed is because of a dogmatic refusal to believe any model except
their own is valid.

Even if every Emacs author suddenly decided they _wanted_ to rewrite
Emacs to accommodate GTK, it wouldn't happen, simply because it's not
even remotely practical.  But as I understand it, GTK could easily
change given a decision to allow it.

I think GTK is in general, a pretty nice toolkit, but not every
application is exactly The Gimp, and this sort of head-in-the-sand
behavior is very disheartening.

-Miles

-- 
Year, n. A period of three hundred and sixty-five disappointments.

Re: Scrollbar thumbs

[Stephen J. Turnbull]

Jason Rumney writes:
 > David Kastrup wrote:
 > 
 > > The "design flaw" is that Emacs has a variable line height.  An editor
 > > is primarily supposed to deal with lines of text, not with continuous
 > > graphical panes.  And previous to Emacs 21, Emacs only dealt with lines
 > > of text of constant height.  Previous to Emacs 19, the lines
 > > corresponded 1:1 to the screen lines.
 > >   
 > Even Emacs 19 had line wrapping, so counting CR characters as the file 
 > is read (as some simple text editors do) has never been the solution to 
 > this problem.

AFAICT XEmacs counts *characters*, not lines.  This means that a page
with lots of empty lines gets a smaller thumb than one with almost all
lines full-width.  (It's observable that the thumbs change size.)
This would also give consistent results between long-lines support
(ie, proper wrapping of newline-delimited paragraphs) and conventional
newline-delimited lines.  Of course it's not possible to jump to a
particular page in this algorithm, but modern apps don't seem to do
this with scrollbars anymore anyway.  They provide a separate tray of
thumbnails (which gets its own scrollbar).

I don't recall seeing a question about this on XEmacs channels, nor
complaints that scrollbars function differently on different platforms.

 > The only solution seems to be to render the entire buffer offscreen
 > in background, adjusting the scrollbar as we get more detailed
 > information about the real contents of the buffer.  This is what
 > word processors, web browsers and other complex applications do -
 > you see the first page quite quickly, then you see the scroll bar
 > growing as it renders the buffer in background.  Most applications
 > do not even make an initial guess, so you cannot scroll to the
 > bottom of the buffer until the background rendering is
 > finished.

That is not acceptable on a 128MB log file, though.  Typically what
you want to look at *first* is the end of the file.

 > The application then needs to use extra memory to cache the metrics
 > it has calculated to avoid having to rerender the entire document
 > on every change.

This is a big deal?  All you need is the heights of the pages.  You
recalculate spillage into or out of the current page as you redisplay,
the previous pages don't change at all, and you approximate the later
pages as not having changed length.

Re: Scrollbar thumbs

[Miles Bader]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
> AFAICT XEmacs counts *characters*, not lines.

That's what Emacs does too, and it gives good, usable results while
working naturally with Emacs rendering.  The problem, of course is that
(as you note) it _also_ results in scrollbar size varying depending on
your location, and GTK's scrollbar does not allow this, although it
could easily do so, apparently because they believe all apps should
rigidly use "vertical height" metrics for scrollbars, no exceptions
permitted.

-Miles

-- 
I'd rather be consing.

Re: Scrollbar thumbs

[David Kastrup]

Jason Rumney <jasonr@gnu.org> writes:

> David Kastrup wrote:
>
>> The "design flaw" is that Emacs has a variable line height.  An editor
>> is primarily supposed to deal with lines of text, not with continuous
>> graphical panes.  And previous to Emacs 21, Emacs only dealt with lines
>> of text of constant height.  Previous to Emacs 19, the lines
>> corresponded 1:1 to the screen lines.
>>   
> Even Emacs 19 had line wrapping, so counting CR characters as the file
> is read (as some simple text editors do) has never been the solution
> to this problem.  The only solution seems to be to render the entire
> buffer offscreen in background, adjusting the scrollbar as we get more
> detailed information about the real contents of the buffer.  This is
> what word processors, web browsers and other complex applications do -
> you see the first page quite quickly, then you see the scroll bar
> growing as it renders the buffer in background.  Most applications do
> not even make an initial guess, so you cannot scroll to the bottom of
> the buffer until the background rendering is finished. The application
> then needs to use extra memory to cache the metrics it has calculated
> to avoid having to rerender the entire document on every change.

Actually, the inability to get such an application in combination with
the castrated scroll bar semantics scroll half a screen down is quite
annoying.  The only way you can scroll a half-screen (without
Athena-style scrollbar behavior) is by dragging, and particularly
dragging in large documents while the scrollbar self-updates is utterly
haphazard.  Usually you completely use your place.

I don't see that this is a useful model of behavior for an editor.

-- 
David Kastrup

Re: Scrollbar thumbs

[Eli Zaretskii]

> Date: Fri, 06 Nov 2009 07:55:51 +0800
> From: Jason Rumney <jasonr@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> The only solution seems to be to render the entire buffer offscreen
> in background, adjusting the scrollbar as we get more detailed
> information about the real contents of the buffer.  This is what
> word processors, web browsers and other complex applications do -
> you see the first page quite quickly, then you see the scroll bar
> growing as it renders the buffer in background.

Except that in Emacs, such rendering would be prohibitively expensive,
even for moderate size files, because we have display features that
those other word processors don't even dream of when displaying plain
text files, and those features are designed and implemented in a way
that is completely incompatible with rendering everything at once.

Doing the above will probably mean a complete redesign and rewrite of
the display engine.  And for what? for satisfying a few purists that
cannot live with marginally weird behavior of the scroll bar?

Re: Scrollbar thumbs

[grischka]

Óscar Fuentes wrote:
> Sure, the Real World is slowly adapting to the rules stated by some
> illuminated GUI toolkit designers.

The entire Real World?  No! A small village of indomitable lisp hackers
still holds out against the visual invasion, fortified by their magic
knowledge of the mantras that allow them to transform pure human will
into virtual user interfaces located in the user herself.

--- grischka

Re: Scrollbar thumbs

[Óscar Fuentes]

"grischka" <grishka@gmx.de> writes:

> Óscar Fuentes wrote:
>> Sure, the Real World is slowly adapting to the rules stated by some
>> illuminated GUI toolkit designers.
>
> The entire Real World?  No! A small village of indomitable lisp hackers
> still holds out against the visual invasion, fortified by their magic
> knowledge of the mantras that allow them to transform pure human will
> into virtual user interfaces located in the user herself.

Just in case someone takes the above comment half-seriously, it is worth
noting that the problem faced by emacs wrt GTK scrollbar would hinder
other applications too. For instance, on some areas it is required to
display very large datasets on a grid. There, it is not often clear how
much rows there are on the dataset, nor exists the concept on n-th row,
so it is not possible to have a precise 1-to-1 mapping among the
scrollbar thumb position and the dataset cursor position. On those
applications it is useful to handle thumb movements as exponential
displacements too.

Nothing on the Holy Scripture says that scrollbars are forbidden for
those uses. Quite the contrary, users have no problem at all using
scrollbars with that behaviour.

GTK chose a very restrictive concept of scrollbar. Not only this, but it
chose to *force* this concept on the users.

You can adapt Emacs to GTK requirements, but I wouldn't consider that a
bug fix, but rather a workaround for a dumb widget.

-- 
Óscar

Re: Scrollbar thumbs

[David De La Harpe Golden]

Óscar Fuentes wrote:


> Just in case someone takes the above comment half-seriously, it is worth
> noting that the problem faced by emacs wrt GTK scrollbar would hinder
> other applications too. For instance, on some areas it is required to
> display very large datasets on a grid. There, it is not often clear how
> much rows there are on the dataset, nor exists the concept on n-th row,
> so it is not possible to have a precise 1-to-1 mapping among the
> scrollbar thumb position and the dataset cursor position.


Hmm. The is an old ui convention for [some of] that, whereby using the 
scrollbar arrows allows pushing "past the end", shrinking the thumb.
Back in the amiga days at least, this was some sort of a FAQ along the 
lines of
"Q. Why do scrollbars have those redundant arrow things at all, anyway?
A. Because they're not redundant, in some situations the arrows grow the 
size of the content area, the bar only scrolls within the current extent 
of the content area"

- you see that in the "workbench" file browser windows on the amiga, for 
example.   The fact people had to have that explained to them on 
occasion suggests it wasn't all that obvious though.

I don't know if gtk actually allows that, and I don't think it really
helps in the emacs case, just saying that convention existed.

Re: Scrollbar thumbs

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 05 Nov 2009 10:18:07 +0900, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> said:

> What the Carbon port (and its descendants) does is to process the
> whole mouse events at the Emacs side.  It includes timer processing
> for repeated scrolling (e.g, pressing an arrow button for a while.)
> The toolkit is used only for displaying the scrollbar with
> highlighting, detecting which part of the bar is pressed, and
> getting the rectangle for a specified part of the bar.

> The result is like a sort of mixture of native and toolkit
> scrollbars on X11: it generates Lisp events and looks like the
> toolkit one, but the size of the thumb usually changes during
> dragging and also overscrolling at the bottom works as in the native
> one.

We can port this thumb handling to scroll bars on Xaw3d compiled with
ARROW_SCROLLBARS, because we can guess the rectangle of the thumb from
available widget resource values.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Index: src/xterm.c
===================================================================
RCS file: /sources/emacs/emacs/src/xterm.c,v
retrieving revision 1.1051
diff -c -p -r1.1051 xterm.c
*** src/xterm.c	6 Nov 2009 08:30:45 -0000	1.1051
--- src/xterm.c	9 Nov 2009 20:33:21 -0000
*************** static void x_create_toolkit_scroll_bar 
*** 4101,4107 ****
  					     struct scroll_bar *));
  static void x_set_toolkit_scroll_bar_thumb P_ ((struct scroll_bar *,
  						int, int, int));
! 
  
  /* Lisp window being scrolled.  Set when starting to interact with
     a toolkit scroll bar, reset to nil when ending the interaction.  */
--- 4101,4108 ----
  					     struct scroll_bar *));
  static void x_set_toolkit_scroll_bar_thumb P_ ((struct scroll_bar *,
  						int, int, int));
! static int x_scroll_bar_handle_thumb_event P_ ((struct scroll_bar *, XEvent *,
! 						struct input_event *));
  
  /* Lisp window being scrolled.  Set when starting to interact with
     a toolkit scroll bar, reset to nil when ending the interaction.  */
*************** x_set_toolkit_scroll_bar_thumb (bar, por
*** 4812,4817 ****
--- 4813,4851 ----
      }
  #else /* !USE_MOTIF i.e. use Xaw */
  
+   if (xaw3d_arrow_scroll)
+     {
+       float old_top, old_shown;
+       Dimension length, thickness, min_thumb, shadow_width;
+       Dimension margin, min_shown_length;
+ 
+       XtVaGetValues (widget, XtNtopOfThumb, &old_top, XtNshown, &old_shown,
+ 		     XtNlength, &length, XtNthickness, &thickness,
+ 		     XtNminimumThumb, &min_thumb,
+ 		     XtNshadowWidth, &shadow_width,
+ 		     NULL);
+       margin = thickness;
+       min_shown_length = min_thumb + 2 * shadow_width;
+       if (whole == 0 || length < 2 * margin + min_shown_length)
+ 	top = 0, shown = 1;
+       else
+ 	{
+ 	  Dimension tzl;
+ 	  float maximum, scale;
+ 
+ 	  tzl = length - margin - margin;
+ 
+ 	  maximum = tzl - min_shown_length;
+ 	  scale = maximum / whole;
+ 	  top = (position * scale) / tzl;
+ 	  shown = (portion * scale + min_shown_length) / tzl;
+ 	}
+       if (top != old_top || shown != old_shown)
+ 	XawScrollbarSetThumb (widget, top, shown);
+     }
+   else
+     {
+ 
    if (whole == 0)
      top = 0, shown = 1;
    else
*************** x_set_toolkit_scroll_bar_thumb (bar, por
*** 4854,4865 ****
--- 4888,5025 ----
  	  }
        }
    }
+ 
+     }
  #endif /* !USE_MOTIF */
  
    UNBLOCK_INPUT;
  }
  #endif /* not USE_GTK */
  
+ /* Handle an X11 event EVENT if it is for the thumb of the scroll bar
+    BAR, and store an input event to *BUFP if necessary.  Return
+    non-zero if the X11 event is handled (i.e., no further handling at
+    the toolkit side needed).  */
+ 
+ static int
+ x_scroll_bar_handle_thumb_event (bar, event, bufp)
+      struct scroll_bar *bar;
+      XEvent *event;
+      struct input_event *bufp;
+ {
+ #if !defined (USE_GTK) && !defined (USE_MOTIF)
+   if (xaw3d_arrow_scroll)
+     switch (event->type)
+       {
+       case ButtonPress:
+ 	{
+ 	  Widget widget = XtWindowToWidget (event->xbutton.display,
+ 					    event->xbutton.window);
+ 	  float top, shown;
+ 	  Dimension length, thickness;
+ 	  XtOrientation orientation;
+ 	  Dimension margin, tzl;
+ 	  Position top_loc, bottom_loc;
+ 	  int pos;
+ 
+ 	  XtVaGetValues (widget, XtNtopOfThumb, &top, XtNshown, &shown,
+ 			 XtNlength, &length, XtNthickness, &thickness,
+ 			 XtNorientation, &orientation,
+ 			 NULL);
+ 	  margin = thickness;
+ 	  if (length > 2 * margin)
+ 	    tzl = length - margin - margin;
+ 	  else
+ 	    tzl = 0;
+ 	  top_loc = margin + (int) (tzl * top);
+ 	  bottom_loc = top_loc + (int) (tzl * shown);
+ 
+ 	  pos = (orientation == XtorientHorizontal
+ 		 ? event->xbutton.x : event->xbutton.y);
+ 	  if (pos >= top_loc && pos <= bottom_loc)
+ 	    {
+ 	      bar->dragging = make_number (- (pos - top_loc) - 1);
+ 	      bar->tracking_thumb_p = 1;
+ 
+ 	      return 1;
+ 	    }
+ 	}
+ 	break;
+ 
+       case ButtonRelease:
+ 	if (bar->tracking_thumb_p)
+ 	  {
+ 	    if (INTEGERP (bar->dragging) && XINT (bar->dragging) >= 0)
+ 	      {
+ 		bufp->kind = SCROLL_BAR_CLICK_EVENT;
+ 		bufp->frame_or_window = bar->window;
+ 		bufp->arg = Qnil;
+ 		bufp->timestamp = event->xbutton.time;
+ 		bufp->part = scroll_bar_end_scroll;
+ 		bufp->code = 0;
+ 		XSETINT (bufp->x, 0);
+ 		XSETINT (bufp->y, 0);
+ 		bufp->modifiers = 0;
+ 	      }
+ 	    bar->dragging = Qnil;
+ 	    bar->tracking_thumb_p = 0;
+ 
+ 	    return 1;
+ 	  }
+ 	break;
+ 
+       case MotionNotify:
+ 	if (bar->tracking_thumb_p)
+ 	  {
+ 	    Widget widget = XtWindowToWidget (event->xmotion.display,
+ 					      event->xmotion.window);
+ 	    Dimension length, thickness, min_thumb, shadow_width;
+ 	    XtOrientation orientation;
+ 	    Dimension margin, min_shown_length, tzl;
+ 	    int pos;
+ 	    int portion, whole;
+ 
+ 	    if (INTEGERP (bar->dragging) && XINT (bar->dragging) < 0)
+ 	      bar->dragging = make_number (- (XINT (bar->dragging) + 1));
+ 
+ 	    XtVaGetValues (widget, XtNlength, &length, XtNthickness, &thickness,
+ 			   XtNminimumThumb, &min_thumb,
+ 			   XtNshadowWidth, &shadow_width,
+ 			   XtNorientation, &orientation,
+ 			   NULL);
+ 	    margin = thickness;
+ 	    min_shown_length = min_thumb + 2 * shadow_width;
+ 	    tzl = length - margin - margin;
+ 
+ 	    pos = (orientation == XtorientHorizontal
+ 		   ? event->xmotion.x : event->xmotion.y);
+ 	    portion = pos - XINT (bar->dragging) - margin;
+ 	    whole = tzl > min_shown_length ? tzl - min_shown_length : tzl;
+ 	    if (portion < 0)
+ 	      portion = 0;
+ 	    if (portion > whole)
+ 	      portion = whole;
+ 
+ 	    bufp->kind = SCROLL_BAR_CLICK_EVENT;
+ 	    bufp->frame_or_window = bar->window;
+ 	    bufp->arg = Qnil;
+ 	    bufp->timestamp = event->xmotion.time;
+ 	    bufp->part = scroll_bar_handle;
+ 	    bufp->code = 0;
+ 	    XSETINT (bufp->x, portion);
+ 	    XSETINT (bufp->y, whole);
+ 	    bufp->modifiers = 0;
+ 	  }
+ 
+ 	return 1;
+ 	break;
+ 
+       default:
+ 	abort ();
+       }
+ #endif	/* !USE_GTK && !USE_MOTIF */
+   return 0;
+ }
  #endif /* USE_TOOLKIT_SCROLL_BARS */
  
  
*************** x_scroll_bar_create (w, top, left, width
*** 4937,4942 ****
--- 5097,5103 ----
    bar->end = 0;
    bar->dragging = Qnil;
    bar->fringe_extended_p = 0;
+   bar->tracking_thumb_p = 0;
  
    /* Add bar to its frame's list of scroll bars.  */
    bar->next = FRAME_SCROLL_BARS (f);
*************** handle_one_xevent (dpyinfo, eventp, fini
*** 6729,6742 ****
            }
          else
            {
- #ifndef USE_TOOLKIT_SCROLL_BARS
              struct scroll_bar *bar
                = x_window_to_scroll_bar (event.xmotion.display,
                                          event.xmotion.window);
  
              if (bar)
                x_scroll_bar_note_movement (bar, &event);
! #endif /* USE_TOOLKIT_SCROLL_BARS */
  
              /* If we move outside the frame, then we're
                 certainly no longer on any text in the frame.  */
--- 6890,6906 ----
            }
          else
            {
              struct scroll_bar *bar
                = x_window_to_scroll_bar (event.xmotion.display,
                                          event.xmotion.window);
  
+ #ifdef USE_TOOLKIT_SCROLL_BARS
+             if (bar && x_scroll_bar_handle_thumb_event (bar, &event, &inev.ie))
+ 	      *finish = X_EVENT_DROP;
+ #else  /* !USE_TOOLKIT_SCROLL_BARS */
              if (bar)
                x_scroll_bar_note_movement (bar, &event);
! #endif	/* !USE_TOOLKIT_SCROLL_BARS */
  
              /* If we move outside the frame, then we're
                 certainly no longer on any text in the frame.  */
*************** handle_one_xevent (dpyinfo, eventp, fini
*** 6891,6896 ****
--- 7055,7063 ----
                  x_scroll_bar_handle_click (bar, &event, &inev.ie);
                  *finish = X_EVENT_DROP;
                }
+ 	    else if (bar
+ 		     && x_scroll_bar_handle_thumb_event (bar, &event, &inev.ie))
+ 	      *finish = X_EVENT_DROP;
  #else /* not USE_TOOLKIT_SCROLL_BARS */
              if (bar)
                x_scroll_bar_handle_click (bar, &event, &inev.ie);
Index: src/xterm.h
===================================================================
RCS file: /sources/emacs/emacs/src/xterm.h,v
retrieving revision 1.217
diff -c -p -r1.217 xterm.h
*** src/xterm.h	21 Oct 2009 18:29:48 -0000	1.217
--- src/xterm.h	9 Nov 2009 20:33:21 -0000
*************** struct scroll_bar
*** 776,781 ****
--- 776,785 ----
    /* 1 if the background of the fringe that is adjacent to a scroll
       bar is extended to the gap between the fringe and the bar.  */
    unsigned int fringe_extended_p : 1;
+ 
+   /* 1 if the toolkit scroll bar thumb is currently being tracked at
+      the Emacs side. */
+   unsigned int tracking_thumb_p : 1;
  };
  
  /* The number of elements a vector holding a struct scroll_bar needs.  */