* Deffering redisplay in COMINT
@ 2012-12-07 1:08 Michael Mauger
2012-12-25 11:09 ` Vitalie Spinu
0 siblings, 1 reply; 9+ messages in thread
From: Michael Mauger @ 2012-12-07 1:08 UTC (permalink / raw)
To: Emacs Devel
I don't believe this is possible, but I figured I'd run it past the brain trust....
In sql-interactive-mode, a derivative of comint, I will submit a SELECT statement that returns many very long rows (2000+ characters per line). The redisplay fireworks that follow are impressive but slow. As text streams to the buffer with `auto-hscroll-mode' enabled the screen flashes as it redraws the buffer with more lines of data and scrolled horizontally in different places until all the data stream is complete.
I have figured out how to turn off horizontal scrolling with a `comint-preoutput-filter-function' and a `pre-command-hook' for the duration of SQL command so the redisplay is limited to a slow march straight down the page without the intervening horizontal scrolling but the column number in the mode line going crazy. I've also discovered that if I switch to another buffer so that the SQL buffer is not displayed and then switch back, the SQL buffer is populated very quickly. (That is, Emacs can fetch the results quickly, it just can't redraw it quickly a couple of hundred times)
So what I'm asking: Is there a way (say by setting `inhibit-redisplay' to t) for the duration of the comint command and then restoring redisplay when I am at the prompt at the end. The trick that I use for `auto-hscroll-mode' won't work because I don't find out about the completion until the user causes an event. I understand also that I don't want to universally block redisplay in case I switch to another buffer while the comint command is running and I want to interact with another buffer (i.e., I want to inhibit redisplay in that buffer only--buffer-local inhibit-redisplay). Looking at comint.el and based on my understanding of how processes are handled, I don't believe that it is easy to do because I don't know when the output is complete. (I've noticed in my disable auto-hscroll-mode code that the comint-preoutput-filter-function will get called a couple of hundred times with chunks of the output. I'm probably getting a redisplay after
each chunk.)
My only thought is that I could route the output to a temp buffer until I see the prompt at the end of a chunk (ala the comint redirection code, a very non-Emacs 20+ way of doing things), and then move the output to the original buffer. Essentially buffering the output until it's complete. Thoughts? Am I missing an obvious hook? (pre-command isn't it, it fires only on input, unless I can feed input to it to force it when done)
Am I SOL?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deffering redisplay in COMINT
2012-12-07 1:08 Deffering redisplay in COMINT Michael Mauger
@ 2012-12-25 11:09 ` Vitalie Spinu
2013-01-06 8:09 ` Michael Mauger
0 siblings, 1 reply; 9+ messages in thread
From: Vitalie Spinu @ 2012-12-25 11:09 UTC (permalink / raw)
To: Michael Mauger; +Cc: Emacs Devel
>> Michael Mauger <mmaug@yahoo.com>
>> on Thu, 6 Dec 2012 17:08:33 -0800 (PST) wrote:
> I don't believe this is possible, but I figured I'd run it past the brain trust....
> In sql-interactive-mode, a derivative of comint, I will submit a SELECT
> statement that returns many very long rows (2000+ characters per line). The
> redisplay fireworks that follow are impressive but slow. As text streams to
> the buffer with `auto-hscroll-mode' enabled the screen flashes as it redraws the
> buffer with more lines of data and scrolled horizontally in different places
> until all the data stream is complete.
> I have figured out how to turn off horizontal scrolling with a
> comint-preoutput-filter-function' and a `pre-command-hook' for the duration of
> SQL command so the redisplay is limited to a slow march straight down the page
> without the intervening horizontal scrolling but the column number in the mode
> line going crazy. I've also discovered that if I switch to another buffer so
> that the SQL buffer is not displayed and then switch back, the SQL buffer is
> populated very quickly. (That is, Emacs can fetch the results quickly, it just
> can't redraw it quickly a couple of hundred times)
> So what I'm asking: Is there a way (say by setting `inhibit-redisplay' to t) for
> the duration of the comint command and then restoring redisplay when I am at the
> prompt at the end. The trick that I use for `auto-hscroll-mode' won't work
> because I don't find out about the completion until the user causes an event. I
> understand also that I don't want to universally block redisplay in case I
> switch to another buffer while the comint command is running and I want to
> interact with another buffer (i.e., I want to inhibit redisplay in that buffer
> only--buffer-local inhibit-redisplay). Looking at comint.el and based on my
> understanding of how processes are handled, I don't believe that it is easy to
> do because I don't know when the output is complete. (I've noticed in my
> disable auto-hscroll-mode code that the comint-preoutput-filter-function will
> get called a couple of hundred times with chunks of the output. I'm probably
> getting a redisplay after
> each chunk.)
> My only thought is that I could route the output to a temp buffer until I see
> the prompt at the end of a chunk
To the best of my knowledge this is the only way out. But it is not bad
at all. You have to replace comint-output-filter with your own filter
that does that. The filter is triggered each time emacs receives output
from the sub-process (roughly 500 chars last time I counted). It's
pretty trivial to wait for a prompt at the end of output chunk, and you
will be surprised how reliable this procedure actually is.
This is how we do it in ESS:
(defun ess-wait-for-process (proc &optional sec-prompt wait force-redisplay)
"Wait for 'busy property of the process to become nil.
If SEC-PROMPT is non-nil return if secondary prompt is detected
regardless of whether primary prompt was detected or not. If
WAIT is non-nil wait for WAIT seconds for process output before
the prompt check, default 0.001s. When FORCE-REDISPLAY is non-nil
force redisplay. You better use WAIT >= 0.1 if you need
FORCE-REDISPLAY to avoid excesive redisplay."
(unless (eq (process-status proc) 'run)
(ess-error "ESS process has died unexpectedly."))
(setq wait (or wait 0.001)) ;;xemacs is stuck if it's 0 here
(save-excursion
(while (or (accept-process-output proc wait)
(if (and sec-prompt (process-get proc 'sec-prompt))
nil
(if force-redisplay (redisplay 'force))
(process-get proc 'busy))))))
The 'busy is set in the process filter by searching for prompt anchored
at the end of output chunk.
HTH,
Vitalie
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deffering redisplay in COMINT
2012-12-25 11:09 ` Vitalie Spinu
@ 2013-01-06 8:09 ` Michael Mauger
2013-01-08 18:19 ` Stefan Monnier
0 siblings, 1 reply; 9+ messages in thread
From: Michael Mauger @ 2013-01-06 8:09 UTC (permalink / raw)
To: Vitalie Spinu; +Cc: Emacs Devel
>________________________________
> Vitalie Spinu <spinuvit@gmail.com>
> on Tuesday, December 25, 2012 6:09 AM wrote:
>
> >> Michael Mauger <mmaug@yahoo.com>
> >> on Thu, 6 Dec 2012 17:08:33 -0800 (PST) wrote:
>
> > In sql-interactive-mode, a derivative of comint, I will submit a SELECT
> > statement that returns many very long rows (2000+ characters per line). The
> > redisplay fireworks that follow are impressive but slow. As text streams to
> > the buffer with `auto-hscroll-mode' enabled the screen flashes as it redraws the
> > buffer with more lines of data and scrolled horizontally in different places
> > until all the data stream is complete.
>
....
>
> > My only thought is that I could route the output to a temp buffer until I see
> > the prompt at the end of a chunk
>
>To the best of my knowledge this is the only way out. But it is not bad
>at all. You have to replace comint-output-filter with your own filter
>that does that. The filter is triggered each time emacs receives output
>from the sub-process (roughly 500 chars last time I counted). It's
>pretty trivial to wait for a prompt at the end of output chunk, and you
>will be surprised how reliable this procedure actually is.
>
>This is how we do it in ESS:
>
> (defun ess-wait-for-process (proc &optional sec-prompt wait force-redisplay)
....
>
>
Thanks for your input. I took a look, but that approach makes emacs
modal, in that it locks up until the output is complete. I want to still
have a responsive editor since it make take a while before results start
to appear. But I've come up with a solution that gets reasonable
performance and only has a minor functional impact.
I'll give a brief description here, and I'll release this as a SQL-related
add-in in the ELPA shortly.
It's set up as a minor mode so it can be disabled in case it causes
problems. When it's enabled, it activates a pre and post output hooks.
The pre hook only activates if the mode is enabled and truncate-lines
is set. When active, it disables auto-hscroll-mode. It then appends
the output text to a temporary buffer and then returns a window's
width worth of characters of each line in the temp buffer to
be written to the output buffer. The output appears to fill each
line without the display slowdown due to long displayed lines.
And because hscroll is turned off, the text merely flows up the
screen quickly.
When the post output hook is called and it detects when the last
output text matches the prompt regexp, then it iterates thru
the temporary buffer and the output buffer starting at
comint-last-input-end. It takes each line after the window-
width worth of characters from the temp buffer and inserts them
at the end of each line in the output buffer. Once the output
buffer has been filled in with all the off-window text it re-enables
the horizontal scrolling.
The result is fast smooth scrolling and an often unnoticeable
hesitation when it is complete. It's not vital to have this to
use sql-mode, but having it as an add-on minor mode eliminates
a significant distraction when doing query development in Emacs.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deffering redisplay in COMINT
2013-01-06 8:09 ` Michael Mauger
@ 2013-01-08 18:19 ` Stefan Monnier
2013-01-11 14:40 ` Michael Mauger
0 siblings, 1 reply; 9+ messages in thread
From: Stefan Monnier @ 2013-01-08 18:19 UTC (permalink / raw)
To: Michael Mauger; +Cc: Vitalie Spinu, Emacs Devel
> The pre hook only activates if the mode is enabled and truncate-lines
> is set. When active, it disables auto-hscroll-mode. It then appends
> the output text to a temporary buffer and then returns a window's
> width worth of characters of each line in the temp buffer to
> be written to the output buffer. The output appears to fill each
> line without the display slowdown due to long displayed lines.
> And because hscroll is turned off, the text merely flows up the
> screen quickly.
Have you tried the (presumably) simpler approach of only inserting the
text one line at a time, i.e. accumulate the output in some undisplayed
"buffer area" (can be a buffer, a string, or a list of strings, this
last choice being probably the most efficient) until you see a \n and
then insert that text (upto and including the \n, so the cursor stays in
column 0 and doesn't risk triggering auto-hscroll-mode) while keeping
the rest in the "buffer area"?
Stefan
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deffering redisplay in COMINT
2013-01-08 18:19 ` Stefan Monnier
@ 2013-01-11 14:40 ` Michael Mauger
2013-01-11 15:55 ` Stefan Monnier
0 siblings, 1 reply; 9+ messages in thread
From: Michael Mauger @ 2013-01-11 14:40 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Vitalie Spinu, Emacs Devel
>
>> The pre hook only activates if the mode is enabled and truncate-lines
>> is set. When active, it disables auto-hscroll-mode. It then appends
>> the output text to a temporary buffer and then returns a window's
>> width worth of characters of each line in the temp buffer to
>> be written to the output buffer. The output appears to fill each
>> line without the display slowdown due to long displayed lines.
>> And because hscroll is turned off, the text merely flows up the
>> screen quickly.
>
> Have you tried the (presumably) simpler approach of only inserting the
> text one line at a time, i.e. accumulate the output in some undisplayed
> "buffer area" (can be a buffer, a string, or a list of strings, this
> last choice being probably the most efficient) until you see a \n and
> then insert that text (upto and including the \n, so the cursor stays in
> column 0 and doesn't risk triggering auto-hscroll-mode) while keeping
> the rest in the "buffer area"?
>
>
> Stefan
>
I hate that after beating my head against the wall for a month, you can
say "Have you tried..." and I'm sitting here going "Duh!". ;-) Thanks.
You are correct that by only passing along the incoming text thru
(and including) the last newline of the preoutput text does reduce the
auto-hscroll-mode related flickering. The trick is detecting the prompt
at the end of the output to force the filter to flush everything. The
implementation is very straightforward just using a string to hold
what is left over from the prior iteration.
Comparing the results to not having the filter, is dramatic. I have a
query that produces about 200 rows of data x 8000 characters per
row (the database columns are waaaay too long for the data present,
but that's what I have to live with). Without the filter and
auto-hscroll-mode engaged, the results take nearly 30 seconds to
complete under Emacs. (It is nearly instantaneous outside of Emacs.)
With the filter you suggest, the results take about 8-10 seconds with
a noticeable stutter between each row. With the solution I had
developed (buffering the long lines in a buffer and restoring them at
the end of output), the results scroll by quickly in about 2-3 seconds,
but then it hesitates at the end (as it restores the long lines in the
buffer), for another 2-3 seconds. So just reducing the horizontal
scrolling is giving us the big payback and the more complex
gymnastics is probably not worth code cost.
Thank you again, Stefan. Thanks for the suggestion and clarifying my
thinking about how process output and display is handled in Emacs.
Your leadership and patience is appreciated.
I'll be adding this filtering to an existing output filter in sql.el.
-- Michael
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deffering redisplay in COMINT
2013-01-11 14:40 ` Michael Mauger
@ 2013-01-11 15:55 ` Stefan Monnier
2013-01-13 8:14 ` Michael Mauger
0 siblings, 1 reply; 9+ messages in thread
From: Stefan Monnier @ 2013-01-11 15:55 UTC (permalink / raw)
To: Michael Mauger; +Cc: Vitalie Spinu, Emacs Devel
> but that's what I have to live with). Without the filter and
> auto-hscroll-mode engaged, the results take nearly 30 seconds to
> complete under Emacs. (It is nearly instantaneous outside of Emacs.)
> With the filter you suggest, the results take about 8-10 seconds with
> a noticeable stutter between each row. With the solution I had
> developed (buffering the long lines in a buffer and restoring them at
> the end of output), the results scroll by quickly in about 2-3 seconds,
> but then it hesitates at the end (as it restores the long lines in the
> buffer), for another 2-3 seconds. So just reducing the horizontal
> scrolling is giving us the big payback and the more complex
> gymnastics is probably not worth code cost.
Hmm... I'm wondering why the "one line at a time" approach ends up
slightly slower than your more complex approach. Using a single string
occurs an O(N^2) overhead (where N is the line length: we receive O(N)
packets per line and for each package we do a concat which costs O(N)),
so that might be the reason (which is why I suggested using a list of
strings, tho it makes detection of the prompt more cumbersome), but for
8KB lines it's not clear it would be that bad.
Could you M-x profiler-start ... M-x profiler-report around your tests
to see where the time is spent?
Stefan
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deffering redisplay in COMINT
2013-01-11 15:55 ` Stefan Monnier
@ 2013-01-13 8:14 ` Michael Mauger
2013-01-13 16:46 ` Eli Zaretskii
2013-01-16 14:39 ` Stefan Monnier
0 siblings, 2 replies; 9+ messages in thread
From: Michael Mauger @ 2013-01-13 8:14 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Vitalie Spinu, Emacs Devel
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Sent: Friday, January 11, 2013 10:55 AM
>
>> but that's what I have to live with). Without the filter and
>> auto-hscroll-mode engaged, the results take nearly 30 seconds to
>> complete under Emacs. (It is nearly instantaneous outside of Emacs.)
>> With the filter you suggest, the results take about 8-10 seconds with
>> a noticeable stutter between each row. With the solution I had
>> developed (buffering the long lines in a buffer and restoring them at
>> the end of output), the results scroll by quickly in about 2-3 seconds,
>> but then it hesitates at the end (as it restores the long lines in the
>> buffer), for another 2-3 seconds. So just reducing the horizontal
>> scrolling is giving us the big payback and the more complex
>> gymnastics is probably not worth code cost.
>
> Hmm... I'm wondering why the "one line at a time" approach ends up
> slightly slower than your more complex approach. Using a single string
> occurs an O(N^2) overhead (where N is the line length: we receive O(N)
> packets per line and for each package we do a concat which costs O(N)),
> so that might be the reason (which is why I suggested using a list of
> strings, tho it makes detection of the prompt more cumbersome), but for
> 8KB lines it's not clear it would be that bad.
>
> Could you M-x profiler-start ... M-x profiler-report around your tests
> to see where the time is spent?
>
>
> Stefan
>
I ran several different configurations trying to identify important
criteria related to the display of this large block of text.
Testing ran a single database query that returns 289 rows of data
and 6000 characters per row. For a total of 1.7MB of data.
First, outside of Emacs, at the bash prompt on a GNU/Linux laptop.
$ time sqlite3 test.db <test_big.sql | wc
292 1684 1747492
real 0m0.175s
user0m0.140s
sys 0m0.004s
Basically, the database is not a constraint on performance.
I ran all tests with -Q. Each test was run in two display configurations:
* (NW) an 80x40 text terminal (GNOME Terminal 2.30.2) window, and
* (GUI) a similar sized X11 frame (GNU Emacs 24.3.50.1
(i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of 2012-12-20)
(BZR revno:111283)
In each display environment I ran the tests with `truncate-lines' set to
`t' and then with it set to`nil'.
I then ran three different output filtering configurations:
* (BASE) Just the OOB sql.el module with it's output filters (eats
leading continuation prompts produced by multi-line commands)
* (SS) Simple filter that emits text thru the last newline and holds
the remainder for the next iteration. Avoids any horizontal scrolling.
* (HSC) My more complex minor mode that passes along the first 90
characters of each line and at the final prompt, it goes back and
inserts the remainder of each output line in the output buffer.
Below is a summary of the user experienced elapsed time of each of these
tests:
These numbers differ from what I had reported earlier but those were
based on my gut feel for the performance and based on a Cygwin/Win7
executable running on a much beefier Core i7 machine at work.
Display| GUI/X| Text/NW|
Truncate-lines| nil| t| nil| t|
---Filter---| -----| -----| -----| -----|
BASE| 3:43 | 5:51| 0:16 | 4:20 |
SS| 3:42 | 3:06 | 0:12 | 2:04 |
HSC| N/A | 0:07 | N/A | 0:04 |
* HSC is disabled when `truncate-lines' is nil; times in those cases
are the same as BASE
* HSC samples both include 1-2 seconds of hanging at the end as the
lines are updated by appending the filtered off-screen text
The two obvious takeaways from this is that GUIs are slower, and
truncate-lines set to non-nil can be painful.
I have the cpu profile saved for each of the tests and have included a
summary of some of them below. I've been staring at numbers for
the last three days so I'm getting a little cross-eyed. Let me know
if there is something more that would help you.
Here's the CPU profiler report for:
SS / NW / Trunc (2:04)
===============
- redisplay_internal (C function) 117989 97%
- jit-lock-function 1832 1%
- jit-lock-fontify-now 1832 1%
- byte-code 1820 1%
- run-hook-with-args 1820 1%
- font-lock-fontify-region 1820 1%
- font-lock-default-fontify-region 1816 1%
- font-lock-fontify-keywords-region 1812 1%
re-search-forward 1800 1%
font-lock-extend-region-wholelines 4 0%
font-lock-set-defaults 4 0%
line-beginning-position 8 0%
match-data 4 0%
- eval 28 0%
- if 20 0%
display-graphic-p 12 0%
frame-parameter 8 0%
- unless 4 0%
- if 4 0%
- display-graphic-p 4 0%
framep-on-display 4 0%
- menu-bar-update-buffers 16 0%
lookup-key 16 0%
- file-remote-p 12 0%
find-file-name-handler 12 0%
- keymap-canonicalize 7 0%
keymap-prompt 4 0%
- map-keymap 3 0%
#<compiled 0x2092fc9> 3 0%
- comint-output-filter 2511 2%
- run-hook-with-args 2150 1%
- comint-watch-for-password-prompt 1393 1%
string-match 1393 1%
- comint-postoutput-scroll-to-bottom 634 0%
recenter 611 0%
- get-buffer-window-list 8 0%
window-list-1 4 0%
vconcat 4 0%
- ansi-color-process-output 123 0%
- ansi-color-apply-on-region 123 0%
re-search-forward 123 0%
- sql-interactive-remove-continuation-prompt 273 0%
- if 270 0%
- progn 270 0%
- let 270 0%
- if 266 0%
string-match 181 0%
- while 77 0%
string-match 77 0%
- setq 4 0%
substring 4 0%
- setq 4 0%
concat 4 0%
string-match 60 0%
- jit-lock-after-change 4 0%
put-text-property 4 0%
comint-carriage-motion 4 0%
buffer-name 4 0%
SS / NW / NoTrunc (0:12)
=================
- redisplay_internal (C function) 9403 80%
- jit-lock-function 1823 15%
- jit-lock-fontify-now 1823 15%
- byte-code 1792 15%
- run-hook-with-args 1792 15%
- font-lock-fontify-region 1792 15%
- font-lock-default-fontify-region 1792 15%
- font-lock-fontify-keywords-region 1789 15%
re-search-forward 1785 15%
font-lock-extend-region-wholelines 3 0%
- run-with-timer 16 0%
- apply 12 0%
- run-at-time 12 0%
- timer-relative-time 8 0%
time-add 4 0%
mod 4 0%
current-time 4 0%
line-beginning-position 7 0%
text-property-any 4 0%
- eval 30 0%
- if 14 0%
frame-parameter 11 0%
display-graphic-p 3 0%
mode-line-frame-control 4 0%
mode-line-eol-desc 4 0%
- menu-bar-update-buffers 15 0%
lookup-key 8 0%
frame-or-buffer-changed-p 3 0%
- file-remote-p 8 0%
find-file-name-handler 8 0%
- keymap-canonicalize 8 0%
- map-keymap 8 0%
#<compiled 0x2092fc9> 4 0%
- comint-output-filter 2058 17%
- run-hook-with-args 1700 14%
- comint-watch-for-password-prompt 1373 11%
string-match 1373 11%
- comint-postoutput-scroll-to-bottom 179 1%
recenter 175 1%
get-buffer-window-list 4 0%
- ansi-color-process-output 148 1%
- ansi-color-apply-on-region 148 1%
re-search-forward 145 1%
- sql-interactive-remove-continuation-prompt 214 1%
- if 214 1%
- progn 211 1%
- let 211 1%
- if 201 1%
string-match 137 1%
- while 60 0%
string-match 60 0%
- setq 4 0%
substring 4 0%
- setq 10 0%
concat 10 0%
string-match 74 0%
comint-carriage-motion 19 0%
- jit-lock-after-change 11 0%
- run-hook-with-args 4 0%
- font-lock-extend-jit-lock-region-after-cha 4 0%
line-beginning-position 4 0%
byte-code 3 0%
move-overlay 4 0%
Automatic GC 162 1%
HSC / NW / Trunc (0:04)
================
- redisplay_internal (C function) 3501 48%
- jit-lock-function 386 5%
- jit-lock-fontify-now 386 5%
- byte-code 359 5%
- run-hook-with-args 359 5%
- font-lock-fontify-region 359 5%
- font-lock-default-fontify-region 359 5%
- font-lock-fontify-keywords-region 359 5%
re-search-forward 355 4%
make-marker 4 0%
- run-with-timer 15 0%
- apply 15 0%
- run-at-time 15 0%
- timer-relative-time 11 0%
time-add 7 0%
floor 4 0%
timer-set-function 4 0%
- eval 28 0%
- if 20 0%
frame-parameter 12 0%
- display-graphic-p 8 0%
framep-on-display 4 0%
- mode-line-eol-desc 8 0%
coding-system-eol-type-mnemonic 8 0%
keymap-canonicalize 11 0%
- menu-bar-update-buffers 8 0%
lookup-key 4 0%
frame-or-buffer-changed-p 4 0%
- file-remote-p 8 0%
find-file-name-handler 8 0%
- comint-output-filter 3185 44%
- run-hook-with-args 3084 43%
- sql-hscroll-post-output 2927 40%
- if 2927 40%
- progn 2904 40%
- let 2904 40%
- save-excursion 2904 40%
- while 2904 40%
- move-end-of-line 2852 39%
- line-move 2852 39%
- line-move-1 2852 39%
vertical-motion 2836 39%
- byte-code 12 0%
line-move-finish 12 0%
- insert 36 0%
- jit-lock-after-change 8 0%
put-text-property 4 0%
- apply 8 0%
propertize 4 0%
quote 4 0%
- save-current-buffer 16 0%
- setq 12 0%
- not 8 0%
- zerop 8 0%
forward-line 8 0%
line-end-position 4 0%
set-buffer 4 0%
- and 23 0%
- sql--hscroll-at-end-p 23 0%
- save-current-buffer 23 0%
- looking-back 23 0%
re-search-backward 19 0%
- comint-postoutput-scroll-to-bottom 133 1%
recenter 125 1%
vconcat 4 0%
select-window 4 0%
- comint-watch-for-password-prompt 12 0%
string-match 12 0%
- ansi-color-process-output 8 0%
- ansi-color-apply-on-region 8 0%
ansi-color-apply-overlay-face 4 0%
ansi-color--find-face 4 0%
- sql-hscroll-pre-output 55 0%
- if 55 0%
- let 55 0%
- save-current-buffer 55 0%
- sql--hscroll-copy-output 51 0%
- save-excursion 25 0%
insert 25 0%
- let* 19 0%
line-beginning-position 8 0%
- while 4 0%
- setq 4 0%
line-end-position 4 0%
line-end-position 3 0%
goto-char 7 0%
- looking-back 16 0%
re-search-backward 16 0%
- comint-carriage-motion 8 0%
line-beginning-position 4 0%
move-overlay 3 0%
- sql-interactive-remove-continuation-prompt 3 0%
- if 3 0%
- progn 3 0%
- let 3 0%
- setq 3 0%
concat 3 0%
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deffering redisplay in COMINT
2013-01-13 8:14 ` Michael Mauger
@ 2013-01-13 16:46 ` Eli Zaretskii
2013-01-16 14:39 ` Stefan Monnier
1 sibling, 0 replies; 9+ messages in thread
From: Eli Zaretskii @ 2013-01-13 16:46 UTC (permalink / raw)
To: Michael Mauger; +Cc: spinuvit, monnier, emacs-devel
> Date: Sun, 13 Jan 2013 00:14:49 -0800 (PST)
> From: Michael Mauger <mmaug@yahoo.com>
> Cc: Vitalie Spinu <spinuvit@gmail.com>, Emacs Devel <emacs-devel@gnu.org>
>
> In each display environment I ran the tests with `truncate-lines' set to
> `t' and then with it set to`nil'.
When it was non-nil, did you also disable auto-hscroll-mode?
Displaying an hscrolled windows disables redisplay optimizations, so
redisplay will be much slower (and it already is slow with such long
lines).
> The two obvious takeaways from this is that GUIs are slower, and
> truncate-lines set to non-nil can be painful.
That's almost trivial.
> I've been staring at numbers for the last three days so I'm getting
> a little cross-eyed. Let me know if there is something more that
> would help you.
AFAIU, the profiles say that redisplay takes the lion's share of the
time, and the less you trigger redisplay, the faster Emacs is. Is
that true? If so, this, too, should be almost obvious, since
displaying long lines is notoriously slow in Emacs.
Once again, why don't you collect the text in a buffer that is not
displayed until you actually want to present it to the user?
Also, if the text is known to be 7-bit ASCII, try displaying it in a
unibyte buffer, that should be faster in Emacs 24.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deffering redisplay in COMINT
2013-01-13 8:14 ` Michael Mauger
2013-01-13 16:46 ` Eli Zaretskii
@ 2013-01-16 14:39 ` Stefan Monnier
1 sibling, 0 replies; 9+ messages in thread
From: Stefan Monnier @ 2013-01-16 14:39 UTC (permalink / raw)
To: Michael Mauger; +Cc: Vitalie Spinu, Emacs Devel
> I ran several different configurations trying to identify important
> criteria related to the display of this large block of text.
> Display | GUI/X | Text/NW|
> Truncate-lines | nil | t | nil | t|
> ---Filter--- | ----- | ----- | ----- | -----|
> BASE | 3:43 | 5:51 | 0:16 | 4:20 |
> SS | 3:42 | 3:06 | 0:12 | 2:04 |
> HSC | N/A | 0:07 | N/A | 0:04 |
From your tests, it seems clear that the problem is always redisplay
time, which is why the extra work of HSC pays off.
I think I can understand the difference between truncate and
non-truncate on a tty (the amount of buffer text redisplay has to
consider (i.e. (- (window-end) (window-start))) when refreshing the
display is larger (proportional to the line lengths) than with
wrap-around). But I don't understand why we don't see the same impact
in the GUI case. IOW I really don't understand the "3:42" of SS+GUI
with truncate-lines==nil. I'd expect it to be *much* lower (like 0:20
or so).
Maybe we could speed it up the 2:04 case with a redisplay optimization that
skips more quickly over the truncated part of the lines (special casing
the "easy common case where there's no before/after-string, no `display'
property, ...").
You could try and tweak the SS code further to reduce the number of
redisplays. E.g. not only you "wait for a complete line" but you also
"wait for the next second" so the buffer is only redisplayed once
a second.
Or you could try to do something halfway between SS and HSC: instead of
keeping the non-displayed part of lines in a separate buffer (as in
HSC), you do insert them (as in SS) but with an `invisible'
text-property set to t so that the redisplay can (hopefully) skip over
it very quickly.
Stefan
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2013-01-16 14:39 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-12-07 1:08 Deffering redisplay in COMINT Michael Mauger
2012-12-25 11:09 ` Vitalie Spinu
2013-01-06 8:09 ` Michael Mauger
2013-01-08 18:19 ` Stefan Monnier
2013-01-11 14:40 ` Michael Mauger
2013-01-11 15:55 ` Stefan Monnier
2013-01-13 8:14 ` Michael Mauger
2013-01-13 16:46 ` Eli Zaretskii
2013-01-16 14:39 ` Stefan Monnier
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).