Re: Long lines and bidi [Was: Re: bug#13623: ...]

[Dmitry Antipov <dmantipov@yandex.ru>]

On 02/08/2013 06:07 PM, Eli Zaretskii wrote:

> Profile alone is not enough.  Please tell how did you "scroll",
> exactly (which commands did you use), and please also show the
> absolute times it took to perform each command.

(defun scroll-both ()
   (interactive)
   (let ((start (float-time)))
     (progn
       (dotimes (n 100) (progn (scroll-up) (redisplay)))
       (goto-char (point-max))
       (dotimes (n 100) (progn (scroll-down) (redisplay)))
       (message "Elapsed %f seconds" (- (float-time) start)))))

With bidi, ~600 second elapsed, and:

     25.18%        emacs  emacs                          [.] scan_buffer
      7.04%        emacs  emacs                          [.] bidi_resolve_weak
      6.47%        emacs  emacs                          [.] get_next_display_element
      6.37%        emacs  emacs                          [.] bidi_level_of_next_char
      5.14%        emacs  libc-2.16.so                   [.] __memcpy_ssse3_back
      5.05%        emacs  emacs                          [.] move_it_in_display_line_to
      4.94%        emacs  emacs                          [.] x_produce_glyphs
      4.84%        emacs  libXft.so.2.3.1                [.] XftCharIndex
      3.72%        emacs  emacs                          [.] bidi_move_to_visually_next
      3.70%        emacs  emacs                          [.] next_element_from_buffer
      2.90%        emacs  libXft.so.2.3.1                [.] XftGlyphExtents
      2.05%        emacs  emacs                          [.] bidi_fetch_char
      2.02%        emacs  emacs                          [.] lookup_glyphless_char_display
      2.01%        emacs  emacs                          [.] bidi_resolve_neutral
      1.76%        emacs  emacs                          [.] bidi_cache_iterator_state
      1.70%        emacs  emacs                          [.] bidi_get_type
      1.51%        emacs  emacs                          [.] bidi_resolve_explicit_1
      1.18%        emacs  libXft.so.2.3.1                [.] XftFontCheckGlyph
      1.12%        emacs  emacs                          [.] xftfont_encode_char
      1.01%        emacs  emacs                          [.] xftfont_text_extents

Without bidi, ~230 seconds elapsed, and:

     21.36%        emacs  emacs                          [.] x_produce_glyphs
     17.92%        emacs  emacs                          [.] get_next_display_element
     15.07%        emacs  emacs                          [.] move_it_in_display_line_to
      8.37%        emacs  emacs                          [.] next_element_from_buffer
      8.34%        emacs  libXft.so.2.3.1                [.] XftCharIndex
      6.12%        emacs  emacs                          [.] lookup_glyphless_char_display
      4.21%        emacs  libXft.so.2.3.1                [.] XftGlyphExtents
      3.07%        emacs  emacs                          [.] xftfont_encode_char
      2.68%        emacs  emacs                          [.] xftfont_text_extents
      1.87%        emacs  emacs                          [.] get_per_char_metric
      1.53%        emacs  libXft.so.2.3.1                [.] XftFontCheckGlyph
      1.49%        emacs  emacs                          [.] composition_compute_stop_pos
      1.35%        emacs  emacs                          [.] set_iterator_to_next

cache-long-line-scans is nil in both cases.

I suspect that scroll should be direction-agnostic in theory; but both profiled
runs shows that scroll-down is much, much slower than scroll-up (that's why
elapsed time is so huge in both cases).

> What was in the file?  bidi_resolve_weak high on the profile hints
> that it was full of punctuation or digits or banks, which is not
> really an interesting case.

Your guess is correct; but I suspect that an average text in human language
contains less punctuations, digits and blanks than the C source code of the
same size :-).

> Ah, _that_ red herring...  Why is that the first question?  What were
> the times with and without bidi-display-reordering in this file?  In
> my testing, the display engine performs awfully slow in both cases, so
> even though turning off reordering makes it faster, it is still so
> terribly slow that the problem is not going to be solved by that.
>
> As to your question: how can we know what characters are or aren't in
> the buffer without scanning it?  And scanning the buffer is exactly
> what bidi.c does.

Hm... insert-file-contents tries to detect encoding by looking at first 1K
and last 3K of the file. Why the similar approach isn't applicable to bidi?

Dmitry