bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>  v22519@debbugs.gnu.org
> Date: Sat, 12 Mar 2016 18:29:03 +0100
> 
> I have rebuilt Emacs from the latest source on the master branch, and
> now it works fine : to that respect there is a huge difference with the
> version of 2016-01-29 which I was using. FYI the version of 2016-01-29
> was still slow even after the 1st scroll through that node, the slow
> down was not going away, and that was very disturbing any edition.
> 
> Now there is no longer this problem, just some slight slow down on the
> first visit of this node, which is fully acceptable not a problem.

So can we close this bug?

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

Le 12/03/2016 18:46, Eli Zaretskii a écrit :
>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>>  v22519@debbugs.gnu.org
>> Date: Sat, 12 Mar 2016 18:29:03 +0100
>>
>> I have rebuilt Emacs from the latest source on the master branch, and
>> now it works fine : to that respect there is a huge difference with the
>> version of 2016-01-29 which I was using. FYI the version of 2016-01-29
>> was still slow even after the 1st scroll through that node, the slow
>> down was not going away, and that was very disturbing any edition.
>>
>> Now there is no longer this problem, just some slight slow down on the
>> first visit of this node, which is fully acceptable not a problem.
>
> So can we close this bug?

Yes, definitely, sorry for the disturbance, I should have rebuilt Emacs
first.

  Vincent.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

Le 13/03/2016 08:11, Vincent Belaïche a écrit :
>
> Le 12/03/2016 18:46, Eli Zaretskii a écrit :
>>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>>>  v22519@debbugs.gnu.org
>>> Date: Sat, 12 Mar 2016 18:29:03 +0100
>>>
>>> I have rebuilt Emacs from the latest source on the master branch, and
>>> now it works fine : to that respect there is a huge difference with the
>>> version of 2016-01-29 which I was using. FYI the version of 2016-01-29
>>> was still slow even after the 1st scroll through that node, the slow
>>> down was not going away, and that was very disturbing any edition.
>>>
>>> Now there is no longer this problem, just some slight slow down on the
>>> first visit of this node, which is fully acceptable not a problem.
>>
>> So can we close this bug?
>
> Yes, definitely, sorry for the disturbance, I should have rebuilt Emacs
> first.
>
>   Vincent.

Oooops... It is back again. It seems that this has to do with how long
the Emacs session has been open. I was comparing the 2016-01-29 Emacs
the session of which had been open for a couple of days with the
recently build 2016-03-11 Emacs the session of which was just open. And
the latter seemed not to have the problem any longer.

Not correct, the editing is lengthy *ALSO* with the latter Emacs.

Please note that only the latex2e-fr.texi buffer is slow, the other
buffers behave OK.

   Vincent.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>  22519@debbugs.gnu.org
> Date: Sun, 13 Mar 2016 09:10:11 +0100
> 
> Oooops... It is back again. It seems that this has to do with how long
> the Emacs session has been open. I was comparing the 2016-01-29 Emacs
> the session of which had been open for a couple of days with the
> recently build 2016-03-11 Emacs the session of which was just open. And
> the latter seemed not to have the problem any longer.

I have now tried this file in a session that has been running for 10
days, and I see no problems with this file.

> Not correct, the editing is lengthy *ALSO* with the latter Emacs.
> 
> Please note that only the latex2e-fr.texi buffer is slow, the other
> buffers behave OK.

I'm afraid this is something specific to your configuration, so you
will have to find which parts of your configuration cause this issue.
Also, you originally talked about Emacs getting stuck, while now you
seem to say about slow editing -- is this really the same problem, and
if so, does it get stuck or just works slowly?

In any case, without additional information I don't see how we could
move further in debugging this problem.  I'd begin with looking into
the minor modes you have enabled in that buffer, disabling them one by
one, until the problem goes way.  If that doesn't help, look at
features that hook into post-command-hook.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

[-- Attachment #1: Type: text/plain, Size: 2372 bytes --]

Dear Eli,

Ok, I decided to make some test bench (herein attached).

I get the following interaction:

 $ ./emacs-bug22519.sh --fundamental no
-| mode: normal
-| latin-9=(0.588 5 0.031999999999999994)
-| utf-8=(111.257 146 1.3299999999999979)
 $ ./emacs-bug22519.sh --fundamental yes
-| mode: fundamental
-| latin-9=(0.209 0 0.0)
-| utf-8=(95.442 129 1.121)


As you see, in normal mode utf-8 is 189 times slower than latin-9, and
in fundamental mode, it is 456 times slower.

So, IMHO it is a bug: user expererience is *GREATLY* damaged when you
have the funny math symbols displayed.

Please feel free to ask me to modify the attached script according to
your wishes, eg to new command line option, like making configurable
number of word insertion for shorter tests.

VBR,
	Vincent.

Le 13/03/2016 17:19, Eli Zaretskii a écrit :
>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>>  22519@debbugs.gnu.org
>> Date: Sun, 13 Mar 2016 09:10:11 +0100
>>
>> Oooops... It is back again. It seems that this has to do with how long
>> the Emacs session has been open. I was comparing the 2016-01-29 Emacs
>> the session of which had been open for a couple of days with the
>> recently build 2016-03-11 Emacs the session of which was just open. And
>> the latter seemed not to have the problem any longer.
>
> I have now tried this file in a session that has been running for 10
> days, and I see no problems with this file.
>
>> Not correct, the editing is lengthy *ALSO* with the latter Emacs.
>>
>> Please note that only the latex2e-fr.texi buffer is slow, the other
>> buffers behave OK.
>
> I'm afraid this is something specific to your configuration, so you
> will have to find which parts of your configuration cause this issue.
> Also, you originally talked about Emacs getting stuck, while now you
> seem to say about slow editing -- is this really the same problem, and
> if so, does it get stuck or just works slowly?
>
> In any case, without additional information I don't see how we could
> move further in debugging this problem.  I'd begin with looking into
> the minor modes you have enabled in that buffer, disabling them one by
> one, until the problem goes way.  If that doesn't help, look at
> features that hook into post-command-hook.


[-- Attachment #2: Benchmark --]
[-- Type: text/plain, Size: 3325 bytes --]

#!/bin/bash

# Instruction for use:
# ~~~~~~~~~~~~~~~~~~~~
# 1) Copy this script into some temporary directory, and
# 2) Download
#    svn.gna.org/viewcvs/*checkout*/latexrefman/trunk/latex2e-fr.texi
#    to some place
# 3) Adapt the two variables TEXI_FILE & EMACS to your configuration, under
#    some unsername entry distincitive of you.
# 4) Run the script.



case $USERNAME in
    Vincent)
	TEXI_FILE=/local/projects/latexrefman/latexrefman/latex2e-fr.texi
	EMACS=/c/Nos_Programmes/GNU/Emacs/bin/emacs.exe;;
    
    *)
	echo "Uknown user!!" >2
	exit;;
esac

if [ "$1" == "--fundamental" ]; then
    FUNDAMENTAL=$2
fi    
case $FUNDAMENTAL in
    yes|true|1) FUNDAMENTAL=1;;
    no|false|0) FUNDAMENTAL=0;;
    *) FUNDAMENTAL=1;;
esac

HERE=$(dirname $0)

# We make some AWK script to remove of the file local variable that
# are not relevant for the test, and configure the coding either to
# latin-9 or to utf-8. We will show that utf-8 (ie math symbol are
# displayed) is horribly slower than latin-9
cat >emacs-bug22519.awk <<EOF
BEGIN                        { enb = 0}
/^@c Local Variables:/       { enb = 1; print}
/^@c End:/                   { enb = 0 }
enb == 1 && \$2 == "coding:" { \$3=coding "-unix";
                               print;
                               if(fundamental) 
                                 print "@c mode: fundamental"
                             }
enb == 0                     { print }
EOF

for CODING in utf-8 latin-9; do
    awk -v coding=$CODING \
	-v fundamental=$FUNDAMENTAL \
	-f emacs-bug22519.awk \
	$TEXI_FILE > $HERE/${CODING}-l2e-fr.texi
done

# Now we make some Emacs script to make the benchmarking. Benchmark
# result is saved to some result.txt file
cat >$HERE/emacs-bug22519.el <<EOF
(defconst max-word-size 10)
(defconst rand-word-string-size 1000)
(defconst rand-word-string (apply 'string
   (let (l)
      (random "Some first seed")
      (dotimes (i (+ max-word-size rand-word-string-size))
         (push (+ 32 (random 80)) l))
      l)))
(let (result)
  (dolist
    (f '("latin-9" "utf-8"))
    (find-file (concat "$HERE/" f "-l2e-fr.texi"))
    (re-search-forward "@node Math symbols")
    (beginning-of-line 10)
    (let ((beg (point))
          (end (save-excursion
                   (re-search-forward "^@node")
                   (beginning-of-line)
                   (point)))
          pos ws spos)
        (push f result)
	(random "Some second seed")
        (push
           (benchmark-run
             (dotimes (i 200)
               (setq pos  (random (- end beg))
                     ws   (1+ (random max-word-size))
	             spos (random rand-word-string-size))
               (goto-char (+ beg pos))
               (insert (substring rand-word-string spos (+ spos ws)))
               (recenter) ; force refreshing the screen !!              
               (setq end (+ end ws))))
           result)
	(set-buffer-modified-p nil)
        (kill-buffer)))
      (setq result (nreverse result))
     (find-file "$HERE/result.txt")
     (erase-buffer)
     (while result
       (insert (pop result) "=" (format "%S" (pop result)) "\n")))
(save-buffers-kill-terminal t)
EOF

$EMACS -Q --load "$HERE/emacs-bug22519.el"

case $FUNDAMENTAL in
     1) echo "mode: fundamental";;
     0) echo "mode: normal";;
esac
cat $HERE/result.txt

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

Dear Eli,

Just one more word: if I omit the `(recenter)' statement after each word
insertion, then processing time is OK, utf-8 does it even a little
shorter than latin-9, probably due to this there are fewer characters in
the buffer.

So, it really has to do with display refresh.

VBR,
	Vincent.


Le 15/03/2016 14:28, Vincent Belaïche a écrit :
> Dear Eli,
>
> Ok, I decided to make some test bench (herein attached).
>
> I get the following interaction:
>
>  $ ./emacs-bug22519.sh --fundamental no
> -| mode: normal
> -| latin-9=(0.588 5 0.031999999999999994)
> -| utf-8=(111.257 146 1.3299999999999979)
>  $ ./emacs-bug22519.sh --fundamental yes
> -| mode: fundamental
> -| latin-9=(0.209 0 0.0)
> -| utf-8=(95.442 129 1.121)
>
>
> As you see, in normal mode utf-8 is 189 times slower than latin-9, and
> in fundamental mode, it is 456 times slower.
>
> So, IMHO it is a bug: user expererience is *GREATLY* damaged when you
> have the funny math symbols displayed.
>
> Please feel free to ask me to modify the attached script according to
> your wishes, eg to new command line option, like making configurable
> number of word insertion for shorter tests.
>
> VBR,
> 	Vincent.
>
> Le 13/03/2016 17:19, Eli Zaretskii a écrit :
>>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>>>  22519@debbugs.gnu.org
>>> Date: Sun, 13 Mar 2016 09:10:11 +0100
>>>
>>> Oooops... It is back again. It seems that this has to do with how long
>>> the Emacs session has been open. I was comparing the 2016-01-29 Emacs
>>> the session of which had been open for a couple of days with the
>>> recently build 2016-03-11 Emacs the session of which was just open. And
>>> the latter seemed not to have the problem any longer.
>>
>> I have now tried this file in a session that has been running for 10
>> days, and I see no problems with this file.
>>
>>> Not correct, the editing is lengthy *ALSO* with the latter Emacs.
>>>
>>> Please note that only the latex2e-fr.texi buffer is slow, the other
>>> buffers behave OK.
>>
>> I'm afraid this is something specific to your configuration, so you
>> will have to find which parts of your configuration cause this issue.
>> Also, you originally talked about Emacs getting stuck, while now you
>> seem to say about slow editing -- is this really the same problem, and
>> if so, does it get stuck or just works slowly?
>>
>> In any case, without additional information I don't see how we could
>> move further in debugging this problem.  I'd begin with looking into
>> the minor modes you have enabled in that buffer, disabling them one by
>> one, until the problem goes way.  If that doesn't help, look at
>> features that hook into post-command-hook.
>

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>  22519@debbugs.gnu.org
> Date: Tue, 15 Mar 2016 14:28:18 +0100
> 
> Ok, I decided to make some test bench (herein attached).
> 
> I get the following interaction:
> 
>  $ ./emacs-bug22519.sh --fundamental no
> -| mode: normal
> -| latin-9=(0.588 5 0.031999999999999994)
> -| utf-8=(111.257 146 1.3299999999999979)
>  $ ./emacs-bug22519.sh --fundamental yes
> -| mode: fundamental
> -| latin-9=(0.209 0 0.0)
> -| utf-8=(95.442 129 1.121)
> 
> As you see, in normal mode utf-8 is 189 times slower than latin-9, and
> in fundamental mode, it is 456 times slower.

My results are in stark contrast to yours:

 . with the current emacs-25 branch, an unoptimized build:

    $ ./emacs-bug22519.sh --fundamental no
    mode: normal
    latin-9=(5.0 5 0.10899999999999999)
    utf-8=(10.14 13 0.44)
    $ ./emacs-bug22519.sh --fundamental yes
    mode: fundamental
    latin-9=(0.359 0 0.0)
    utf-8=(2.938 4 0.126)

 . with Emacs 25.0.92 pretest, compiled with -Og:

    $ ./emacs-bug22519.sh --fundamental no
    mode: normal
    latin-9=(0.64 5 0.046)
    utf-8=(1.953 13 0.156)
    $ ./emacs-bug22519.sh --fundamental yes
    mode: fundamental
    latin-9=(0.109 0 0.0)
    utf-8=(0.984 4 0.062)

So while there is some slowdown when showing these symbols (which
could be due to the need to load the necessary font), the slowdown is
nowhere as dramatic as on your system.

IOW, I still cannot reproduce the problem on my system.  I suspect
some issue with your font configuration, but that's just a guess.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

[-- Attachment #1: Type: text/plain, Size: 4081 bytes --]

Dear Eli,

Answers below...

Le 15/03/2016 19:11, Eli Zaretskii a écrit :
>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>>  22519@debbugs.gnu.org
>> Date: Tue, 15 Mar 2016 14:28:18 +0100
>>
>> Ok, I decided to make some test bench (herein attached).
>>
>> I get the following interaction:
>>
>>  $ ./emacs-bug22519.sh --fundamental no
>> -| mode: normal
>> -| latin-9=(0.588 5 0.031999999999999994)
>> -| utf-8=(111.257 146 1.3299999999999979)
>>  $ ./emacs-bug22519.sh --fundamental yes
>> -| mode: fundamental
>> -| latin-9=(0.209 0 0.0)
>> -| utf-8=(95.442 129 1.121)
>>
>> As you see, in normal mode utf-8 is 189 times slower than latin-9, and
>> in fundamental mode, it is 456 times slower.
>
> My results are in stark contrast to yours:
>
>  . with the current emacs-25 branch, an unoptimized build:
>
>     $ ./emacs-bug22519.sh --fundamental no
>     mode: normal
>     latin-9=(5.0 5 0.10899999999999999)
>     utf-8=(10.14 13 0.44)
>     $ ./emacs-bug22519.sh --fundamental yes
>     mode: fundamental
>     latin-9=(0.359 0 0.0)
>     utf-8=(2.938 4 0.126)
>
>  . with Emacs 25.0.92 pretest, compiled with -Og:
>
>     $ ./emacs-bug22519.sh --fundamental no
>     mode: normal
>     latin-9=(0.64 5 0.046)
>     utf-8=(1.953 13 0.156)
>     $ ./emacs-bug22519.sh --fundamental yes
>     mode: fundamental
>     latin-9=(0.109 0 0.0)
>     utf-8=(0.984 4 0.062)
>
> So while there is some slowdown when showing these symbols (which
> could be due to the need to load the necessary font), the slowdown is
> nowhere as dramatic as on your system.

FYI, my Emacs was generated with the following flags:

  export CPPFLAGS="$CPPFLAGS -DFOR_MSW=1 -I ${HERE_DIR}/libXpm-3.5.8/include -I ${HERE_DIR}/libXpm-3.5.8/src -L ${HERE_DIR}/libXpm-3.5.8/src"
  export CFLAGS="$CFLAGS -Og -g3 -L ${HERE_DIR}/libXpm-3.5.8/src"

>
> IOW, I still cannot reproduce the problem on my system.  I suspect
> some issue with your font configuration, but that's just a guess.

I have the same type of suspicion that it has to do with the display. Ie
some code running outside Emacs in some standard system library of
Windows API. However, there may also be some suboptimal coding in the
ways that Emacs, or some graphic library used by Emacs, interacts with
these library fuctions.

I have attached a picture of my display, as you see I have a few not
displayable characters. They are displayed with small retangles
containing the Unicode code point in 4 hexadecimal letters --- let us
call this an hexa-rectangle.

My speculation is that every time MSWindows is requested for such not
displayable character, then it spends a lot of time seeking whether this
character can be displayed in any of the supported fonts, and if not, it
spends again time making on the fly the special hexa-rectangle
characters, or maybe those hexa-rectangle bitmaps are placed in some
memory area that is not fast enough...

So, do you have also any hexa-rectangles displayed on your system ?

Maybe you could try and reproduce the problem by replacing in the file
some of the math symbols by characters that are not displayable on your
machine. E.g. you could do some `M-: (insert (string #xhhhh)) RET' with
hhhh in { FFF0, FFC0, FFD0, FFC1, FFD1, FFE7, FFC8, FFD8, FFC9, FFD9,
FFDD, FFDE, FFBF, FFDF, FFEF, 0380, 0381, 0382, 03A2, 0383, 0377, 0378,
038B, 038D}, based on taking greyed cells from page 2 of these
documents:

   http://www.unicode.org/charts/PDF/UFF00.pdf
   http://www.unicode.org/charts/PDF/U0370.pdf

Just to make it as close to my situation, it is certainly better to take
a number of different values for hhhh.

BTW, even if we end up concluding that the problem is some sort of MSW
bug, I still think that there is also something in Emacs: now the `Emacs
stuck' does not happen so often, maybe this is due to this that there
has been some MSW update meanwhile, but still it can happen after system
wake-up.

VBR,
	Vincent.



[-- Attachment #2: Sans titre.png --]
[-- Type: image/png, Size: 114580 bytes --]

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>  22519@debbugs.gnu.org
> Date: Wed, 16 Mar 2016 12:16:00 +0100
> 
> > IOW, I still cannot reproduce the problem on my system.  I suspect
> > some issue with your font configuration, but that's just a guess.
> 
> I have the same type of suspicion that it has to do with the display. Ie
> some code running outside Emacs in some standard system library of
> Windows API. However, there may also be some suboptimal coding in the
> ways that Emacs, or some graphic library used by Emacs, interacts with
> these library fuctions.
> 
> I have attached a picture of my display, as you see I have a few not
> displayable characters. They are displayed with small retangles
> containing the Unicode code point in 4 hexadecimal letters --- let us
> call this an hexa-rectangle.
> 
> My speculation is that every time MSWindows is requested for such not
> displayable character, then it spends a lot of time seeking whether this
> character can be displayed in any of the supported fonts, and if not, it
> spends again time making on the fly the special hexa-rectangle
> characters, or maybe those hexa-rectangle bitmaps are placed in some
> memory area that is not fast enough...
> 
> So, do you have also any hexa-rectangles displayed on your system ?

Only one.  All the rest are displayed correctly.

Do you have the Symbola font installed?  If not, I suggest to install
it, then I expect your problem to go away.  If you need to work a lot
with mathematical symbols, having Symbola installed is a must, IMO.

> Maybe you could try and reproduce the problem by replacing in the file
> some of the math symbols by characters that are not displayable on your
> machine. E.g. you could do some `M-: (insert (string #xhhhh)) RET' with
> hhhh in { FFF0, FFC0, FFD0, FFC1, FFD1, FFE7, FFC8, FFD8, FFC9, FFD9,
> FFDD, FFDE, FFBF, FFDF, FFEF, 0380, 0381, 0382, 03A2, 0383, 0377, 0378,
> 038B, 038D}, based on taking greyed cells from page 2 of these
> documents:
> 
>    http://www.unicode.org/charts/PDF/UFF00.pdf
>    http://www.unicode.org/charts/PDF/U0370.pdf

I don't see the point of such a test.  What would it demonstrate?

> BTW, even if we end up concluding that the problem is some sort of MSW
> bug, I still think that there is also something in Emacs: now the `Emacs
> stuck' does not happen so often, maybe this is due to this that there
> has been some MSW update meanwhile, but still it can happen after system
> wake-up.

If you are talking about the fact that Emacs searches for a character
in any fonts it thinks might be able to display it, then this isn't a
bug.  What else could Emacs do, when presented with a character that
cannot be displayed with fonts that are already loaded?

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

Answers below:

Le 16/03/2016 17:08, Eli Zaretskii a écrit :
>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>>  22519@debbugs.gnu.org
>> Date: Wed, 16 Mar 2016 12:16:00 +0100
>>
>>> IOW, I still cannot reproduce the problem on my system.  I suspect
>>> some issue with your font configuration, but that's just a guess.
>>
>> I have the same type of suspicion that it has to do with the display. Ie
>> some code running outside Emacs in some standard system library of
>> Windows API. However, there may also be some suboptimal coding in the
>> ways that Emacs, or some graphic library used by Emacs, interacts with
>> these library fuctions.
>>
>> I have attached a picture of my display, as you see I have a few not
>> displayable characters. They are displayed with small retangles
>> containing the Unicode code point in 4 hexadecimal letters --- let us
>> call this an hexa-rectangle.
>>
>> My speculation is that every time MSWindows is requested for such not
>> displayable character, then it spends a lot of time seeking whether this
>> character can be displayed in any of the supported fonts, and if not, it
>> spends again time making on the fly the special hexa-rectangle
>> characters, or maybe those hexa-rectangle bitmaps are placed in some
>> memory area that is not fast enough...
>>
>> So, do you have also any hexa-rectangles displayed on your system ?
>
> Only one.  All the rest are displayed correctly.
>
> Do you have the Symbola font installed?  If not, I suggest to install
> it, then I expect your problem to go away.  If you need to work a lot
> with mathematical symbols, having Symbola installed is a must, IMO.
>

That was a good suggestion, I have just installed Symbola and here is
what I get:

$ ./emacs-bug22519.sh
mode: fundamental
latin-9=(0.234 0 0.0)
utf-8=(0.234 0 0.0)

That is a damatic change!

I think that I should maybe write some warning in the document for the
Info output. The slowdown would occur for anybody without appropriate
math font installed also for the English version when viewing the Info.

>> Maybe you could try and reproduce the problem by replacing in the file
>> some of the math symbols by characters that are not displayable on your
>> machine. E.g. you could do some `M-: (insert (string #xhhhh)) RET' with
>> hhhh in { FFF0, FFC0, FFD0, FFC1, FFD1, FFE7, FFC8, FFD8, FFC9, FFD9,
>> FFDD, FFDE, FFBF, FFDF, FFEF, 0380, 0381, 0382, 03A2, 0383, 0377, 0378,
>> 038B, 038D}, based on taking greyed cells from page 2 of these
>> documents:
>>
>>    http://www.unicode.org/charts/PDF/UFF00.pdf
>>    http://www.unicode.org/charts/PDF/U0370.pdf
>
> I don't see the point of such a test.  What would it demonstrate?
>

That would confirm that the not displayable characters are causing the
problem. But having Symbola installed solved the problem, seems to
confirm it per se.

>> BTW, even if we end up concluding that the problem is some sort of MSW
>> bug, I still think that there is also something in Emacs: now the `Emacs
>> stuck' does not happen so often, maybe this is due to this that there
>> has been some MSW update meanwhile, but still it can happen after system
>> wake-up.
>
> If you are talking about the fact that Emacs searches for a character
> in any fonts it thinks might be able to display it, then this isn't a
> bug.  What else could Emacs do, when presented with a character that
> cannot be displayed with fonts that are already loaded?

I don't this that _this_ as such is a bug, what is more annoying is that
the slow-down lasts longer than the first scroll into the area where
these characters are. I could stay on the same region for hours, and
whenever I made an edit it was slow --- loosing the WYSIWYGiness of the
edition, that is I was seeing the full result of what I had typed one or
two seconds after the typing had started.

Furthermore there was also this « Emacs stuck forever » problem which
was the original cause for opening bug #22519. This behaviour should not
happen, this clearly is a bug just like a crash is. It then became quite
less frequent after some MSW update, and now after Symbola installed
will probably remain as a silent bug, unless you try to reproduce the
problem by inserting undisplayable characters and understand why the
slowdown is lasting at any edit.

VBR,
	Vincent.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com>,
>   22519@debbugs.gnu.org
> Date: Wed, 16 Mar 2016 18:40:40 +0100
> 
> > If you are talking about the fact that Emacs searches for a character
> > in any fonts it thinks might be able to display it, then this isn't a
> > bug.  What else could Emacs do, when presented with a character that
> > cannot be displayed with fonts that are already loaded?
> 
> I don't this that _this_ as such is a bug, what is more annoying is that
> the slow-down lasts longer than the first scroll into the area where
> these characters are.

If the character wasn't found, then it makes sense to try looking up
the fonts again -- the user could install new fonts since the last
time.

> I could stay on the same region for hours, and whenever I made an
> edit it was slow --- loosing the WYSIWYGiness of the edition, that
> is I was seeing the full result of what I had typed one or two
> seconds after the typing had started.

Every redisplay might trigger another search for a suitable font.  The
solution really is to install the fonts you need.

Can we now close this bug?

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

[-- Attachment #1: Type: text/plain, Size: 2902 bytes --]

Answers below...

Le 16/03/2016 21:24, Eli Zaretskii a écrit :
>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com>,
>>   22519@debbugs.gnu.org
>> Date: Wed, 16 Mar 2016 18:40:40 +0100
>>
>>> If you are talking about the fact that Emacs searches for a character
>>> in any fonts it thinks might be able to display it, then this isn't a
>>> bug.  What else could Emacs do, when presented with a character that
>>> cannot be displayed with fonts that are already loaded?
>>
>> I don't this that _this_ as such is a bug, what is more annoying is that
>> the slow-down lasts longer than the first scroll into the area where
>> these characters are.
>
> If the character wasn't found, then it makes sense to try looking up
> the fonts again -- the user could install new fonts since the last
> time.

Yes, but there are several ways to make this polling for new fonts less
annoying:

- it would suffice to raise some character specific flag when a
  character is not found in any font in ordder to prevent subsequent
  search for the same character after an edit, and also to put this
  character in some queue so as to trigger the actual font search
  _again_ for the same character only when Emacs gets the focus.

- there could be some counter incremented every time a character cannot
  be displayed in a known font and alternative font would be extensively
  searched only when this counter is below some threshold, and this
  counter would be periodically reset. This method has however the
  drawback that the display would not be deterministic.

- I am wondering whether it is to the application to make this search,
  the system is supposed to know which fonts have been installed, isn't
  it possible that Emacs just ask to MSW for each character that cannot
  be displayed in the default font to supply the best alternative
  font. Then, since the OS is supposed to know whether or not a new font
  has been installed it may be more efficient in performing this search.

>
>> I could stay on the same region for hours, and whenever I made an
>> edit it was slow --- loosing the WYSIWYGiness of the edition, that
>> is I was seeing the full result of what I had typed one or two
>> seconds after the typing had started.
>
> Every redisplay might trigger another search for a suitable font.  The
> solution really is to install the fonts you need.

I agree that the solution is definitely to install the needed fonts. But
when the slow down occurred to me, I had no idea that this was the
solution. If Emacs had some sort of monitoring about display slow down
caused by non displayable characters and would make some warning to the
user that would certainly be an improvement.

>
> Can we now close this bug?

Would it be agreable if I make the following addition to the manual:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: bug22519.diff --]
[-- Type: text/x-patch, Size: 1585 bytes --]

diff --git a/doc/emacs/trouble.texi b/doc/emacs/trouble.texi
index bd347b0..e049bd1 100644
--- a/doc/emacs/trouble.texi
+++ b/doc/emacs/trouble.texi
@@ -148,6 +148,7 @@ Lossage
 * DEL Does Not Delete::   What to do if @key{DEL} doesn't delete.
 * Stuck Recursive::       '[...]' in mode line around the parentheses.
 * Screen Garbled::        Garbage on the screen.
+* Slow Display::          Slow screen refresh.
 * Text Garbled::          Garbage in the text.
 * Memory Full::           How to cope when you run out of memory.
 * Crashing::              What Emacs does when it crashes.
@@ -247,6 +248,25 @@ Screen Garbled
 entry, it is possible that there is a bug in the terminfo entry, or a
 bug in Emacs that appears for certain terminal types.
 
+@node Slow Display
+@subsection Slow screen refresh
+@cindex display, slow
+@cindex refresh, display slow
+@cindex font, missing
+
+  If the display is too slow in refreshing when you scroll to a new
+region, or when you edit the buffer, it may be due to this that some
+characters cannot be displayed in the default font, and Emacs is
+spending too much time in looking for a suitable font to display them.
+
+  You can suspect this if you have several characters that are
+displayed as small rectangles containing a hexadecimal code inside.
+
+  The solution is to install the appropriate fonts on your
+machine. For instance if you are editing a text with a lot of math
+symbols, then installing a font like @file{Symbola} should solve this
+problem.
+
 @node Text Garbled
 @subsection Garbage in the Text
 @cindex garbled text

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com>,
>   22519@debbugs.gnu.org
> Date: Thu, 17 Mar 2016 10:14:04 +0100
> 
> - it would suffice to raise some character specific flag when a
>   character is not found in any font in ordder to prevent subsequent
>   search for the same character after an edit, and also to put this
>   character in some queue so as to trigger the actual font search
>   _again_ for the same character only when Emacs gets the focus.

The fonts are searched when Emacs is about to redisplay the window.
Whether Emacs has the focus at that time is not really relevant, since
if there's a need to redisplay, Emacs must do that regardless.

I don't quite understand your proposal about "preventing subsequent
search for the same character after an edit": installation of
additional fonts is unrelated to anything don inside Emacs, so whether
there was or wasn't any editing doesn't seem relevant.

> - there could be some counter incremented every time a character cannot
>   be displayed in a known font and alternative font would be extensively
>   searched only when this counter is below some threshold, and this
>   counter would be periodically reset. This method has however the
>   drawback that the display would not be deterministic.

I think we already do something like that, but I'm far from being an
expert on this part in Emacs.

> - I am wondering whether it is to the application to make this search,
>   the system is supposed to know which fonts have been installed, isn't
>   it possible that Emacs just ask to MSW for each character that cannot
>   be displayed in the default font to supply the best alternative
>   font. Then, since the OS is supposed to know whether or not a new font
>   has been installed it may be more efficient in performing this search.

The OS knows about installed fonts, but I'm not sure there's a service
to announce that to applications.  The OS certainly doesn't know
whether the installed font supports some character; it provides APIs
for applications to find that out, which is what Emacs does.

> Would it be agreable if I make the following addition to the manual:

I think this should be rewritten to go to etc/PROBLEMS instead of into
the manual.

Thanks.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]



Le 17/03/2016 à 17:20, Eli Zaretskii a écrit :
>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com>,
>>   22519@debbugs.gnu.org
>> Date: Thu, 17 Mar 2016 10:14:04 +0100
>>

[...]

>
>> Would it be agreable if I make the following addition to the manual:
>
> I think this should be rewritten to go to etc/PROBLEMS instead of into
> the manual.

Ok, should I make this edit only for master branch, or for emacs-25
branch first & then merge it into master branch. In the latter case, it
would be helpful if you send me a pointer to instructions how to carry
out the merge with git.

FYI I have not tested that the bug is also there for emacs-25 branch, I
just can't imagine that it would not be there also.

VBR,
	Vincent.

PS: FYI, attached is patch to master branch.

>
> Thanks.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: bug22519.diff --]
[-- Type: text/x-patch, Size: 1113 bytes --]

diff --git a/etc/PROBLEMS b/etc/PROBLEMS
index f61921a..39a5849 100644
--- a/etc/PROBLEMS
+++ b/etc/PROBLEMS
@@ -720,6 +720,20 @@ the following variables: tex-font-script-display (how much to
 lower/raise); tex-suscript-height-ratio (how much smaller than
 normal); tex-suscript-height-minimum (minimum height).
 
+** Screen refresh is slow when there are special characters for which no suitable font is available
+
+If the display is too slow in refreshing when you scroll to a new
+region, or when you edit the buffer, it may be due to this that some
+characters cannot be displayed in the default font, and Emacs is
+spending too much time in looking for a suitable font to display them.
+
+You can suspect this if you have several characters that are displayed
+as small rectangles containing a hexadecimal code inside.
+
+The solution is to install the appropriate fonts on your machine. For
+instance if you are editing a text with a lot of math symbols, then
+installing a font like 'Symbola' should solve this problem.
+
 * Internationalization problems
 
 ** M-{ does not work on a Spanish PC keyboard.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>  22519@debbugs.gnu.org
> Date: Tue, 24 May 2016 12:16:24 +0200
> 
> >> Would it be agreable if I make the following addition to the manual:
> >
> > I think this should be rewritten to go to etc/PROBLEMS instead of into
> > the manual.
> 
> Ok, should I make this edit only for master branch, or for emacs-25
> branch first & then merge it into master branch.

The latter.  The problem could happen with the emacs-25 branch as
well, and adding this piece of information cannot possibly break
anything on the release branch.

> In the latter case, it would be helpful if you send me a pointer to
> instructions how to carry out the merge with git.

You don't need to merge yourself, someone else will.  But if you want
to do it yourself, you will find the information in CONTRIBUTE, in
admin/notes/git-workflow, and in comments inside admin/gitmerge.el.

> +If the display is too slow in refreshing when you scroll to a new
> +region, or when you edit the buffer, it may be due to this that some
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
"it might be due to the fact that some characters ..."

Thanks.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Glenn Morris]

PROBLEMS text was added in 0be6725, closing.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

[-- Attachment #1: Type: text/plain, Size: 157 bytes --]


I was runnning emacs built with Mingw W64 under a debug session. It gets
stucks while doing an incremental search. I cannot even make a screen
copy of it.


[-- Attachment #2: Sans titre.png --]
[-- Type: image/png, Size: 85380 bytes --]

[-- Attachment #3: Type: text/plain, Size: 4102 bytes --]


This report is sent from another Emacs instance than that running the
debugger, and that which is stuck.




In GNU Emacs 25.1.50.1 (i686-w64-mingw32)
 of 2016-01-29 built on AIGLEROYAL
Repository revision: 487bd7aedf9bbbf0b939b653912253fbeb7d16b9
Windowing system distributor 'Microsoft Corp.', version 10.0.10586
Configured using:
 'configure --prefix=c:/Nos_Programmes/GNU/Emacs --without-jpeg
 --without-tiff --without-gif --without-png 'CFLAGS= -Og -g3 -L
 C:/Programmes/installation/emacs-install/libXpm-3.5.8/src' 'CPPFLAGS=
 -DFOR_MSW=1 -I
 C:/Programmes/installation/emacs-install/libXpm-3.5.8/include -I
 C:/Programmes/installation/emacs-install/libXpm-3.5.8/src -L
 C:/Programmes/installation/emacs-install/libXpm-3.5.8/src''

Configured features:
XPM SOUND NOTIFY ACL ZLIB TOOLKIT_SCROLL_BARS

Important settings:
  value of $LANG: FRA
  locale-coding-system: cp1252

Major mode: Fundamental

Minor modes in effect:
  recentf-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
Loading c:/Users/Vincent/AppData/Roaming/.emacs.d/etc/my_loaddefs.el (source)...done
Loading c:/Users/Vincent/AppData/Roaming/.emacs.d/recentf...done
Cleaning up the recentf list...done (0 removed)
Source file ‘c:/Nos_Programmes/GNU/emacs-extension/lisp/template.el’ newer than byte-compiled file
Loading c:/Programmes/installation/bbdb-git/bbdb/lisp/bbdb-loaddefs.el (source)...done
Loading c:/Users/Vincent/AppData/Roaming/.emacs.d/etc/my_autoloads.el (source)...done
For information about GNU Emacs and the GNU system, type C-h C-a.
GNU Emacs 25.1.50.1 (i686-w64-mingw32) of 2016-01-29
Making completion list...
Loading dired-x...done

Load-path shadows:
c:/Programmes/installation/cedet-install/cedet-git/lisp/speedbar/loaddefs hides c:/Nos_Programmes/GNU/Emacs/share/emacs/25.1.50/lisp/loaddefs
c:/Programmes/installation/cedet-install/cedet-git/lisp/speedbar/loaddefs hides c:/Programmes/installation/cedet-install/cedet-git/lisp/cedet/loaddefs

Features:
(shadow sort bbdb-message mail-extr emacsbug message dired-x dired
dired-loaddefs format-spec rfc822 mml mml-sec epg epg-config gnus-util
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util help-fns
help-mode mail-prsvr mail-utils edmacro kmacro skeleton calc-misc
calc-arith calc-ext calc calc-loaddefs calc-macs tex-mik preview-latex
tex-site auto-loads bbdb bbdb-site timezone bbdb-loaddefs template
w32utils cl-seq cl-macs cl gv recentf tree-widget wid-edit cl-loaddefs
pcase cl-lib easymenu load-path-to-cedet-svn time-date mule-util tooltip
eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
dos-w32 ls-lisp disp-table w32-win w32-vars term/common-win tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame
cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese charscript
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote w32notify w32 multi-tty
make-network-process emacs)

Memory information:
((conses 8 137476 9420)
 (symbols 32 24807 0)
 (miscs 32 79 193)
 (strings 16 27702 4554)
 (string-bytes 1 819707)
 (vectors 8 15543)
 (vector-slots 4 460529 5876)
 (floats 8 169 27)
 (intervals 28 367 17)
 (buffers 520 13))

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Date: Mon, 01 Feb 2016 10:04:56 +0100
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com>
> 
> I was runnning emacs built with Mingw W64 under a debug session. It gets
> stucks while doing an incremental search. I cannot even make a screen
> copy of it.

Not sure what can be done with this report.  You give no recipe, and
the screenshot AFAIU is of the Emacs that run the debug session, not
of the one that got stuck, is that right?  So there's no evidence here
about what happened to the one that got stuck.

When I debug Emacs on Windows, I sometimes have the other Emacs
session stop responding to keyboard input, and the only way to fix
that is to kill the debugging session.  Maybe this is what you bumped
into here.  I don't know what causes this, probably some semaphore or
other similar system-wide resource that Emacs being debugged holds and
prevents the other session from working.

Unless you can provide more information about this, I'm afraid I will
have to close it as not reproducible.

Thanks.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]



Le 01/02/2016 20:06, Eli Zaretskii a écrit :
>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>> Date: Mon, 01 Feb 2016 10:04:56 +0100
>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com>
>>
>> I was runnning emacs built with Mingw W64 under a debug session. It gets
>> stucks while doing an incremental search. I cannot even make a screen
>> copy of it.
>
> Not sure what can be done with this report.  You give no recipe, and
> the screenshot AFAIU is of the Emacs that run the debug session, not
> of the one that got stuck, is that right?

Yes this is right. I could get a picture of the stuck Emacs (attached)


[-- Attachment #2: Sans titre.png --]
[-- Type: image/png, Size: 106875 bytes --]

[-- Attachment #3: Type: text/plain, Size: 884 bytes --]


it does not get the focus of MSW, I could have the picture only by
iconizing all the other apps, so that the stuck Emacs gets visible. As
you see I was running an `M-x diff-revision' on the stuck Emacs when it
happened.

> So there's no evidence here about what happened to the one that got
> stuck.
>
> When I debug Emacs on Windows, I sometimes have the other Emacs
> session stop responding to keyboard input, and the only way to fix
> that is to kill the debugging session.  Maybe this is what you bumped
> into here.  I don't know what causes this, probably some semaphore or
> other similar system-wide resource that Emacs being debugged holds and
> prevents the other session from working.
>
> Unless you can provide more information about this, I'm afraid I will
> have to close it as not reproducible.
>

Please let me know, and then I can close it.

> Thanks.

   Vincent.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Eli Zaretskii]

> From: vincent.belaiche@gmail.com (Vincent Belaïche)
> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>  22519@debbugs.gnu.org
> Date: Tue, 02 Feb 2016 08:18:54 +0100
> 
> Please let me know, and then I can close it.

You can close it, thanks.

bug#22519: 25.1.50; Emacs gets stuck while doing incremental search forward

[Vincent Belaïche]

Hello Eli,

Sorry to come back on this. In fact there is a problem easy to
reproduce, when I edit this file:

http://svn.gna.org/viewcvs/*checkout*/latexrefman/trunk/latex2e-fr.texi?revision=505&content-type=text%2Fplain

then Emacs is quite quite slow when I am on the ``Math symbols'' node. I
can a couple of words and display is refreshed some 1 or 2 second after.

It has also happened to me that Emacs gets stuck when on this node, if
MSWindows snoozes and I wake it up.

VBR,
	Vincent.

Le 02/02/2016 17:16, Eli Zaretskii a écrit :
>> From: vincent.belaiche@gmail.com (Vincent Belaïche)
>> Cc: Vincent Belaïche <vincent.belaiche@gmail.com> ,
>>  22519@debbugs.gnu.org
>> Date: Tue, 02 Feb 2016 08:18:54 +0100
>>
>> Please let me know, and then I can close it.
>
> You can close it, thanks.