bug#15555: 24.3; Bidirectional display very slow with long lines

bug#15555: 24.3; Bidirectional display very slow with long lines

[Jerome L Quinn]

If I have a buffer with a very long line (14000 chars) with a mix of
ASCII and Arabic text, emacs gets painfully slow when point is further
along the line.  It looks like there's an N^2 algorithm dependent on the
column of point.  We encounter this kind of data in our work looking at
generated files.  The only solution at the moment is to disable
bidirectional reordering.

There are also problems with the display.  If you search for the following:

74,346,347

my display looks like:

... ,[["74,346,347],".",".","","."], ...

Whereas it should display:

... ,[".","",".",".",[74,346,347]]]], ...

Note that the first " is misplaced, even if you accept that the rest
should be reversed.

The following is my example text:

[[null,["raw"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[-1,0,322]]]],["raw",["rawtext"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[0,0,322]]]],["rawtext",["text"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[0,0,322]]]],["text",["rawtok","toktype"],[["كل","0",[0,0,2]],["هذا","0",[0,3,6]],["الكره","0",[0,7,12]],["،","2048",[0,12,13]],["فائض","0",[0,14,18]],["الكراهية","0",[0,19,27]],["حالة","0",[0,28,32]],["غبية","0",[0,33,37]],["تلفنا","0",[0,38,43]],["من","0",[0,44,46]],["كل","0",[0,47,49]],["جانب","0",[0,50,54]],["،","2048",[0,54,55]],["عندما","0",[0,56,61]],["يتمنطق","0",[0,62,68]],["البعض","0",[0,69,74]],["بالكراهية","0",[0,75,84]],["،","2048",[0,84,85]],["عندما","0",[0,86,91]],["يعتمل","0",[0,92,97]],["الثأر","0",[0,98,103]],["فى","0",[0,104,106]],["النفوس","0",[0,107,113]],["،","2048",[0,113,114]],["تصير","0",[0,115,119]],["قنابل","0",[0,120,125]],["بشرية","0",[0,126,131]],["تمشى","0",[0,132,136]],["على","0",[0,137,140]],["قدمين","0",[0,141,146]],["،","2048",[0,146,147]],["إن","0",[0,148,150]],["رأت","0",[0,151,154]],["نارا","0",[0,155,159]],["تصب","0",[0,160,163]],["من","0",[0,164,166]],["نفوسها","0",[0,167,173]],["بنزينا","0",[0,174,180]],["،","2048",[0,180,181]],["الكراهية","0",[0,182,190]],["حالة","0",[0,191,195]],["مرضية","0",[0,196,201]],["تستبد","0",[0,202,207]],["بالوطن","0",[0,208,214]],["،","2048",[0,214,215]],["تعبير","0",[0,216,221]],["كاره","0",[0,222,226]],["عن","0",[0,227,229]],["غضب","0",[0,230,233]],["مستطير","0",[0,234,240]],["..","0",[0,240,242]],["لا","0",[0,243,245]],["يفرق","0",[0,246,250]],["كثيرا","0",[0,251,256]],["عن","0",[0,257,259]],["الشرر","0",[0,260,265]],["المستطير","0",[0,266,274]],["،","2048",[0,274,275]],["نيران","0",[0,276,281]],["الكراهية","0",[0,282,290]],["تعتمل","0",[0,291,296]],["فى","0",[0,297,299]],["الصدور","0",[0,300,306]],["،","2048",[0,306,307]],["حمم","0",[0,308,311]],["لو","0",[0,312,314]],["تعلمون","0",[0,315,321]],[".","6144",[0,321,322]]]],["rawtok",["tok"],[["كل",[0,0,2]],["هذا",[1,3,6]],["الكره",[2,7,12]],[",",[3,13,14]],["فائض",[4,15,19]],["الكراهية",[5,20,28]],["حالة",[6,29,33]],["غبية",[7,34,38]],["تلفنا",[8,39,44]],["من",[9,45,47]],["كل",[10,48,50]],["جانب",[11,51,55]],[",",[12,56,57]],["عندما",[13,58,63]],["يتمنطق",[14,64,70]],["البعض",[15,71,76]],["بالكراهية",[16,77,86]],[",",[17,87,88]],["عندما",[18,89,94]],["يعتمل",[19,95,100]],["الثأر",[20,101,106]],["فى",[21,107,109]],["النفوس",[22,110,116]],[",",[23,117,118]],["تصير",[24,119,123]],["قنابل",[25,124,129]],["بشرية",[26,130,135]],["تمشى",[27,136,140]],["على",[28,141,144]],["قدمين",[29,145,150]],[",",[30,151,152]],["إن",[31,153,155]],["رأت",[32,156,159]],["نارا",[33,160,164]],["تصب",[34,165,168]],["من",[35,169,171]],["نفوسها",[36,172,178]],["بنزينا",[37,179,185]],[",",[38,186,187]],["الكراهية",[39,188,196]],["حالة",[40,197,201]],["مرضية",[41,202,207]],["تستبد",[42,208,213]],["بالوطن",[43,214,220]],[",",[44,221,222]],["تعبير",[45,223,228]],["كاره",[46,229,233]],["عن",[47,234,236]],["غضب",[48,237,240]],["مستطير",[49,241,247]],["..",[50,248,250]],["لا",[51,251,253]],["يفرق",[52,254,258]],["كثيرا",[53,259,264]],["عن",[54,265,267]],["الشرر",[55,268,273]],["المستطير",[56,274,282]],[",",[57,283,284]],["نيران",[58,285,290]],["الكراهية",[59,291,299]],["تعتمل",[60,300,305]],["فى",[61,306,308]],["الصدور",[62,309,315]],[",",[63,316,317]],["حمم",[64,318,321]],["لو",[65,322,324]],["تعلمون",[66,325,331]],[".",[67,332,333]]]],["tok",["rawatb"],[["كل",[0,0,2]],["هذا",[1,3,6]],["الكره",[2,7,12]],[",",[3,13,14]],["فائض",[4,15,19]],["الكراهية",[5,20,28]],["حالة",[6,29,33]],["غبية",[7,34,38]],["تلف",[8,39,42]],["+نا",[8,42,44]],["من",[9,45,47]],["كل",[10,48,50]],["جانب",[11,51,55]],[",",[12,56,57]],["عند",[13,58,61]],["+ما",[13,61,63]],["يتمنطق",[14,64,70]],["البعض",[15,71,76]],["ب#",[16,77,78]],["الكراهية",[16,78,86]],[",",[17,87,88]],["عند",[18,89,92]],["+ما",[18,92,94]],["يعتمل",[19,95,100]],["الثأر",[20,101,106]],["فى",[21,107,109]],["النفوس",[22,110,116]],[",",[23,117,118]],["تصير",[24,119,123]],["قنابل",[25,124,129]],["بشرية",[26,130,135]],["تمشى",[27,136,140]],["على",[28,141,144]],["قدمين",[29,145,150]],[",",[30,151,152]],["إن",[31,153,155]],["رأت",[32,156,159]],["نارا",[33,160,164]],["تصب",[34,165,168]],["من",[35,169,171]],["نفوس",[36,172,176]],["+ها",[36,176,178]],["بنزينا",[37,179,185]],[",",[38,186,187]],["الكراهية",[39,188,196]],["حالة",[40,197,201]],["مرضية",[41,202,207]],["تستبد",[42,208,213]],["ب#",[43,214,215]],["الوطن",[43,215,220]],[",",[44,221,222]],["تعبير",[45,223,228]],["كار",[46,229,232]],["+ه",[46,232,233]],["عن",[47,234,236]],["غضب",[48,237,240]],["مستطير",[49,241,247]],["..",[50,248,250]],["لا",[51,251,253]],["يفرق",[52,254,258]],["كثيرا",[53,259,264]],["عن",[54,265,267]],["الشرر",[55,268,273]],["المستطير",[56,274,282]],[",",[57,283,284]],["نيران",[58,285,290]],["الكراهية",[59,291,299]],["تعتمل",[60,300,305]],["فى",[61,306,308]],["الصدور",[62,309,315]],[",",[63,316,317]],["حمم",[64,318,321]],["لو",[65,322,324]],["تعلمون",[66,325,331]],[".",[67,332,333]]]],["rawatb",["atb"],[["كل",[0,0,2]],["هذا",[1,3,6]],["الكره",[2,7,12]],[",",[3,13,14]],["فائض",[4,15,19]],["الكراهية",[5,20,28]],["حالة",[6,29,33]],["غبية",[7,34,38]],["تلف",[8,39,42]],["+نا",[9,43,46]],["من",[10,47,49]],["كل",[11,50,52]],["جانب",[12,53,57]],[",",[13,58,59]],["عند",[14,60,63]],["+ما",[15,64,67]],["يتمنطق",[16,68,74]],["البعض",[17,75,80]],["ب#",[18,81,83]],["الكراهية",[19,84,92]],[",",[20,93,94]],["عند",[21,95,98]],["+ما",[22,99,102]],["يعتمل",[23,103,108]],["الثأر",[24,109,114]],["فى",[25,115,117]],["النفوس",[26,118,124]],[",",[27,125,126]],["تصير",[28,127,131]],["قنابل",[29,132,137]],["بشرية",[30,138,143]],["تمشى",[31,144,148]],["على",[32,149,152]],["قدمين",[33,153,158]],[",",[34,159,160]],["إن",[35,161,163]],["رأت",[36,164,167]],["نارا",[37,168,172]],["تصب",[38,173,176]],["من",[39,177,179]],["نفوس",[40,180,184]],["+ها",[41,185,188]],["بنزينا",[42,189,195]],[",",[43,196,197]],["الكراهية",[44,198,206]],["حالة",[45,207,211]],["مرضية",[46,212,217]],["تستبد",[47,218,223]],["ب#",[48,224,226]],["الوطن",[49,227,232]],[",",[50,233,234]],["تعبير",[51,235,240]],["كار",[52,241,244]],["+ه",[53,245,247]],["عن",[54,248,250]],["غضب",[55,251,254]],["مستطير",[56,255,261]],["..",[57,262,264]],["لا",[58,265,267]],["يفرق",[59,268,272]],["كثيرا",[60,273,278]],["عن",[61,279,281]],["الشرر",[62,282,287]],["المستطير",[63,288,296]],[",",[64,297,298]],["نيران",[65,299,304]],["الكراهية",[66,305,313]],["تعتمل",[67,314,319]],["فى",[68,320,322]],["الصدور",[69,323,329]],[",",[70,330,331]],["حمم",[71,332,335]],["لو",[72,336,338]],["تعلمون",[73,339,345]],[".",[74,346,347]]]],["atb",["src","ne","morph","decorated"],[["كل","","كل","كل",[0,0,2]],["هذا","","هذا","هذا",[1,3,6]],["الكره","","ال# كره","الكره",[2,7,12]],[",","",",",",",[3,13,14]],["فائض","","فائض","فائض",[4,15,19]],["الكراهية","","ال# كراهي +ة","الكراهية",[5,20,28]],["حالة","","حال +ة","حالة",[6,29,33]],["غبية","","غبي +ة","غبية",[7,34,38]],["تلف","","ت# لف","تلف",[8,39,42]],["+نا","","+نا","+نا",[9,43,46]],["من","","من","من",[10,47,49]],["كل","","كل","كل",[11,50,52]],["جانب","","جانب","جانب",[12,53,57]],[",","",",",",",[13,58,59]],["عند","","عند","عند",[14,60,63]],["+ما","","+ما","+ما",[15,64,67]],["يتمنطق","","ي# تمنطق","يتمنطق",[16,68,74]],["البعض","","ال# بعض","البعض",[17,75,80]],["ب#","","ب#","ب#",[18,81,83]],["الكراهية","","ال# كراهي +ة","الكراهية",[19,84,92]],[",","",",",",",[20,93,94]],["عند","","عند","عند",[21,95,98]],["+ما","","+ما","+ما",[22,99,102]],["يعتمل","","ي# عتمل","يعتمل",[23,103,108]],["الثار","","ال# ثار","الثار",[24,109,114]],["في","","في","في",[25,115,117]],["النفوس","","ال# نفوس","النفوس",[26,118,124]],[",","",",",",",[27,125,126]],["تصير","","ت# صير","تصير",[28,127,131]],["قنابل","","قنابل","قنابل",[29,132,137]],["بشرية","","بشري +ة","بشرية",[30,138,143]],["تمشي","","تمشي","تمشي",[31,144,148]],["علي","","علي","علي",[32,149,152]],["قدمين","","قدم +ين","قدمين",[33,153,158]],[",","",",",",",[34,159,160]],["ان","","ان","ان",[35,161,163]],["رات","","رات","رات",[36,164,167]],["نارا","","نارا","نارا",[37,168,172]],["تصب","","ت# صب","تصب",[38,173,176]],["من","","من","من",[39,177,179]],["نفوس","","نفوس","نفوس",[40,180,184]],["+ها","","+ها","+ها",[41,185,188]],["بنزينا","","بنزينا","بنزينا",[42,189,195]],[",","",",",",",[43,196,197]],["الكراهية","","ال# كراهي +ة","الكراهية",[44,198,206]],["حالة","","حال +ة","حالة",[45,207,211]],["مرضية","","مرضي +ة","مرضية",[46,212,217]],["تستبد","","ت# ستبد","تستبد",[47,218,223]],["ب#","","ب#","ب#",[48,224,226]],["الوطن","","ال# وطن","الوطن",[49,227,232]],[",","",",",",",[50,233,234]],["تعبير","","تعبير","تعبير",[51,235,240]],["كار","","كار","كار",[52,241,244]],["+ه","","+ه","+ه",[53,245,247]],["عن","","عن","عن",[54,248,250]],["غضب","","غضب","غضب",[55,251,254]],["مستطير","","مستطير","مستطير",[56,255,261]],["..","","..","..",[57,262,264]],["لا","","لا","لا",[58,265,267]],["يفرق","","ي# فرق","يفرق",[59,268,272]],["كثيرا","","كثير +ا","كثيرا",[60,273,278]],["عن","","عن","عن",[61,279,281]],["الشرر","","ال# شرر","الشرر",[62,282,287]],["المستطير","","ال# مستطير","المستطير",[63,288,296]],[",","",",",",",[64,297,298]],["نيران","","نيران","نيران",[65,299,304]],["الكراهية","","ال# كراهي +ة","الكراهية",[66,305,313]],["تعتمل","","ت# عتمل","تعتمل",[67,314,319]],["في","","في","في",[68,320,322]],["الصدور","","ال# صدور","الصدور",[69,323,329]],[",","",",",",",[70,330,331]],["حمم","","حمم","حمم",[71,332,335]],["لو","","لو","لو",[72,336,338]],["تعلمون","","تعلم +ون","تعلمون",[73,339,345]],[".","",".",".",[74,346,347]]]],["tok",["toknorm"],[["كل",[0,-1,-1]],["هذا",[1,-1,-1]],["الكره",[2,-1,-1]],[",",[3,-1,-1]],["فائض",[4,-1,-1]],["الكراهية",[5,-1,-1]],["حالة",[6,-1,-1]],["غبية",[7,-1,-1]],["تلفنا",[8,-1,-1]],["من",[9,-1,-1]],["كل",[10,-1,-1]],["جانب",[11,-1,-1]],[",",[12,-1,-1]],["عندما",[13,-1,-1]],["يتمنطق",[14,-1,-1]],["البعض",[15,-1,-1]],["بالكراهية",[16,-1,-1]],[",",[17,-1,-1]],["عندما",[18,-1,-1]],["يعتمل",[19,-1,-1]],["الثار",[20,-1,-1]],["في",[21,-1,-1]],["النفوس",[22,-1,-1]],[",",[23,-1,-1]],["تصير",[24,-1,-1]],["قنابل",[25,-1,-1]],["بشرية",[26,-1,-1]],["تمشي",[27,-1,-1]],["علي",[28,-1,-1]],["قدمين",[29,-1,-1]],[",",[30,-1,-1]],["ان",[31,-1,-1]],["رات",[32,-1,-1]],["نارا",[33,-1,-1]],["تصب",[34,-1,-1]],["من",[35,-1,-1]],["نفوسها",[36,-1,-1]],["بنزينا",[37,-1,-1]],[",",[38,-1,-1]],["الكراهية",[39,-1,-1]],["حالة",[40,-1,-1]],["مرضية",[41,-1,-1]],["تستبد",[42,-1,-1]],["بالوطن",[43,-1,-1]],[",",[44,-1,-1]],["تعبير",[45,-1,-1]],["كاره",[46,-1,-1]],["عن",[47,-1,-1]],["غضب",[48,-1,-1]],["مستطير",[49,-1,-1]],["..",[50,-1,-1]],["لا",[51,-1,-1]],["يفرق",[52,-1,-1]],["كثيرا",[53,-1,-1]],["عن",[54,-1,-1]],["الشرر",[55,-1,-1]],["المستطير",[56,-1,-1]],[",",[57,-1,-1]],["نيران",[58,-1,-1]],["الكراهية",[59,-1,-1]],["تعتمل",[60,-1,-1]],["في",[61,-1,-1]],["الصدور",[62,-1,-1]],[",",[63,-1,-1]],["حمم",[64,-1,-1]],["لو",[65,-1,-1]],["تعلمون",[66,-1,-1]],[".",[67,-1,-1]]]],["morph",["fin"],[["كل",[0,-1,-1]],["هذا",[1,-1,-1]],["ال# كره",[2,-1,-1]],[",",[3,-1,-1]],["فائض",[4,-1,-1]],["ال# كراهي +ة",[5,-1,-1]],["حال +ة",[6,-1,-1]],["غبي +ة",[7,-1,-1]],["ت# لف",[8,-1,-1]],["+نا",[9,-1,-1]],["من",[10,-1,-1]],["كل",[11,-1,-1]],["جانب",[12,-1,-1]],[",",[13,-1,-1]],["عند",[14,-1,-1]],["+ما",[15,-1,-1]],["ي# تمنطق",[16,-1,-1]],["ال# بعض",[17,-1,-1]],["ب#",[18,-1,-1]],["ال# كراهي +ة",[19,-1,-1]],[",",[20,-1,-1]],["عند",[21,-1,-1]],["+ما",[22,-1,-1]],["ي# عتمل",[23,-1,-1]],["ال# ثار",[24,-1,-1]],["في",[25,-1,-1]],["ال# نفوس",[26,-1,-1]],[",",[27,-1,-1]],["ت# صير",[28,-1,-1]],["قنابل",[29,-1,-1]],["بشري +ة",[30,-1,-1]],["تمشي",[31,-1,-1]],["علي",[32,-1,-1]],["قدم +ين",[33,-1,-1]],[",",[34,-1,-1]],["ان",[35,-1,-1]],["رات",[36,-1,-1]],["نارا",[37,-1,-1]],["ت# صب",[38,-1,-1]],["من",[39,-1,-1]],["نفوس",[40,-1,-1]],["+ها",[41,-1,-1]],["بنزينا",[42,-1,-1]],[",",[43,-1,-1]],["ال# كراهي +ة",[44,-1,-1]],["حال +ة",[45,-1,-1]],["مرضي +ة",[46,-1,-1]],["ت# ستبد",[47,-1,-1]],["ب#",[48,-1,-1]],["ال# وطن",[49,-1,-1]],[",",[50,-1,-1]],["تعبير",[51,-1,-1]],["كار",[52,-1,-1]],["+ه",[53,-1,-1]],["عن",[54,-1,-1]],["غضب",[55,-1,-1]],["مستطير",[56,-1,-1]],["..",[57,-1,-1]],["لا",[58,-1,-1]],["ي# فرق",[59,-1,-1]],["كثير +ا",[60,-1,-1]],["عن",[61,-1,-1]],["ال# شرر",[62,-1,-1]],["ال# مستطير",[63,-1,-1]],[",",[64,-1,-1]],["نيران",[65,-1,-1]],["ال# كراهي +ة",[66,-1,-1]],["ت# عتمل",[67,-1,-1]],["في",[68,-1,-1]],["ال# صدور",[69,-1,-1]],[",",[70,-1,-1]],["حمم",[71,-1,-1]],["لو",[72,-1,-1]],["تعلم +ون",[73,-1,-1]],[".",[74,-1,-1]]]],["raw",["align"],[["0",[0,0,2]],["0",[0,3,6]],["0",[0,7,12]],["2048",[0,12,13]],["0",[0,14,18]],["0",[0,19,27]],["0",[0,28,32]],["0",[0,33,37]],["0",[0,38,41]],["512",[0,41,43]],["0",[0,44,46]],["0",[0,47,49]],["0",[0,50,54]],["2048",[0,54,55]],["0",[0,56,59]],["512",[0,59,61]],["0",[0,62,68]],["0",[0,69,74]],["256",[0,75,76]],["0",[0,76,84]],["2048",[0,84,85]],["0",[0,86,89]],["512",[0,89,91]],["0",[0,92,97]],["0",[0,98,103]],["0",[0,104,106]],["0",[0,107,113]],["2048",[0,113,114]],["0",[0,115,119]],["0",[0,120,125]],["0",[0,126,131]],["0",[0,132,136]],["0",[0,137,140]],["0",[0,141,146]],["2048",[0,146,147]],["0",[0,148,150]],["0",[0,151,154]],["0",[0,155,159]],["0",[0,160,163]],["0",[0,164,166]],["0",[0,167,171]],["512",[0,171,173]],["0",[0,174,180]],["2048",[0,180,181]],["0",[0,182,190]],["0",[0,191,195]],["0",[0,196,201]],["0",[0,202,207]],["256",[0,208,209]],["0",[0,209,214]],["2048",[0,214,215]],["0",[0,216,221]],["0",[0,222,225]],["512",[0,225,226]],["0",[0,227,229]],["0",[0,230,233]],["0",[0,234,240]],["0",[0,240,242]],["0",[0,243,245]],["0",[0,246,250]],["0",[0,251,256]],["0",[0,257,259]],["0",[0,260,265]],["0",[0,266,274]],["2048",[0,274,275]],["0",[0,276,281]],["0",[0,282,290]],["0",[0,291,296]],["0",[0,297,299]],["0",[0,300,306]],["2048",[0,306,307]],["0",[0,308,311]],["0",[0,312,314]],["0",[0,315,321]],["6144",[0,321,322]]]]]


Thanks
Jerry



In GNU Emacs 24.3.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.18.9)
 of 2013-08-13 on smaug.watson.ibm.com
Windowing system distributor `CentOS', version 11.0.11300000
System Description:	CentOS release 6.4 (Final)

Configured using:
 `configure '--prefix=/ikm/77/ws' '--with-x-toolkit=gtk2''

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=none
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
  diff-auto-refine-mode: t
  iswitchb-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-g C-g C-h v b i d i <tab> <tab> C-g C-h i q C-h v 
b i d i - d i <tab> <tab> i <tab> <return> M-x s e 
t SPC v a <tab> <return> b i d i - d i s p l a y - 
r e o r <tab> d e r i n g <return> C-g C-g C-g C-g 
M-: M-( <up> C-e <C-left> <C-left> <C-right> i n g 
<return> C-x 1 <C-right> <C-right> <C-right> <C-right> 
<C-right> <C-right> <C-right> <C-right> <C-right> <C-right> 
<C-right> <C-right> <C-right> <C-right> <C-right> <C-right> 
<C-right> <C-right> <C-right> <C-right> <C-right> <C-right> 
<C-right> <C-right> <C-right> <C-right> C-e <C-left> 
<C-left> <C-left> <C-left> <C-left> <C-left> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <down> <down> <up> <up> <up> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <up> <up> <up> <up> <up> C-e C-x = M-x 
r e p o r t SPC m e <tab> <backspace> <backspace> e 
m <tab> <return>

Recent messages:
Mark set [2 times]
Quit [5 times]
Making completion list...
Quit
Making completion list...
Type C-x 1 to delete the help window.
Quit [4 times]
nil
byte-code: Beginning of buffer [10 times]
Char: C-j (10, #o12, #xa) point=14572 of 6051900 (0%) column=14571

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message cl-macs gv format-spec rfc822
mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils
shell pcomplete grep compile mule-util cal-move cal-menu calendar
cal-loaddefs js json imenu thingatpt ediff-merg ediff-diff ediff-wind
ediff-help ediff-util ediff-mult ediff-init ediff ffap url-parse
auth-source eieio byte-opt bytecomp byte-compile cconv gnus-util mm-util
mail-prsvr password-cache url-vars tabify man log-edit vc-cvs vc-rcs
vc-dir ewoc diff-mode add-log log-view easy-mmode pcvs-util vc rect
dabbrev misearch multi-isearch python rx comint ring ansi-color
help-mode iso-transl info arc-mode archive-mode dired-aux make-mode
cc-langs cl cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs sh-script smie executable nroff-mode
nxml-uchnm rng-xsd xsd-regexp rng-cmpct rng-nxml rng-valid rng-loc
rng-uri rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns
easymenu nxml-mode nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc
xmltok sgml-mode perl-mode vc-dispatcher vc-svn conf-mode dired desktop
jka-compr saveplace uniquify advice help-fns cl-lib advice-preload
iswitchb time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel
x-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list
newcomment lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty emacs)

bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> From: Jerome L Quinn <jlquinn@us.ibm.com>
> Date: Mon, 7 Oct 2013 16:05:42 -0400
> 
> If I have a buffer with a very long line (14000 chars) with a mix of
> ASCII and Arabic text, emacs gets painfully slow when point is further
> along the line.

Emacs is in general painfully slow with such long lines, even if there
are only ASCII characters there.  Presence of Arabic characters makes
it slower, but it is unbearably slow even before that.  This is a
known deficiency of the Emacs display engine.  This is bug #13675.

> It looks like there's an N^2 algorithm dependent on the column of
> point.

No, there are no N² algorithms.  However, for many display operations,
Emacs needs to go to the beginning of the previous _physical_ line,
and then go forward to the place it needs to redisplay.  With very
long lines, this takes a lot of time.

If your data files don't include empty lines, there's perhaps another
source of slowdown, which _is_ peculiar to bidirectional display:
Emacs needs to find the beginning of the current paragraph to
determine its "base direction".  To see if this is your problem, try
setting bidi-paragraph-direction to either left-to-right or
right-to-left, depending on whether most of the text should be read
left to right or right to left.

> We encounter this kind of data in our work looking at generated
> files.  The only solution at the moment is to disable bidirectional
> reordering.

With 14 thousand characters per line, Emacs is slow even without
reordering.  Again, this is an existing bug.  Disabling reordering is
probably not a very good thing when you work with Arabic text.

> There are also problems with the display.  If you search for the following:
> 
> 74,346,347
> 
> my display looks like:
> 
> ... ,[["74,346,347],".",".","","."], ...
> 
> Whereas it should display:
> 
> ... ,[".","",".",".",[74,346,347]]]], ...
> 
> Note that the first " is misplaced, even if you accept that the rest
> should be reversed.

This is what the Unicode Bidirectional Algorithm mandates, since " is
a "weak" character, i.e. it gets its directionality from the
surrounding text, and because your paragraph has the left-to-right
base direction (because it begins with strong L2R character).  Perhaps
you should set bidi-paragraph-direction to right-to-left, as your text
is predominantly Arabic.

So I see no bug here wrt the display order.  As for slow redisplay
with very long lines, that is an existing bug report.

bug#15555: 24.3; Bidirectional display very slow with long lines

[Jerome L Quinn]

[-- Attachment #1: Type: text/plain, Size: 2829 bytes --]




Eli Zaretskii <eliz@gnu.org> wrote on 10/08/2013 02:42:53 AM:

> From: Eli Zaretskii <eliz@gnu.org>
> To: Jerome L Quinn/Watson/IBM@IBMUS
> Cc: 15555@debbugs.gnu.org
> Date: 10/08/2013 02:43 AM
> Subject: Re: bug#15555: 24.3; Bidirectional display very slow with long
lines
>
> > From: Jerome L Quinn <jlquinn@us.ibm.com>
> > Date: Mon, 7 Oct 2013 16:05:42 -0400
> >
> > If I have a buffer with a very long line (14000 chars) with a mix of
> > ASCII and Arabic text, emacs gets painfully slow when point is further
> > along the line.
>
> Emacs is in general painfully slow with such long lines, even if there
> are only ASCII characters there.  Presence of Arabic characters makes
> it slower, but it is unbearably slow even before that.  This is a
> known deficiency of the Emacs display engine.  This is bug #13675.

Hi Eli,

I'm not sure it is the same bug.  When I disable the bidi reordering
variable,
navigation speed becomes reasonable, even with the very long line length.
I'm on a very fast machine, so #13675 may not be impacting me that badly.

When bidi reordering is enabled, it is multiple seconds to move the cursor
up
and down, which is unusable.


> > It looks like there's an N^2 algorithm dependent on the column of
> > point.
>
> No, there are no N² algorithms.  However, for many display operations,
> Emacs needs to go to the beginning of the previous _physical_ line,
> and then go forward to the place it needs to redisplay.  With very
> long lines, this takes a lot of time.
>
> If your data files don't include empty lines, there's perhaps another
> source of slowdown, which _is_ peculiar to bidirectional display:
> Emacs needs to find the beginning of the current paragraph to
> determine its "base direction".  To see if this is your problem, try
> setting bidi-paragraph-direction to either left-to-right or
> right-to-left, depending on whether most of the text should be read
> left to right or right to left.

Setting bidi-paragraph-direction to right-to-left improves the situation
some
but I'd still call it unusable in this situation.  Disabling reordering
makes
emacs as responsive as I'd expect, comparable to emacs23.

OK, I played with it a bit more.  I think the issue is exacerbated by
splitting
the frame into 2 side-by-side windows.

Try this:

Expand emacs to fill the screen.  For me this is 194x65.
Create a new buffer with just the text I included in the bug report.
C-x 3
Put something else in the left window and go the right window.
Page down several times.

The last page down I find takes 5-8 seconds repeatedly.

I don't see as bad behavior when the text is in the left buffer or if I
have only
1 window.

And disabling bidi reordering completely eliminates the bad behavior.


Thanks
Jerry

[-- Attachment #2: Type: text/html, Size: 4266 bytes --]

bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Cc: 15555@debbugs.gnu.org
> From: Jerome L Quinn <jlquinn@us.ibm.com>
> Date: Tue, 8 Oct 2013 11:39:58 -0400
> 
> I'm not sure it is the same bug.  When I disable the bidi reordering
> variable,
> navigation speed becomes reasonable, even with the very long line length.
> I'm on a very fast machine, so #13675 may not be impacting me that badly.
> 
> When bidi reordering is enabled, it is multiple seconds to move the
> cursor up and down, which is unusable.

You are on a very fast machine, so you don't see the slow redisplay
with such lines.  But the basic problem remains: Emacs does not cope
well with long lines.  It's just that in your case the border of
"unbearable" is farther.

> Setting bidi-paragraph-direction to right-to-left improves the
> situation some but I'd still call it unusable in this situation.
> Disabling reordering makes emacs as responsive as I'd expect,
> comparable to emacs23.

Emacs 24 cannot possibly work as fast as Emacs 23, since reordering of
bidirectional text does need a lot of additional processing.  There's
nothing that can be done about that, without a complete rewrite of the
Emacs display engine (or purchasing a faster machine).

> I don't see as bad behavior when the text is in the left buffer or
> if I have only 1 window.

Then don't do that.

Again, these all are signs of slow redisplay with long lines.  They
just become a bot faster or slower in specific situations and screen
arrangements.

> And disabling bidi reordering completely eliminates the bad behavior.

If you can afford that, go for it.

bug#15555: 24.3; Bidirectional display very slow with long lines

[Stefan Monnier]

>> And disabling bidi reordering completely eliminates the bad behavior.
> If you can afford that, go for it.

IIRC this is the first report where setting bidi-display-reordering to
nil is really the best recommendation we can offer (and where it
apparently indeed helps significantly).

I consider bidi-display-reordering as a debugging tool rather than
a user config, so I'm not very happy about this situation.


        Stefan

bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Jerome L Quinn <jlquinn@us.ibm.com>,  15555@debbugs.gnu.org
> Date: Wed, 09 Oct 2013 08:26:58 -0400
> 
> >> And disabling bidi reordering completely eliminates the bad behavior.
> > If you can afford that, go for it.
> 
> IIRC this is the first report where setting bidi-display-reordering to
> nil is really the best recommendation we can offer (and where it
> apparently indeed helps significantly).

Actually, it's not my recommendation.  But the OP keeps claiming that
nothing else works for him.

My recommendation would be rather to make lines shorter.

> I consider bidi-display-reordering as a debugging tool rather than
> a user config, so I'm not very happy about this situation.

I'm not happy either (probably even less than you), but I'm not going
to agree that slow redisplay of 14K-character lines has anything to do
with bidirectional editing support.  _Anything_ that slows down
redisplay even a bit will have the same effect with such long lines,
e.g., JIT font lock, Flyspell, invisible text, you name it.  In fact,
even on a reasonably fast machine (mine is a core i7 screamer) Emacs
is unbearably slow with such long lines without reordering as well.
Maybe the OP has an unreasonably fast machine, but that just makes his
use case even more rare.

IOW, this is bug #13675, which has nothing to do with bidi.  As long
as the basic display algorithms are not changed to fix that bug, I'm
going to claim that bidi is not the issue here.

bug#15555: 24.3; Bidirectional display very slow with long lines

[Jerome L Quinn]

[-- Attachment #1: Type: text/plain, Size: 2392 bytes --]



Eli Zaretskii <eliz@gnu.org> wrote on 10/09/2013 12:59:26 PM:

> From: Eli Zaretskii <eliz@gnu.org>
> To: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Jerome L Quinn/Watson/IBM@IBMUS, 15555@debbugs.gnu.org
> Date: 10/09/2013 12:59 PM
> Subject: Re: bug#15555: 24.3; Bidirectional display very slow with long
lines
>
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: Jerome L Quinn <jlquinn@us.ibm.com>,  15555@debbugs.gnu.org
> > Date: Wed, 09 Oct 2013 08:26:58 -0400
> >
> > >> And disabling bidi reordering completely eliminates the bad
behavior.
> > > If you can afford that, go for it.
> >
> > IIRC this is the first report where setting bidi-display-reordering to
> > nil is really the best recommendation we can offer (and where it
> > apparently indeed helps significantly).
>
> Actually, it's not my recommendation.  But the OP keeps claiming that
> nothing else works for him.
>
> My recommendation would be rather to make lines shorter.

I can't make the lines shorter.  The file I'm looking at is
computer-generated.

I can disable reordering, which does solve the speed problem for me.  I'd
just like
to help identify the source of the issue so that it can be resolved at some
point.

> > I consider bidi-display-reordering as a debugging tool rather than
> > a user config, so I'm not very happy about this situation.
>
> I'm not happy either (probably even less than you), but I'm not going
> to agree that slow redisplay of 14K-character lines has anything to do
> with bidirectional editing support.  _Anything_ that slows down
> redisplay even a bit will have the same effect with such long lines,
> e.g., JIT font lock, Flyspell, invisible text, you name it.  In fact,
> even on a reasonably fast machine (mine is a core i7 screamer) Emacs
> is unbearably slow with such long lines without reordering as well.
> Maybe the OP has an unreasonably fast machine, but that just makes his
> use case even more rare.

I'm not sure what else I can do.  I timed how long it takes to page down
and
move the cursor and it's much slower with reordering enabled on my test
case.

No, moving around with 14K lines and reordering off is not lightning fast.
It
is however subsecond response on my machine.  Reordering brings subsecond
response up to multiple seconds as you are further along the line.

I am on a high-end 12 core xeon machine, so yes, the hardware is fast.


[-- Attachment #2: Type: text/html, Size: 3386 bytes --]

bug#3219: bug#15555: 24.3; Bidirectional display very slow with long lines

[Andrew Hyatt]

Jerome L Quinn <jlquinn@us.ibm.com> writes:

> Eli Zaretskii <eliz@gnu.org> wrote on 10/09/2013 12:59:26 PM:
>
>> From: Eli Zaretskii <eliz@gnu.org>
>> To: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: Jerome L Quinn/Watson/IBM@IBMUS, 15555@debbugs.gnu.org
>> Date: 10/09/2013 12:59 PM
>> Subject: Re: bug#15555: 24.3; Bidirectional display very slow with long lines
>> 
>> > From: Stefan Monnier <monnier@iro.umontreal.ca>
>> > Cc: Jerome L Quinn <jlquinn@us.ibm.com>, 15555@debbugs.gnu.org
>> > Date: Wed, 09 Oct 2013 08:26:58 -0400
>> > 
>> > >> And disabling bidi reordering completely eliminates the bad behavior.
>> > > If you can afford that, go for it.
>> > 
>> > IIRC this is the first report where setting bidi-display-reordering to
>> > nil is really the best recommendation we can offer (and where it
>> > apparently indeed helps significantly).
>> 
>> Actually, it's not my recommendation. But the OP keeps claiming that
>> nothing else works for him.
>> 
>> My recommendation would be rather to make lines shorter.
>
> I can't make the lines shorter. The file I'm looking at is computer-generated.
>
> I can disable reordering, which does solve the speed problem for me. I'd just like
> to help identify the source of the issue so that it can be resolved at some point.
>
>> > I consider bidi-display-reordering as a debugging tool rather than
>> > a user config, so I'm not very happy about this situation.
>> 
>> I'm not happy either (probably even less than you), but I'm not going
>> to agree that slow redisplay of 14K-character lines has anything to do
>> with bidirectional editing support. _Anything_ that slows down
>> redisplay even a bit will have the same effect with such long lines,
>> e.g., JIT font lock, Flyspell, invisible text, you name it. In fact,
>> even on a reasonably fast machine (mine is a core i7 screamer) Emacs
>> is unbearably slow with such long lines without reordering as well.
>> Maybe the OP has an unreasonably fast machine, but that just makes his
>> use case even more rare.
>
> I'm not sure what else I can do. I timed how long it takes to page down and
> move the cursor and it's much slower with reordering enabled on my test
> case.
>
> No, moving around with 14K lines and reordering off is not lightning fast. It
> is however subsecond response on my machine. Reordering brings subsecond
> response up to multiple seconds as you are further along the line.
>
> I am on a high-end 12 core xeon machine, so yes, the hardware is fast.

FWIW, on Emacs 25 on Mac OS X, the bidi text as reported in the initial
bug is still very slow.

bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> From: Andrew Hyatt <ahyatt@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  3219@debbugs.gnu.org,  15555@debbugs.gnu.org,  Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 26 Jan 2016 00:13:01 -0500
> 
> FWIW, on Emacs 25 on Mac OS X, the bidi text as reported in the initial
> bug is still very slow.

Nothing was done since then to speed up redisplay for such long lines.

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

[-- Attachment #1: Type: text/plain, Size: 796 bytes --]

On 10/09/2013 08:59 PM, Eli Zaretskii wrote:

> IOW, this is bug #13675, which has nothing to do with bidi.  As long
> as the basic display algorithms are not changed to fix that bug, I'm
> going to claim that bidi is not the issue here.

Hm... I have two files, with 2000 and 4000 first chars extracted from the beginning
of the monster string from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15555#5.

The following function:

(defun bug15555 ()
   (interactive)
   (while (not (eobp))
     (right-char 1)
     (redisplay)
     (sleep-for 0.01)))

runs smoothly over 2000.txt, but painfully slow over 4000.txt.  The latter
case also shows the very pathological profile with ~25% CPU spent in memcpy.

Are you sure that bidi_copy_it doesn't add one more bottleneck to the whole
stuff?

Dmitry


[-- Attachment #2: 2000.txt --]
[-- Type: text/plain, Size: 2935 bytes --]

[[null,["raw"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[-1,0,322]]]],["raw",["rawtext"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[0,0,322]]]],["rawtext",["text"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[0,0,322]]]],["text",["rawtok","toktype"],[["كل","0",[0,0,2]],["هذا","0",[0,3,6]],["الكره","0",[0,7,12]],["،","2048",[0,12,13]],["فائض","0",[0,14,18]],["الكراهية","0",[0,19,27]],["حالة","0",[0,28,32]],["غبية","0",[0,33,37]],["تلفنا","0",[0,38,43]],["من","0",[0,44,46]],["كل","0",[0,47,49]],["جانب","0",[0,50,54]],["،","2048",[0,54,55]],["عندما","0",[0,56,61]],["يتمنطق","0",[0,62,68]],["البعض","0",[0,69,74]],["بالكراهية","0",[0,75,84]],["،","2048",[0,84,85]],["عندما","0",[0,86,91]],["يعتمل","0",[0,92,97]],["الثأر","0",[0,98,103]],["فى","0",[0,104,106]],["النفوس","0",[0,107,113]],["،","2048",[0,113,114]],["تصير","0",[0,115,119]],["قنابل","0",[0,120,125]],["بشرية","0",[0,126,131]],["تمشى","0",[0,132,136]],["على","0",[0,137,140]],["قدمين","0",[0,141,146]],["،","2048",[0,146,147]],["إن","0",[0,148,150]],["رأت","0",[0,151,154]],["نارا","0",[0,155,159]],["تصب","0",[0,160,163]],["من","0",[0,164,166]],["نفوسها","0",[0,167,173]],["بنزي

[-- Attachment #3: 4000.txt --]
[-- Type: text/plain, Size: 5269 bytes --]

[[null,["raw"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[-1,0,322]]]],["raw",["rawtext"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[0,0,322]]]],["rawtext",["text"],[["كل هذا الكره، فائض الكراهية حالة غبية تلفنا من كل جانب، عندما يتمنطق البعض بالكراهية، عندما يعتمل الثأر فى النفوس، تصير قنابل بشرية تمشى على قدمين، إن رأت نارا تصب من نفوسها بنزينا، الكراهية حالة مرضية تستبد بالوطن، تعبير كاره عن غضب مستطير.. لا يفرق كثيرا عن الشرر المستطير، نيران الكراهية تعتمل فى الصدور، حمم لو تعلمون.",[0,0,322]]]],["text",["rawtok","toktype"],[["كل","0",[0,0,2]],["هذا","0",[0,3,6]],["الكره","0",[0,7,12]],["،","2048",[0,12,13]],["فائض","0",[0,14,18]],["الكراهية","0",[0,19,27]],["حالة","0",[0,28,32]],["غبية","0",[0,33,37]],["تلفنا","0",[0,38,43]],["من","0",[0,44,46]],["كل","0",[0,47,49]],["جانب","0",[0,50,54]],["،","2048",[0,54,55]],["عندما","0",[0,56,61]],["يتمنطق","0",[0,62,68]],["البعض","0",[0,69,74]],["بالكراهية","0",[0,75,84]],["،","2048",[0,84,85]],["عندما","0",[0,86,91]],["يعتمل","0",[0,92,97]],["الثأر","0",[0,98,103]],["فى","0",[0,104,106]],["النفوس","0",[0,107,113]],["،","2048",[0,113,114]],["تصير","0",[0,115,119]],["قنابل","0",[0,120,125]],["بشرية","0",[0,126,131]],["تمشى","0",[0,132,136]],["على","0",[0,137,140]],["قدمين","0",[0,141,146]],["،","2048",[0,146,147]],["إن","0",[0,148,150]],["رأت","0",[0,151,154]],["نارا","0",[0,155,159]],["تصب","0",[0,160,163]],["من","0",[0,164,166]],["نفوسها","0",[0,167,173]],["بنزينا","0",[0,174,180]],["،","2048",[0,180,181]],["الكراهية","0",[0,182,190]],["حالة","0",[0,191,195]],["مرضية","0",[0,196,201]],["تستبد","0",[0,202,207]],["بالوطن","0",[0,208,214]],["،","2048",[0,214,215]],["تعبير","0",[0,216,221]],["كاره","0",[0,222,226]],["عن","0",[0,227,229]],["غضب","0",[0,230,233]],["مستطير","0",[0,234,240]],["..","0",[0,240,242]],["لا","0",[0,243,245]],["يفرق","0",[0,246,250]],["كثيرا","0",[0,251,256]],["عن","0",[0,257,259]],["الشرر","0",[0,260,265]],["المستطير","0",[0,266,274]],["،","2048",[0,274,275]],["نيران","0",[0,276,281]],["الكراهية","0",[0,282,290]],["تعتمل","0",[0,291,296]],["فى","0",[0,297,299]],["الصدور","0",[0,300,306]],["،","2048",[0,306,307]],["حمم","0",[0,308,311]],["لو","0",[0,312,314]],["تعلمون","0",[0,315,321]],[".","6144",[0,321,322]]]],["rawtok",["tok"],[["كل",[0,0,2]],["هذا",[1,3,6]],["الكره",[2,7,12]],[",",[3,13,14]],["فائض",[4,15,19]],["الكراهية",[5,20,28]],["حالة",[6,29,33]],["غبية",[7,34,38]],["تلفنا",[8,39,44]],["من",[9,45,47]],["كل",[10,48,50]],["جانب",[11,51,55]],[",",[12,56,57]],["عندما",[13,58,63]],["يتمنطق",[14,64,70]],["البعض",[15,71,76]],["بالكراهية",[16,77,86]],[",",[17,87,88]],["عندما",[18,89,94]],["يعتمل",[19,95,100]],["الثأر",[20,101,106]],["فى",[21,107,109]],["النفوس",[22,110,116]],[",",[23,117,118]],["تصير",[24,119,123]],["قنابل",[25,124,129]],["بشرية",[26,130,135]],["تمشى",[27,136,140]],["على",[28,141,144]],["قدمين",[29,145,150]],[",",[30,151,152]],["إن",[31,153,155]],["رأت",[32,156,159]],["نارا",[33,160,164]],["تصب",[34,165,168]],["من",[35,169,171]],["نفوسها",[36,172,178]],["بنزينا",[37,179,185]],[",",[38,186,187]],["الكراهية",[39,188,196]],["حالة",[40,197,201]],["مرضية",[41,202,207]],["تستبد",[42,208,213]],["بالوطن",[43,214,220]],[",",[44,221,222]],["تعبير",[45,223,228]],["كاره",[46,229,233]],["عن",[47,234,236]],["غضب",[48,237,240]],["مستطير",[49,241,247]],["..",[50,248,250]],["لا",[51,251,253]],["يفرق",[52,254,258]],["كثيرا",[53,259,264]],["عن",[54,265,267]],["الشرر",[55,268,273]],["المستطير",[56,274,282

bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

On 02/18/2014 04:43 PM, Dmitry Antipov wrote:

> Are you sure that bidi_copy_it doesn't add one more bottleneck to the whole
> stuff?

There is a busy bidi_shelve_cache/bidi_unshelve_cache loop which processes
~1.5M of cache data per each iteration. That's why memcpy takes ~25% in the
overall profile...

Dmitry

bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Tue, 18 Feb 2014 18:01:05 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15555@debbugs.gnu.org
> 
> On 02/18/2014 04:43 PM, Dmitry Antipov wrote:
> 
> > Are you sure that bidi_copy_it doesn't add one more bottleneck to the whole
> > stuff?
> 
> There is a busy bidi_shelve_cache/bidi_unshelve_cache loop which processes
> ~1.5M of cache data per each iteration.

Where's that loop in the sources?

> That's why memcpy takes ~25% in the overall profile...

Why doesn't this happen in the other (smaller) file?

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Stefan Monnier]

> Are you sure that bidi_copy_it doesn't add one more bottleneck to the whole
> stuff?

Eli's argument is not that bidi adds its own bottlenecks, but that there
are sufficiently other bottlenecks that there's not much point trying to
address bidi's bottlenecks until we start tackling the other ones.


        Stefan

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

On 02/18/2014 06:04 PM, Stefan Monnier wrote:

> Eli's argument is not that bidi adds its own bottlenecks, but that there
> are sufficiently other bottlenecks that there's not much point trying to
> address bidi's bottlenecks until we start tackling the other ones.

Now I'm seeing the use case where each trivial cursor movement causes
~100 calls to bidi_shelve_cache, and each call copies ~1.5M of data.
If there is a redisplay problem which makes this unavoidable without
rewriting 1/2 of xdisp.c, then "broken by design" is the only thing
I can say about all of that mess.

Dmitry

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Tue, 18 Feb 2014 18:31:22 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> Cc: 15555@debbugs.gnu.org
> 
> Now I'm seeing the use case where each trivial cursor movement causes
> ~100 calls to bidi_shelve_cache, and each call copies ~1.5M of data.
> If there is a redisplay problem which makes this unavoidable without
> rewriting 1/2 of xdisp.c, then "broken by design" is the only thing
> I can say about all of that mess.

If you show me where these calls are made, I might be able to say
something intelligent about that.  And maybe we could even discuss the
issue and find some clever solution, if it exists.

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

On 02/18/2014 08:24 PM, Eli Zaretskii wrote:

> If you show me where these calls are made, I might be able to say
> something intelligent about that.  And maybe we could even discuss the
> issue and find some clever solution, if it exists.

(gdb) b bidi.c:669 ;;; at xmalloc
Breakpoint 1 at 0x500890: file ../../trunk/src/bidi.c, line 669.
(gdb) b bidi.c:756 ;;; at xfree
Breakpoint 2 at 0x500b55: file ../../trunk/src/bidi.c, line 756.
(gdb) r -Q /tmp/4000.txt

In 4000.txt, eval (goto-char 2769), then press up arrow ==>

Breakpoint 1, bidi_shelve_cache () at ../../trunk/src/bidi.c:669
669	  databuf = xmalloc (alloc);
(gdb) p alloc
$1 = 292820 ;;; ~290K
(gdb) bt 8
#0  bidi_shelve_cache () at ../../trunk/src/bidi.c:669
#1  0x00000000004518d2 in move_it_in_display_line_to (it=0x7fffffff9b60, to_charpos=2769, to_x=0, op=(MOVE_TO_X | MOVE_TO_POS))
     at ../../trunk/src/xdisp.c:8339
#2  0x00000000004542d7 in move_it_to (it=0x7fffffff9b60, to_charpos=2769, to_x=-1, to_y=593, to_vpos=-1, op=10)
     at ../../trunk/src/xdisp.c:8941
#3  0x000000000043a193 in pos_visible_p (w=0x10e6518, charpos=2769, x=0x7fffffffa93c, y=0x7fffffffa938, rtop=0x7fffffffa94c,
     rbot=0x7fffffffa948, rowh=0x7fffffffa944, vpos=0x7fffffffa940) at ../../trunk/src/xdisp.c:1409
#4  0x00000000004aba8c in Fpos_visible_in_window_p (pos=..., window=..., partially=...) at ../../trunk/src/window.c:1812
#5  0x000000000057f82b in Fposn_at_point (pos=..., window=...) at ../../trunk/src/keyboard.c:10730
#6  0x000000000060cf02 in Ffuncall (nargs=1, args=0x7fffffffab10) at ../../trunk/src/eval.c:2818
#7  0x000000000065681f in exec_byte_code (bytestr=..., vector=..., maxdepth=..., args_template=..., nargs=2, args=0x7fffffffb3b0)
     at ../../trunk/src/bytecode.c:919
(More stack frames follow...)
(gdb) c
Continuing.

Breakpoint 4, bidi_unshelve_cache (databuf=0x7fffe1a16010, just_free=true) at ../../trunk/src/bidi.c:756
756	      xfree (p);

etc. I can even do:

(gdb) c 1000
Will ignore next 999 crossings of breakpoint 1.  Continuing.

Breakpoint 4, bidi_unshelve_cache (databuf=0x7fffe19ce010, just_free=true) at ../../trunk/src/bidi.c:756
756	      xfree (p);

and the cursor is not moved yet.

Dmitry

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Tue, 18 Feb 2014 21:34:46 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15555@debbugs.gnu.org
> 
> On 02/18/2014 08:24 PM, Eli Zaretskii wrote:
> 
> > If you show me where these calls are made, I might be able to say
> > something intelligent about that.  And maybe we could even discuss the
> > issue and find some clever solution, if it exists.
> 
> (gdb) b bidi.c:669 ;;; at xmalloc
> Breakpoint 1 at 0x500890: file ../../trunk/src/bidi.c, line 669.
> (gdb) b bidi.c:756 ;;; at xfree
> Breakpoint 2 at 0x500b55: file ../../trunk/src/bidi.c, line 756.
> (gdb) r -Q /tmp/4000.txt
> 
> In 4000.txt, eval (goto-char 2769), then press up arrow ==>
> 
> Breakpoint 1, bidi_shelve_cache () at ../../trunk/src/bidi.c:669
> 669	  databuf = xmalloc (alloc);
> (gdb) p alloc
> $1 = 292820 ;;; ~290K
> (gdb) bt 8
> #0  bidi_shelve_cache () at ../../trunk/src/bidi.c:669
> #1  0x00000000004518d2 in move_it_in_display_line_to (it=0x7fffffff9b60, to_charpos=2769, to_x=0, op=(MOVE_TO_X | MOVE_TO_POS))
>      at ../../trunk/src/xdisp.c:8339
> #2  0x00000000004542d7 in move_it_to (it=0x7fffffff9b60, to_charpos=2769, to_x=-1, to_y=593, to_vpos=-1, op=10)
>      at ../../trunk/src/xdisp.c:8941
> #3  0x000000000043a193 in pos_visible_p (w=0x10e6518, charpos=2769, x=0x7fffffffa93c, y=0x7fffffffa938, rtop=0x7fffffffa94c,
>      rbot=0x7fffffffa948, rowh=0x7fffffffa944, vpos=0x7fffffffa940) at ../../trunk/src/xdisp.c:1409
> #4  0x00000000004aba8c in Fpos_visible_in_window_p (pos=..., window=..., partially=...) at ../../trunk/src/window.c:1812
> #5  0x000000000057f82b in Fposn_at_point (pos=..., window=...) at ../../trunk/src/keyboard.c:10730
> #6  0x000000000060cf02 in Ffuncall (nargs=1, args=0x7fffffffab10) at ../../trunk/src/eval.c:2818
> #7  0x000000000065681f in exec_byte_code (bytestr=..., vector=..., maxdepth=..., args_template=..., nargs=2, args=0x7fffffffb3b0)
>      at ../../trunk/src/bytecode.c:919
> (More stack frames follow...)
> (gdb) c
> Continuing.
> 
> Breakpoint 4, bidi_unshelve_cache (databuf=0x7fffe1a16010, just_free=true) at ../../trunk/src/bidi.c:756
> 756	      xfree (p);
> 
> etc. I can even do:
> 
> (gdb) c 1000
> Will ignore next 999 crossings of breakpoint 1.  Continuing.
> 
> Breakpoint 4, bidi_unshelve_cache (databuf=0x7fffe19ce010, just_free=true) at ../../trunk/src/bidi.c:756
> 756	      xfree (p);
> 
> and the cursor is not moved yet.

Thanks.  That's the "infloop" I described in my other message.  It
should disappear if you try one of the "remedies" I described there.

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Tue, 18 Feb 2014 21:34:46 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15555@debbugs.gnu.org
> 
> (gdb) b bidi.c:669 ;;; at xmalloc
> Breakpoint 1 at 0x500890: file ../../trunk/src/bidi.c, line 669.
> (gdb) b bidi.c:756 ;;; at xfree
> Breakpoint 2 at 0x500b55: file ../../trunk/src/bidi.c, line 756.
> (gdb) r -Q /tmp/4000.txt
> 
> In 4000.txt, eval (goto-char 2769), then press up arrow ==>
> 
> Breakpoint 1, bidi_shelve_cache () at ../../trunk/src/bidi.c:669
> 669	  databuf = xmalloc (alloc);
> (gdb) p alloc
> $1 = 292820 ;;; ~290K

That's 290K, not 1.5M.  But even 1.5M should take about 100 usec to
move with memcpy.  So this is not a catastrophe.

Of course, doing this in a never-ending loop is not nice, but that's
not a slowdown, that's a bug.

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

On 02/19/2014 09:43 PM, Eli Zaretskii wrote:

> That's 290K, not 1.5M.  But even 1.5M should take about 100 usec to
> move with memcpy.  So this is not a catastrophe.

Even if it happens 1000 times per second?

If 190M doesn't bother you, it's still worth
to see at http://37.139.80.10/tmp/4000.avi.

Dmitry

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Thu, 20 Feb 2014 11:32:31 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15555@debbugs.gnu.org
> 
> On 02/19/2014 09:43 PM, Eli Zaretskii wrote:
> 
> > That's 290K, not 1.5M.  But even 1.5M should take about 100 usec to
> > move with memcpy.  So this is not a catastrophe.
> 
> Even if it happens 1000 times per second?
> 
> If 190M doesn't bother you, it's still worth
> to see at http://37.139.80.10/tmp/4000.avi.

Does it look better now?

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

On 02/20/2014 09:45 PM, Eli Zaretskii wrote:

> Does it look better now?

Definitely. Thanks.

Dmitry

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Tue, 18 Feb 2014 18:31:22 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> Cc: 15555@debbugs.gnu.org
> 
> Now I'm seeing the use case where each trivial cursor movement causes
> ~100 calls to bidi_shelve_cache, and each call copies ~1.5M of data.

Is this still the same use case with your bug15555 function and the
4000.txt file?  If not, please describe this use case.

As for calls to bidi_shelve_cache, they are a side effect of saving
and restoring the iterator object, see the commentary before the
definition of the SAVE_IT macro.

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

[-- Attachment #1: Type: text/plain, Size: 420 bytes --]

On 02/18/2014 09:58 PM, Eli Zaretskii wrote:

> Is this still the same use case with your bug15555 function and the
> 4000.txt file?  If not, please describe this use case.

Yes.

There is a simple patch to trace bidi_shelve_cache/bidi_unshelve_cache and
two screencasts, with bug15555 over 2000.txt [1] and 4000.txt [2], respectively.

[1] http://37.139.80.10/tmp/2000.mkv
[2] http://37.139.80.10/tmp/4000.mkv

Dmitry


[-- Attachment #2: trace_bug15555.patch --]
[-- Type: text/x-patch, Size: 882 bytes --]

=== modified file 'src/bidi.c'
--- src/bidi.c	2014-01-01 17:44:48 +0000
+++ src/bidi.c	2014-02-19 04:07:08 +0000
@@ -62,6 +62,7 @@
 #include "buffer.h"
 #include "dispextern.h"
 #include "region-cache.h"
+#include "systime.h"
 
 static bool bidi_initialized = 0;
 
@@ -668,6 +669,8 @@
 	   + bidi_cache_idx * sizeof (struct bidi_it));
   databuf = xmalloc (alloc);
   bidi_cache_total_alloc += alloc;
+  fprintf (stderr, "%s [%f]: alloc %"pI"d bytes -> %p\n", __func__,
+	   timespectod (current_timespec ()), alloc, databuf);
 
   memcpy (databuf, &bidi_cache_idx, sizeof (bidi_cache_idx));
   memcpy (databuf + sizeof (bidi_cache_idx),
@@ -752,6 +755,8 @@
 	    -= (bidi_shelve_header_size
 		+ bidi_cache_idx * sizeof (struct bidi_it));
 	}
+      fprintf (stderr, "%s [%f]: free %p\n", __func__,
+	       timespectod (current_timespec ()), databuf);
 
       xfree (p);
     }

bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Wed, 19 Feb 2014 09:48:21 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15555@debbugs.gnu.org
> 
> There is a simple patch to trace bidi_shelve_cache/bidi_unshelve_cache and
> two screencasts, with bug15555 over 2000.txt [1] and 4000.txt [2], respectively.
> 
> [1] http://37.139.80.10/tmp/2000.mkv
> [2] http://37.139.80.10/tmp/4000.mkv

I cannot play MKV files here, sorry.

Do they show something different from what I described?

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Tue, 18 Feb 2014 18:31:22 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> Cc: 15555@debbugs.gnu.org
> 
> Now I'm seeing the use case where each trivial cursor movement causes
> ~100 calls to bidi_shelve_cache, and each call copies ~1.5M of data.

Starting with revision 116494, bidi_shelve_cache should be called much
less, normally just once and sometimes twice per call to
move_it_in_display_line_to.

This doesn't solve the original bug, so the cursor will occasionally
still get stuck in that 4000.txt file, but at least it will do that
with much less noise.

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  15555@debbugs.gnu.org
> Date: Tue, 18 Feb 2014 09:04:56 -0500
> 
> > Are you sure that bidi_copy_it doesn't add one more bottleneck to the whole
> > stuff?
> 
> Eli's argument is not that bidi adds its own bottlenecks, but that there
> are sufficiently other bottlenecks that there's not much point trying to
> address bidi's bottlenecks until we start tackling the other ones.

Yes.  Basically, I claim that half of infinity is still infinity.

But the situation described by Dmitry does not belong to that class of
redisplay problem.

bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Tue, 18 Feb 2014 16:43:52 +0400
> From: Dmitry Antipov <antipov@dev.rtsoft.ru>
> CC: 15555@debbugs.gnu.org
> 
> Hm... I have two files, with 2000 and 4000 first chars extracted from the beginning
> of the monster string from http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15555#5.
> 
> The following function:
> 
> (defun bug15555 ()
>    (interactive)
>    (while (not (eobp))
>      (right-char 1)
>      (redisplay)
>      (sleep-for 0.01)))
> 
> runs smoothly over 2000.txt, but painfully slow over 4000.txt.

I'm not sure I can reproduce this.  Is it possible that the difference
you see between the speed of moving the cursor in these the two files
is due solely to the fact that the smaller file fits completely in the
window, while the larger file does not?  IOW, if you enlarge the
window such that the larger file is visible in its entirety, don't you
see the same speed as with the smaller file?  (I need about 52 lines
of text in the window to show the whole of the file 4000.txt.)

Likewise if you insert a R2L character at the beginning of 4000.txt,
thus making its paragraph take the R2L base direction: now the cursor
motion is fast even if the display needs to scroll to keep point
visible, i.e. with the original window size you get in "emacs -Q".

Also, the cursor movement is quite fast, and uses about 8% of a single
execution unit of my Core i7, until the first time it needs to go
outside the visible portion of the buffer, at which point the CPU
usage of that single execution unit soars to almost 80%.

If you see something different, please describe what you see.

If you see what I described above, then this is a known (to me ;-)
problem: when we have a continued line that takes more than 2 screen
lines, and that line consists of mostly R2L characters, but the
paragraph direction is L2R (or vice versa), then redisplay sometimes
infloops when it needs to scroll the text in the window.  The loop is
not on the C level, it just causes a continuous series of re-entering
redisplay, so you can stop this "infloop" by some radical cursor
motion command, like C-v.

This problem is hard to solve, because the code which finds a suitable
window-start intrinsically assumes that the window-start position is
always at column zero of the window, which is false with bidirectional
text.  Some of the code that is related to this needs to be redesigned
to avoid this assumption.  I hope to get to this some day, but since
this situation is quite rare (paragraphs full of R2L characters
normally have R2L base direction), this problem was never high on my
todo list.

> The latter case also shows the very pathological profile with ~25%
> CPU spent in memcpy.
> 
> Are you sure that bidi_copy_it doesn't add one more bottleneck to the whole
> stuff?

No, I don't think it does.  But doing that in an infloop might ;-)

Anyway, just moving cursor horizontally cannot possibly be slow due to
bidi, especially as long as point stays in the same screenful.  The
redisplay becomes unbearably slow with long lines only when you either
scroll the display (e.g., C-v) or for vertical cursor motion, because
these require the display engine to traverse many buffer positions,
many more than is needed to just move the cursor, and it currently can
only start that traversal from the beginning of a physical line.

bug#15555: Re: bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

On 02/18/2014 09:42 PM, Eli Zaretskii wrote:

> Anyway, just moving cursor horizontally cannot possibly be slow due to
> bidi, especially as long as point stays in the same screenful.  The
> redisplay becomes unbearably slow with long lines only when you either
> scroll the display (e.g., C-v) or for vertical cursor motion, because
> these require the display engine to traverse many buffer positions,
> many more than is needed to just move the cursor, and it currently can
> only start that traversal from the beginning of a physical line.

1) I realize that vertical motion is slower than horizontal, but [2] from
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15555#65 shows that the major
slowdown happens when cursor is moved horizontally (by right-char) within
the same line.

2) (setq bidi-display-reordering nil) helps bug15555 to run over 4000.txt
just as expected.

Dmitry

bug#15555: 24.3; Bidirectional display very slow with long lines

[Eli Zaretskii]

> Date: Wed, 19 Feb 2014 14:49:39 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15555@debbugs.gnu.org
> 
> On 02/18/2014 09:42 PM, Eli Zaretskii wrote:
> 
> > Anyway, just moving cursor horizontally cannot possibly be slow due to
> > bidi, especially as long as point stays in the same screenful.  The
> > redisplay becomes unbearably slow with long lines only when you either
> > scroll the display (e.g., C-v) or for vertical cursor motion, because
> > these require the display engine to traverse many buffer positions,
> > many more than is needed to just move the cursor, and it currently can
> > only start that traversal from the beginning of a physical line.
> 
> 1) I realize that vertical motion is slower than horizontal, but [2] from
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15555#65 shows that the major
> slowdown happens when cursor is moved horizontally (by right-char) within
> the same line.

According to the value of point that you gave, this happens when the
next buffer position, the one where right-char should move, is beyond
the window, i.e. not on the same screen line.

Did you try enlarging the window so that the entire text of 4000.txt
fits in the window?  Do you still see slow cursor movement in that
case?

> 2) (setq bidi-display-reordering nil) helps bug15555 to run over 4000.txt
> just as expected.

Of course, because then the bug I described, which causes endless
re-entering of redisplay, doesn't happen.

IOW, this is a bug in that specific situation, not a slow-down.

bug#15555: 24.3; Bidirectional display very slow with long lines

[Dmitry Antipov]

On 02/19/2014 09:39 PM, Eli Zaretskii wrote:

> According to the value of point that you gave, this happens when the
> next buffer position, the one where right-char should move, is beyond
> the window, i.e. not on the same screen line.
>
> Did you try enlarging the window so that the entire text of 4000.txt
> fits in the window?  Do you still see slow cursor movement in that
> case?

When I toggle fullscreen and the whole text fits in the window,
it works smoothly as with 2000.txt example.

Dmitry