bug#37620: 26.1; ada-mode 6.2.1 partial parsing occasionally hangs while moving point

bug#37620: 26.1; ada-mode 6.2.1 partial parsing occasionally hangs while moving point

[Ludovic Brenta]

X-Debbugs-CC: ada-mode-users@nongnu.org
Package: emacs, ada-mode
Version: 26.1
Severity: minor

I'm sorry but I cannot trigger this problem reliably.  Occasionally,
while simply moving in a source file (using up, down, page up, page
down etc.), the *Messages* buffer receives these errors:

Error during redisplay: (jit-lock-function 32410) signaled (error 
"wisi-process-parse not getting more text (or bad syntax in process 
output)")
Error during redisplay: (jit-lock-function 32910) signaled (error 
"wisi-process-parse not getting more text (or bad syntax in process 
output)")
Error during redisplay: (jit-lock-function 32940) signaled (error 
"wisi-process-parse not getting more text (or bad syntax in process 
output)")
Error during redisplay: (jit-lock-function 32971) signaled (error 
"wisi-process-parse not getting more text (or bad syntax in process 
output)")
Error during redisplay: (jit-lock-function 32973) signaled (error 
"wisi-process-parse not getting more text (or bad syntax in process 
output)")
Error during redisplay: (jit-lock-function 32988) signaled (error 
"wisi-process-parse not getting more text (or bad syntax in process 
output)")

and the redisplay freezes.  When this happens I usually have to do
M-x wiki-kill-parser to recover.

This happens once every 3-4 days, both on files that are syncatically
correct and on some being edited (I don't think it makes a difference
anymore as the partial parser is in use in both cases).  Has anyone
observed these errors before?  Could it be caused by moving point too
rapidly for the parser process to keep up?

-- 
Ludovic Brenta.

bug#37620: update

[Stephen Leake]

The error message:

Error during redisplay: (jit-lock-function 32410) signaled (error 
"wisi-process-parse not getting more text (or bad syntax in process 
output)")

indicates that the parser running in the external process is taking too
long to responde, so the elisp interface is timing out.

This can be due to two causes;

1. the code has a syntax error that is particularly hard to correct

2. the computer is otherwise loaded, so the parser is running slowly.

Since you report it happening in several files at different places,
cause 1. is not likely.

You can try increasing wisi-process-time-out (default 5 seconds); that
is how long the elisp interace waits for the external parser.

I'll change the error message to mention wisi-process-time-out.

-- 
-- Stephe

bug#37620: 26.1; ada-mode 6.2.1 partial parsing occasionally hangs while moving point

[Ludovic Brenta]

No, the machine is not particularly loaded.  I believe the most
likely cause is "the code has a syntax error that is particularly
hard to correct"; but this is only because of partial parsing as
the entire file is syntactically correct.

I'll send you the next source file where this happens; maybe
you can reproduce.

Increasing wisi-process-time-out to more than 5 seconds does not
seem like a solution to me; parsing the entire buffer (resulting in
no syntax errors, no attempted recovery and no time-out) takes
less than that.  Also, since we are only moving point in an
unmodified buffer, interactive use requires 0.1 s response time or
less.  Therefore, perhaps a solution would be to /decrease/
wisi-process-time-out and automatically launch a full parse
whenever it happens; as long as the buffer remains unmodified, no
re-parsing will take place.

-- 
Ludovic Brenta.

bug#37620: 26.1; ada-mode 6.2.1 partial parsing occasionally hangs while moving point

[Ludovic Brenta]

Triggering this bug is much easier when you jump into the
middle of a file using cross-references from another file
or etags, as opposed to opening the file at the beginning
and then scrolling.  Because if you simply visit a file
(even a large one), partial parsing starts from the beginning
of the file and has a much lower chance of finding syntax
errors.

-- 
Ludovic Brenta.

bug#37620: 26.1; ada-mode 6.2.1 partial parsing occasionally hangs while moving point

[Stephen Leake]

[-- Attachment #1: Type: text/plain, Size: 134 bytes --]

There is a bug in error recovery that can cause the parser to hang; that
may be the cause of this bug.

Patch attached.
-- 
-- Stephe

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch.diff --]
[-- Type: text/x-patch, Size: 9327 bytes --]

--- a/wisitoken-parse-lr-mckenzie_recover-base.adb
+++ b/wisitoken-parse-lr-mckenzie_recover-base.adb
@@ -2,7 +2,7 @@
 --
 --  Base utilities for McKenzie_Recover
 --
---  Copyright (C) 2018, 2019 Free Software Foundation, Inc.
+--  Copyright (C) 2018 - 2020 Free Software Foundation, Inc.
 --
 --  This library is free software;  you can redistribute it and/or modify it
 --  under terms of the  GNU General Public License  as published by the Free
@@ -30,49 +30,56 @@ package body WisiToken.Parse.LR.McKenzie_Recover.Base is
      return Boolean
    is
       Done_Count : SAL.Base_Peek_Type := 0;
+      Skip : Boolean;
    begin
       --  Return True if all parsers are done, or if any parser has a config
       --  available to check.
       for P_Status of Parser_Status loop
+         Skip := False;
+
          case P_Status.Recover_State is
          when Active | Ready =>
             if P_Status.Parser_State.Recover.Config_Heap.Count > 0 then
                if P_Status.Parser_State.Recover.Check_Count - Check_Delta_Limit >= Min_Success_Check_Count then
                   --  fail; another parser succeeded, this one taking too long.
                   Done_Count := Done_Count + 1;
+                  Skip := True;
 
                elsif Total_Enqueue_Count + P_Status.Parser_State.Recover.Config_Full_Count >= Enqueue_Limit then
                   --  fail
                   Done_Count := Done_Count + 1;
+                  Skip := True;
                end if;
             end if;
 
-            case P_Status.Recover_State is
-            when Active =>
-               if P_Status.Parser_State.Recover.Config_Heap.Count > 0 then
-                  --  Still working
-                  return True;
-               else
-                  if P_Status.Active_Workers = 0 then
-                     --  fail; no configs left to check.
-                     Done_Count := Done_Count + 1;
+            if not Skip then
+               case P_Status.Recover_State is
+               when Active =>
+                  if P_Status.Parser_State.Recover.Config_Heap.Count > 0 then
+                     --  Still working
+                     return True;
+                  else
+                     if P_Status.Active_Workers = 0 then
+                        --  fail; no configs left to check.
+                        Done_Count := Done_Count + 1;
+                     end if;
                   end if;
-               end if;
 
-            when Ready =>
-               if P_Status.Parser_State.Recover.Config_Heap.Count > 0 and then
-                 P_Status.Parser_State.Recover.Config_Heap.Min_Key <= P_Status.Parser_State.Recover.Results.Min_Key
-               then
-                  --  Still more to check.
-                  return True;
+               when Ready =>
+                  if P_Status.Parser_State.Recover.Config_Heap.Count > 0 and then
+                    P_Status.Parser_State.Recover.Config_Heap.Min_Key <= P_Status.Parser_State.Recover.Results.Min_Key
+                  then
+                     --  Still more to check.
+                     return True;
 
-               elsif P_Status.Active_Workers = 0 then
-                  Done_Count := Done_Count + 1;
-               end if;
+                  elsif P_Status.Active_Workers = 0 then
+                     Done_Count := Done_Count + 1;
+                  end if;
 
-            when others =>
-               null;
-            end case;
+               when others =>
+                  null;
+               end case;
+            end if;
 
          when Success | Fail =>
             Done_Count := Done_Count + 1;
@@ -135,6 +142,7 @@ package body WisiToken.Parse.LR.McKenzie_Recover.Base is
           (Parsers, Parser_Status, Min_Success_Check_Count, Total_Enqueue_Count, Check_Delta_Limit, Enqueue_Limit)
       is
          Done_Count     : SAL.Base_Peek_Type := 0;
+         Skip           : Boolean;
          Min_Cost       : Integer            := Integer'Last;
          Min_Cost_Index : SAL.Base_Peek_Type;
 
@@ -168,6 +176,8 @@ package body WisiToken.Parse.LR.McKenzie_Recover.Base is
          --  No task_id in outline trace messages, because they may appear in
          --  .parse_good
          for I in Parser_Status'Range loop
+            Skip := False;
+
             declare
                P_Status : Base.Parser_Status renames Parser_Status (I);
             begin
@@ -186,6 +196,7 @@ package body WisiToken.Parse.LR.McKenzie_Recover.Base is
                         P_Status.Fail_Mode     := Fail_Check_Delta;
 
                         Done_Count := Done_Count + 1;
+                        Skip := True;
 
                      elsif Total_Enqueue_Count + P_Status.Parser_State.Recover.Config_Full_Count >= Enqueue_Limit then
                         if Trace_McKenzie > Outline then
@@ -200,48 +211,51 @@ package body WisiToken.Parse.LR.McKenzie_Recover.Base is
                         P_Status.Fail_Mode     := Fail_Enqueue_Limit;
 
                         Done_Count := Done_Count + 1;
+                        Skip := True;
                      end if;
                   end if;
 
-                  case P_Status.Recover_State is
-                  when Active =>
-                     if P_Status.Parser_State.Recover.Config_Heap.Count > 0 then
-                        if P_Status.Parser_State.Recover.Config_Heap.Min_Key < Min_Cost then
-                           Min_Cost       := P_Status.Parser_State.Recover.Config_Heap.Min_Key;
-                           Min_Cost_Index := I;
-                           --  not done
-                        end if;
-                     else
-                        if P_Status.Active_Workers = 0 then
-                           --  No configs left to check (rarely happens with real languages).
-                           if Trace_McKenzie > Outline then
-                              Put_Line
-                                (Trace.all, P_Status.Parser_State.Label, "fail; no configs left", Task_ID => False);
+                  if not Skip then
+                     case P_Status.Recover_State is
+                     when Active =>
+                        if P_Status.Parser_State.Recover.Config_Heap.Count > 0 then
+                           if P_Status.Parser_State.Recover.Config_Heap.Min_Key < Min_Cost then
+                              Min_Cost       := P_Status.Parser_State.Recover.Config_Heap.Min_Key;
+                              Min_Cost_Index := I;
+                              --  not done
+                           end if;
+                        else
+                           if P_Status.Active_Workers = 0 then
+                              --  No configs left to check (rarely happens with real languages).
+                              if Trace_McKenzie > Outline then
+                                 Put_Line
+                                   (Trace.all, P_Status.Parser_State.Label, "fail; no configs left", Task_ID => False);
+                              end if;
+                              P_Status.Recover_State := Fail;
+                              P_Status.Fail_Mode     := Fail_No_Configs_Left;
+
+                              Done_Count := Done_Count + 1;
                            end if;
-                           P_Status.Recover_State := Fail;
-                           P_Status.Fail_Mode     := Fail_No_Configs_Left;
-
-                           Done_Count := Done_Count + 1;
                         end if;
-                     end if;
 
-                  when Ready =>
-                     if P_Status.Parser_State.Recover.Config_Heap.Count > 0 and then
-                       P_Status.Parser_State.Recover.Config_Heap.Min_Key <=
-                       P_Status.Parser_State.Recover.Results.Min_Key
-                     then
-                        --  Still more to check. We don't check Min_Cost here so this parser
-                        --  can finish quickly.
-                        Set_Outputs (I);
-                        return;
-
-                     elsif P_Status.Active_Workers = 0 then
-                        P_Status.Recover_State := Success;
-                        Done_Count             := Done_Count + 1;
-                     end if;
-                  when others =>
-                     null;
-                  end case;
+                     when Ready =>
+                        if P_Status.Parser_State.Recover.Config_Heap.Count > 0 and then
+                          P_Status.Parser_State.Recover.Config_Heap.Min_Key <=
+                          P_Status.Parser_State.Recover.Results.Min_Key
+                        then
+                           --  Still more to check. We don't check Min_Cost here so this parser
+                           --  can finish quickly.
+                           Set_Outputs (I);
+                           return;
+
+                        elsif P_Status.Active_Workers = 0 then
+                           P_Status.Recover_State := Success;
+                           Done_Count             := Done_Count + 1;
+                        end if;
+                     when others =>
+                        null;
+                     end case;
+                  end if;
 
                when Success | Fail =>
                   Done_Count := Done_Count + 1;

bug#37620: closed by ada-mode version 7.1.3

[Stephen Leake]

-- 
-- Stephe