bug#41740: 26.1; completion-at-point fails in the Pacal mode

bug#41740: 26.1; completion-at-point fails in the Pacal mode

[Shinichi Sakata]

[-- Attachment #1: Type: text/plain, Size: 4738 bytes --]

 Use of C-M-i (completion-at-point) in the Pascal mode results in errors.

Here is a way to reproduce the problem:

   1. Start emacs by "emacs -Q".
   2. Open a new file with extension "pas" (say, test.pas).
   3. Type in: program test; var and then use C-M-i.

This should yields an error message like:

   completion--some: Args out of range: #("var" 0 3 (fontified t face
font-lock-type-face)), 0, -14
In GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.5)
 of 2019-09-22, modified by Debian built on x86-grnet-01
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux 10 (buster)

Recent messages:
Auto-saving...
Mark set [2 times]
Saving file /home/ssakata/codesignal/pascal/test.pas...
Wrote /home/ssakata/codesignal/pascal/test.pas
pascal-completion: End of buffer [6 times]
completion--some: Args out of range: #("var" 0 3 (fontified t face
font-lock-type-face)), 0, -14
pascal-completion: End of buffer
No match [7 times]
completion--some: Args out of range: #("in" 0 2 (fontified t face
font-lock-keyword-face)), 0, -25
Quit

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-StqULU/emacs-26.1+1=.
-fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LANG: en_US.utf8
  value of $XMODIFIERS: @im=fcitx
  locale-coding-system: utf-8-unix

Major mode: Pascal

Minor modes in effect:
  diff-auto-refine-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml mml-sec password-cache epa derived epg epg-config
gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils vc-git diff-mode easymenu
easy-mmode pascal elec-pair time-date mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 100828 15914)
 (symbols 48 20975 1)
 (miscs 40 58 270)
 (strings 32 30377 1460)
 (string-bytes 1 794739)
 (vectors 16 15486)
 (vector-slots 8 506994 9878)
 (floats 8 53 384)
 (intervals 56 424 0)
 (buffers 992 14))

[-- Attachment #2: Type: text/html, Size: 5378 bytes --]

bug#41740: 26.1; completion-at-point fails in the Pacal mode

[Dmitry Gutov]

On 06.06.2020 20:46, Shinichi Sakata wrote:
> Use of C-M-i (completion-at-point) in the Pascal mode results in errors.
> 
> Here is a way to reproduce the problem:
> 
>     1. Start emacs by "emacs -Q".
>     2. Open a new file with extension "pas" (say, test.pas).
>     3. Type in: program test; var and then use C-M-i.
> 
> This should yields an error message like:
> 
>     completion--some: Args out of range: #("var" 0 3 (fontified t face 
> font-lock-type-face)), 0, -14
> In GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.5)
>   of 2019-09-22, modified by Debian built on x86-grnet-01
> Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
> System Description: Debian GNU/Linux 10 (buster)

And I can confirm it on master.

bug#41740: 26.1; completion-at-point fails in the Pascal mode

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

Thanks for the report! It looks like pascal-mode didn't obey the rules for completion functions: they mustn't move point.

Would you try these patches? The first should fix your bug, and the second deals with an apparent bug in C-M-a in pascal-mode found when fixing the first one.


[-- Attachment #2: 0001-Preserve-point-in-pascal-mode-completion-bug-41740.patch --]
[-- Type: application/octet-stream, Size: 5970 bytes --]

From a3e9f2c67c2feacbb6941d37966c251f03f99b87 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Sun, 21 Jun 2020 21:04:30 +0200
Subject: [PATCH 1/2] Preserve point in pascal-mode completion (bug#41740)

Failure to do so caused errors in several cases.
Reported by Shinichi Sakata.

* lisp/progmodes/pascal.el (pascal-type-completion)
(pascal-completion): Wrap code that may move point in save-excursion.
* test/lisp/progmodes/pascal-tests.el: New file.
---
 lisp/progmodes/pascal.el            | 51 +++++++++++++-------------
 test/lisp/progmodes/pascal-tests.el | 55 +++++++++++++++++++++++++++++
 2 files changed, 82 insertions(+), 24 deletions(-)
 create mode 100644 test/lisp/progmodes/pascal-tests.el

diff --git a/lisp/progmodes/pascal.el b/lisp/progmodes/pascal.el
index 536a16dbb3..b0191c029b 100644
--- a/lisp/progmodes/pascal.el
+++ b/lisp/progmodes/pascal.el
@@ -1170,26 +1170,27 @@ pascal-get-completion-decl
 
 (defun pascal-type-completion (pascal-str)
   "Calculate all possible completions for types."
-  (let ((start (point))
-        (pascal-all ())
-	goon)
-    ;; Search for all reachable type declarations
-    (while (or (pascal-beg-of-defun)
-	       (setq goon (not goon)))
-      (save-excursion
-	(if (and (< start (prog1 (save-excursion (pascal-end-of-defun)
-						 (point))
-			    (forward-char 1)))
-		 (re-search-forward
-		  "\\<type\\>\\|\\<\\(begin\\|function\\|procedure\\)\\>"
-		  start t)
-		 (not (match-end 1)))
-	    ;; Check current type declaration
-            (setq pascal-all
-                  (nconc (pascal-get-completion-decl pascal-str)
-                         pascal-all)))))
+  (save-excursion
+    (let ((start (point))
+          (pascal-all ())
+	  goon)
+      ;; Search for all reachable type declarations
+      (while (or (pascal-beg-of-defun)
+	         (setq goon (not goon)))
+        (save-excursion
+	  (if (and (< start (prog1 (save-excursion (pascal-end-of-defun)
+						   (point))
+			      (forward-char 1)))
+		   (re-search-forward
+		    "\\<type\\>\\|\\<\\(begin\\|function\\|procedure\\)\\>"
+		    start t)
+		   (not (match-end 1)))
+	      ;; Check current type declaration
+              (setq pascal-all
+                    (nconc (pascal-get-completion-decl pascal-str)
+                           pascal-all)))))
 
-    pascal-all))
+      pascal-all)))
 
 (defun pascal-var-completion (prefix)
   "Calculate all possible completions for variables (or constants)."
@@ -1263,11 +1264,13 @@ pascal-completion
                     (and (eq state 'defun)
                          (save-excursion
                            (re-search-backward ")[ \t]*:" (point-at-bol) t))))
-                (if (or (eq state 'paramlist) (eq state 'defun))
-                    (pascal-beg-of-defun))
-                (nconc
-                 (pascal-type-completion pascal-str)
-                 (pascal-keyword-completion pascal-type-keywords pascal-str)))
+                (save-excursion
+                  (if (or (eq state 'paramlist) (eq state 'defun))
+                      (pascal-beg-of-defun))
+                  (nconc
+                   (pascal-type-completion pascal-str)
+                   (pascal-keyword-completion pascal-type-keywords
+                                              pascal-str))))
                (                        ;--Starting a new statement
                 (and (not (eq state 'contexp))
                      (save-excursion
diff --git a/test/lisp/progmodes/pascal-tests.el b/test/lisp/progmodes/pascal-tests.el
new file mode 100644
index 0000000000..10d6e0433d
--- /dev/null
+++ b/test/lisp/progmodes/pascal-tests.el
@@ -0,0 +1,55 @@
+;;; pascal-tests.el --- tests for pascal.el    -*- lexical-binding: t -*-
+
+;; Copyright (C) 2020 Free Software Foundation, Inc.
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+
+(require 'ert)
+(require 'pascal)
+
+(ert-deftest pascal-completion ()
+  ;; Bug#41740: completion functions must preserve point.
+  (let ((pascal-completion-cache nil))
+    (with-temp-buffer
+      (pascal-mode)
+      (insert "program test; var")
+      (let* ((point-before (point))
+             (completions (pascal-completion "var" nil 'metadata))
+             (point-after (point)))
+        (should (equal completions nil))
+        (should (equal point-before point-after)))))
+
+  (let ((pascal-completion-cache nil))
+    (with-temp-buffer
+      (pascal-mode)
+      (insert "program test; function f(x : i")
+      (let* ((point-before (point))
+             (completions (pascal-completion "i" nil 'metadata))
+             (point-after (point)))
+        (should (equal completions nil))
+        (should (equal point-before point-after)))))
+
+  (let ((pascal-completion-cache nil))
+    (with-temp-buffer
+      (pascal-mode)
+      (insert "program test; function f(x : integer) : real")
+      (let* ((point-before (point))
+             (completions (pascal-completion "real" nil 'metadata))
+             (point-after (point)))
+        (should (equal completions nil))
+        (should (equal point-before point-after))))))
+
+(provide 'pascal-tests)
-- 
2.21.1 (Apple Git-122.3)


[-- Attachment #3: 0002-Fix-spurious-error-in-beginning-of-defun-in-pascal-m.patch --]
[-- Type: application/octet-stream, Size: 1694 bytes --]

From 6f20bd6baef7bcc490abd81fd177427d86cd0458 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Sun, 21 Jun 2020 21:11:17 +0200
Subject: [PATCH 2/2] Fix spurious error in beginning-of-defun in pascal-mode
 (bug#41740)

* lisp/progmodes/pascal.el (pascal-beg-of-defun):
Ignore errors in forward-sexp.
* test/lisp/progmodes/pascal-tests.el (pascal-beg-of-defun): New test.
---
 lisp/progmodes/pascal.el            | 2 +-
 test/lisp/progmodes/pascal-tests.el | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/pascal.el b/lisp/progmodes/pascal.el
index b0191c029b..fce059bafc 100644
--- a/lisp/progmodes/pascal.el
+++ b/lisp/progmodes/pascal.el
@@ -589,7 +589,7 @@ pascal-beg-of-defun
   (interactive)
   (catch 'found
     (if (not (looking-at (concat "\\s \\|\\s)\\|" pascal-defun-re)))
-	(forward-sexp 1))
+	(ignore-errors (forward-sexp 1)))
     (let ((nest 0) (max -1) (func 0)
 	  (reg (concat pascal-beg-block-re "\\|"
 		       pascal-end-block-re "\\|"
diff --git a/test/lisp/progmodes/pascal-tests.el b/test/lisp/progmodes/pascal-tests.el
index 10d6e0433d..ed4c6fb03e 100644
--- a/test/lisp/progmodes/pascal-tests.el
+++ b/test/lisp/progmodes/pascal-tests.el
@@ -52,4 +52,12 @@ pascal-completion
         (should (equal completions nil))
         (should (equal point-before point-after))))))
 
+(ert-deftest pascal-beg-of-defun ()
+  (with-temp-buffer
+    (pascal-mode)
+    (insert "program test; procedure p(")
+    (forward-char -1)
+    (pascal-beg-of-defun)
+    (should (equal (point) 15))))
+
 (provide 'pascal-tests)
-- 
2.21.1 (Apple Git-122.3)


[-- Attachment #4: Type: text/plain, Size: 2 bytes --]

bug#41740: 26.1; completion-at-point fails in the Pascal mode

[Mattias Engdegård]

22 juni 2020 kl. 00.08 skrev Shinichi Sakata <shinichi.sakata@gmail.com>:
> 
> Thanks Mattias! Yes, the patches have fixed the problem. Thanks again for your help!

Good to hear -- pushed to master.