bug#64069: 30.0.50; Mistyped shy group regexps

bug#64069: 30.0.50; Mistyped shy group regexps

[Basil Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1655 bytes --]

Severity: minor
Tags: patch

Further to https://bugs.gnu.org/64019#14, I attach a patch which tweaks
four redundant or seemingly mistyped instances of \(:?...\), i.e. a
numbered group starting with an optional colon in place of the likelier
shy group.

CCing:
- Alan to review the change to c-or-c++-mode--regexp
  (along with its clone c-ts-mode--c-or-c++-regexp)
- Dmitry to review the change in vc-git
- Paul for any comments on the time handling in vc-git-annotate-time

In vc-git-annotate-time, the mistyped group added in [1] throws off the
match string indices that are later passed to encode-time: by my reading
the hour argument by chance continues to be specified correctly, but the
minutes argument receives the number of hours, the seconds argument the
number of minutes, and the timezone argument the number of seconds.

[1]: Display shorter dates in Git annotate output
576fba5f58d 2015-05-17 02:47:17 +0300
https://git.sv.gnu.org/cgit/emacs.git/commit/?id=576fba5f58d

Even after correcting the shy group, the timezone parsing doesn't seem
right to me: encode-time expects a UTC offset in seconds, but is passed
e.g. (string-to-number "+0100")=100s instead of the expected 1hr=3600s.

IMO there is another minor issue with the regexp towards its end:

  \([-+0-9]+\) *[0-9]+

The trailing line number should be separated from the preceding
timestamp by at least one space.  So, the optional space would ideally
precede rather than follow the optional time+zone components.

The patch fixes this and the match/timezone handling by splitting the
datetime string into only three components (date, time, zone) and
delegating to iso8601-parse.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-some-shy-group-regexps.patch --]
[-- Type: text/x-diff, Size: 5726 bytes --]

From 9ead0ba1dd2676105269ff1472a206a250539b18 Mon Sep 17 00:00:00 2001
From: "Basil L. Contovounesios" <contovob@tcd.ie>
Date: Mon, 12 Jun 2023 00:23:56 +0100
Subject: [PATCH] Fix some shy group regexps

These issues were caught by modified versions of the GNU ELPA
packages xr and relint:
- https://github.com/mattiase/xr/pull/6
- https://github.com/mattiase/relint/pull/14

* lisp/gnus-art.el (gnus-parse-news-url): Remove redundant numbered
group and calls to match-string.

* lisp/progmodes/c-ts-mode.el (c-ts-mode--c-or-c++-regexp):
* lisp/progmodes/cc-mode.el (c-or-c++-mode--regexp): Fix shy group
mistyped as optional colon (bug#64019#29).

* lisp/vc/vc-git.el (vc-git-annotate-time): Ditto.  Also fix
timezone parsing by using iso8601-parse.
* test/lisp/vc/vc-git-tests.el (vc-git-test-annotate-time): New
test.
---
 lisp/gnus/gnus-art.el        |  7 +++----
 lisp/progmodes/c-ts-mode.el  |  2 +-
 lisp/progmodes/cc-mode.el    |  2 +-
 lisp/vc/vc-git.el            | 19 ++++++++++++-------
 test/lisp/vc/vc-git-tests.el | 17 +++++++++++++++++
 5 files changed, 34 insertions(+), 13 deletions(-)

diff --git a/lisp/gnus/gnus-art.el b/lisp/gnus/gnus-art.el
index 6a7a3f41746..6f201f9c3df 100644
--- a/lisp/gnus/gnus-art.el
+++ b/lisp/gnus/gnus-art.el
@@ -8331,11 +8331,10 @@ gnus-parse-news-url
       (when (looking-at "\\([A-Za-z]+\\):")
 	(setq scheme (match-string 1))
 	(goto-char (match-end 0)))
-      (when (looking-at "//\\([^:/]+\\)\\(:?\\)\\([0-9]+\\)?/")
+      (when (looking-at "//\\([^:/]+\\):?\\([0-9]+\\)?/")
 	(setq server (match-string 1))
-	(setq port (if (stringp (match-string 3))
-		       (string-to-number (match-string 3))
-		     (match-string 3)))
+        (setq port (and (match-beginning 2)
+                        (string-to-number (match-string 2))))
 	(goto-char (match-end 0)))
 
       (cond
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 4b83c6733a2..9e911192be3 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -1355,7 +1355,7 @@ c-ts-mode--c-or-c++-regexp
               "\\|" id "::"
               "\\|" id ws-maybe "=\\)"
               "\\|" "\\(?:inline" ws "\\)?namespace"
-              "\\(:?" ws "\\(?:" id "::\\)*" id "\\)?" ws-maybe "{"
+              "\\(?:" ws "\\(?:" id "::\\)*" id "\\)?" ws-maybe "{"
               "\\|" "class"     ws id
               "\\(?:" ws "final" "\\)?" ws-maybe "[:{;\n]"
               "\\|" "struct"     ws id "\\(?:" ws "final" ws-maybe "[:{\n]"
diff --git a/lisp/progmodes/cc-mode.el b/lisp/progmodes/cc-mode.el
index 11a1d3fe6c2..5cf9b7e17f8 100644
--- a/lisp/progmodes/cc-mode.el
+++ b/lisp/progmodes/cc-mode.el
@@ -2859,7 +2859,7 @@ c-or-c++-mode--regexp
                                      "\\|" id "::"
                                      "\\|" id ws-maybe "=\\)"
               "\\|" "\\(?:inline" ws "\\)?namespace"
-                    "\\(:?" ws "\\(?:" id "::\\)*" id "\\)?" ws-maybe "{"
+                    "\\(?:" ws "\\(?:" id "::\\)*" id "\\)?" ws-maybe "{"
               "\\|" "class"     ws id
                     "\\(?:" ws "final" "\\)?" ws-maybe "[:{;\n]"
               "\\|" "struct"     ws id "\\(?:" ws "final" ws-maybe "[:{\n]"
diff --git a/lisp/vc/vc-git.el b/lisp/vc/vc-git.el
index a3469b71386..dfca944dc74 100644
--- a/lisp/vc/vc-git.el
+++ b/lisp/vc/vc-git.el
@@ -1723,14 +1723,19 @@ vc-git-annotate-command
 
 (declare-function vc-annotate-convert-time "vc-annotate" (&optional time))
 
+(autoload 'decoded-time-set-defaults "time-date")
+(autoload 'iso8601-parse "iso8601")
+
 (defun vc-git-annotate-time ()
-  (and (re-search-forward "^[0-9a-f^]+[^()]+(.*?\\([0-9]+\\)-\\([0-9]+\\)-\\([0-9]+\\) \\(:?\\([0-9]+\\):\\([0-9]+\\):\\([0-9]+\\) \\([-+0-9]+\\)\\)? *[0-9]+) " nil t)
-       (vc-annotate-convert-time
-        (apply #'encode-time (mapcar (lambda (match)
-                                       (if (match-beginning match)
-                                           (string-to-number (match-string match))
-                                         0))
-                                     '(6 5 4 3 2 1 7))))))
+  (and (re-search-forward "^[0-9a-f^]+[^()]+(.*?\\([0-9]+-[0-9]+-[0-9]+\\)\\(?: \\([0-9]+:[0-9]+:[0-9]+\\) \\([-+0-9]+\\)\\)? +[0-9]+) " nil t)
+       (let* ((dt (match-string 1))
+              (dt (if (not (match-beginning 2)) dt
+                    ;; Format as ISO 8601.
+                    (concat dt "T" (match-string 2) (match-string 3))))
+              (decoded (ignore-errors (iso8601-parse dt))))
+         (and decoded
+              (vc-annotate-convert-time
+               (encode-time (decoded-time-set-defaults decoded)))))))
 
 (defun vc-git-annotate-extract-revision-at-line ()
   (save-excursion
diff --git a/test/lisp/vc/vc-git-tests.el b/test/lisp/vc/vc-git-tests.el
index f12c5d3434b..b331b77cf01 100644
--- a/test/lisp/vc/vc-git-tests.el
+++ b/test/lisp/vc/vc-git-tests.el
@@ -64,4 +64,21 @@ vc-git-test--run-program-version-test
              (actual-output (vc-git--program-version)))
     (should (equal actual-output expected-output))))
 
+(ert-deftest vc-git-test-annotate-time ()
+  "Test `vc-git-annotate-time'."
+  (require 'vc-annotate)
+  (with-temp-buffer
+    (insert "\
+00000000 (Foo Bar 2023-06-14  1) a
+00000001 (Foo Bar 2023-06-14 00:00:00 -0130  2) b
+00000002 (Foo Bar 2023-06-14 00:00:00 +0145  3) c
+00000003 (Foo Bar 2023-06-14 00:00:00  4) d
+00000004 (Foo Bar 0-0-0  5) \n")
+    (goto-char (point-min))
+    (should (floatp (vc-git-annotate-time)))
+    (should (> (vc-git-annotate-time)
+               (vc-git-annotate-time)))
+    (should-not (vc-git-annotate-time))
+    (should-not (vc-git-annotate-time))))
+
 ;;; vc-git-tests.el ends here
-- 
2.34.1


[-- Attachment #3: Type: text/plain, Size: 3265 bytes --]


WDYT?

Thanks,

-- 
Basil

In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0, Xaw3d scroll bars) of 2023-06-13 built on blc
Repository revision: 81932ebcfa56a33fcb1c7d9f91094e2b1f6e9b77
Repository branch: blc/treesit/master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Ubuntu 22.04.2 LTS

Configured using:
 'configure CC=gcc-12 'CFLAGS=-Og -ggdb3' --prefix=/home/bic/.local
 --with-file-notification=yes --with-x --with-x-toolkit=lucid'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XAW3D XDBE XIM XINPUT2 XPM
LUCID ZLIB

Important settings:
  value of $LC_MONETARY: en_IE.UTF-8
  value of $LC_NUMERIC: en_IE.UTF-8
  value of $LC_TIME: en_IE.UTF-8
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
xinput2 x multi-tty make-network-process emacs)

Memory information:
((conses 16 36749 9186)
 (symbols 48 5178 0)
 (strings 32 13895 1202)
 (string-bytes 1 379735)
 (vectors 16 9299)
 (vector-slots 8 148629 8492)
 (floats 8 23 25)
 (intervals 56 244 0)
 (buffers 984 10))

bug#64069: 30.0.50; Mistyped shy group regexps

[Dmitry Gutov]

Hi Basil,

On 14/06/2023 19:43, Basil Contovounesios wrote:
> - Dmitry to review the change in vc-git
> - Paul for any comments on the time handling in vc-git-annotate-time
> 
> In vc-git-annotate-time, the mistyped group added in [1] throws off the
> match string indices that are later passed to encode-time: by my reading
> the hour argument by chance continues to be specified correctly, but the
> minutes argument receives the number of hours, the seconds argument the
> number of minutes, and the timezone argument the number of seconds.
> 
> [1]: Display shorter dates in Git annotate output
> 576fba5f58d 2015-05-17 02:47:17 +0300
> https://git.sv.gnu.org/cgit/emacs.git/commit/?id=576fba5f58d
> 
> Even after correcting the shy group, the timezone parsing doesn't seem
> right to me: encode-time expects a UTC offset in seconds, but is passed
> e.g. (string-to-number "+0100")=100s instead of the expected 1hr=3600s.

Thank you for the effort, but I think most of this nuance could be 
simplified away.

When you say "encode-time ... is passed ... +0100", what is your testing 
scenario?

IIUC, the change in commit 576fba5f58d removed the complex dates from 
the output (which we parse), replacing them with the simple yyyy-mm-dd 
format (that's what --date=short does). Seems like I tried (8 years ago) 
to retain the compatibility with the previous output in case we'll make 
the format configurable someday, but that still hasn't happened. So we 
could do away with the 'if' condition and simplify the regexp accordingly.

bug#64069: 30.0.50; Mistyped shy group regexps

[Basil Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov [2023-06-15 04:45 +0300] wrote:

> When you say "encode-time ... is passed ... +0100", what is your testing
> scenario?

Eyeballing the code and git-blame's output.

> IIUC, the change in commit 576fba5f58d removed the complex dates from the output
> (which we parse), replacing them with the simple yyyy-mm-dd format (that's what
> --date=short does). Seems like I tried (8 years ago) to retain the compatibility
> with the previous output in case we'll make the format configurable someday, but
> that still hasn't happened.

Is it not configurable via vc-git-annotate-switches?
When invoking git with multiple --date= options, the last one wins.

> So we could do away with the 'if' condition and simplify the regexp
> accordingly.

You mean ignoring anything other than the YYYY-MM-DD format?

No objections from me, but it's not that hard to fix&keep support for
the default --date=iso output from Git, if desired.

Thanks,

-- 
Basil

bug#64069: 30.0.50; Mistyped shy group regexps

[Dmitry Gutov]

On 15/06/2023 10:39, Basil Contovounesios wrote:
>> IIUC, the change in commit 576fba5f58d removed the complex dates from the output
>> (which we parse), replacing them with the simple yyyy-mm-dd format (that's what
>> --date=short does). Seems like I tried (8 years ago) to retain the compatibility
>> with the previous output in case we'll make the format configurable someday, but
>> that still hasn't happened.
> Is it not configurable via vc-git-annotate-switches?
> When invoking git with multiple --date= options, the last one wins.

All right. I'm not sure if people took advantage of this capability, but 
if you want to keep supporting it, that is fine by me, too.

Thanks.

bug#64069: 30.0.50; Mistyped shy group regexps

[Alan Mackenzie]

Hello, Basil.

On Wed, Jun 14, 2023 at 17:43:39 +0100, Basil Contovounesios wrote:
> Severity: minor
> Tags: patch

> Further to https://bugs.gnu.org/64019#14, I attach a patch which tweaks
> four redundant or seemingly mistyped instances of \(:?...\), i.e. a
> numbered group starting with an optional colon in place of the likelier
> shy group.

> CCing:
> - Alan to review the change to c-or-c++-mode--regexp

Yes, that looks like a bug, just as you've surmised.

There are also three other occurrences of \(:? in CC Mode, namely in
cc-langs.el.

I will fix all of these in master.

[ .... ]

> diff --git a/lisp/progmodes/cc-mode.el b/lisp/progmodes/cc-mode.el
> index 11a1d3fe6c2..5cf9b7e17f8 100644
> --- a/lisp/progmodes/cc-mode.el
> +++ b/lisp/progmodes/cc-mode.el
> @@ -2859,7 +2859,7 @@ c-or-c++-mode--regexp
>                                       "\\|" id "::"
>                                       "\\|" id ws-maybe "=\\)"
>                "\\|" "\\(?:inline" ws "\\)?namespace"
> -                    "\\(:?" ws "\\(?:" id "::\\)*" id "\\)?" ws-maybe "{"
> +                    "\\(?:" ws "\\(?:" id "::\\)*" id "\\)?" ws-maybe "{"
>                "\\|" "class"     ws id
>                      "\\(?:" ws "final" "\\)?" ws-maybe "[:{;\n]"
>                "\\|" "struct"     ws id "\\(?:" ws "final" ws-maybe "[:{\n]"

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64069: 30.0.50; Mistyped shy group regexps

[Alan Mackenzie]

Hello, Basil.

On Thu, Jun 15, 2023 at 20:58:38 +0000, Alan Mackenzie wrote:
> On Wed, Jun 14, 2023 at 17:43:39 +0100, Basil Contovounesios wrote:
> > Severity: minor
> > Tags: patch

> > Further to https://bugs.gnu.org/64019#14, I attach a patch which tweaks
> > four redundant or seemingly mistyped instances of \(:?...\), i.e. a
> > numbered group starting with an optional colon in place of the likelier
> > shy group.

> > CCing:
> > - Alan to review the change to c-or-c++-mode--regexp

> Yes, that looks like a bug, just as you've surmised.

> There are also three other occurrences of \(:? in CC Mode, namely in
> cc-langs.el.

Actually, there were four.

> I will fix all of these in master.

DONE.

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64069: 30.0.50; Mistyped shy group regexps

[Basil Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

close 64069 30.1
quit

Alan Mackenzie [2023-06-17 13:26 +0000] wrote:
> On Thu, Jun 15, 2023 at 20:58:38 +0000, Alan Mackenzie wrote:
>> There are also three other occurrences of \(:? in CC Mode, namely in
>> cc-langs.el.
>
> Actually, there were four.
>
>> I will fix all of these in master.
>
> DONE.

Thanks.  I installed the rest of the patch, and am closing this report.

Fix more shy group regexps
fef27d28fa7 2023-06-17 16:36:27 +0100
https://git.sv.gnu.org/cgit/emacs.git/commit/?id=fef27d28fa7

-- 
Basil