bug#67569: 29.1; ruby-mode syntax highlighting breaks with variable named "index" and "/" operator

[Dmitry Gutov <dmitry@gutov.dev>]

[-- Attachment #1: Type: text/plain, Size: 2206 bytes --]

On 01/12/2023 16:27, Michael Hoy wrote:
> In the standard ruby-mode, a variable named "index" followed by a "/"
> causes the syntax highlighting to break -- it appears that after the
> slash everything is interpreted as part of a regular expression until
> another "/" appears.
> 
> You can reproduce this fairly easily. using 'emacs -Q', go to the
> scratch buffer and clear it. Run 'M-x ruby-mode'. Enter something like
> the following:
> 
> def foo
>    index = 5
>    x = index / 3
>    puts "#{x}"
> end
> 
> Note that starting after the "/" until the end, syntax highlighting and
> indentation break (everything, in my color scheme, is purple). If you
> add a "/" for instance to the "puts" line, syntax highlighting and
> indentation will be restored after that.

Thanks for filing this. The reasons is our regexp detection mechanism, 
with a whitelist for methods that call accept regexps (of which 'index' 
is one), and could be called without parens.

Anyway, it looks like Ruby actually looks at whether there is a space 
after "/", and if so, interprets this as division (when there is nothing 
else to disambiguate, such as parens around the call). So we can follow 
suit, the attached patch seems to fix it.

Also, perhaps you'll want to try ruby-ts-mode which is in Emacs 29.1, 
and which uses a "real" (tree-sitter) grammar. Indentation-wise, it 
should behave more or less the same as ruby-mode.

Eli, should we put this in Emacs 29.2? It's not a regression (a fairly 
old problem), but the fix looks simple enough.

> There is a report here about the bug:
> https://emacs.stackexchange.com/questions/23802/division-sometimes-breaks-syntax-highlighting-in-ruby-mode
> The author says in the comments they used "M-x report-emacs-bug", but I
> couldn't find it (and the comments suggest that it may not have been
> sent correctly). Apologies if this is already reported!

I was hoping the author would follow through with investigating why the 
bug did not reach the tracker, but that didn't happen. I've seen this 
kind of feedback a few times, but not lately. Maybe whatever the issue 
was, got resolved in 2016. Or people still send reports into nowhere, 
and we just don't hear about it.

[-- Attachment #2: ruby-index-slash-no-regexp.diff --]
[-- Type: text/x-patch, Size: 1658 bytes --]

diff --git a/lisp/progmodes/ruby-mode.el b/lisp/progmodes/ruby-mode.el
index 5c34ddc562b..b2d833fa2b0 100644
--- a/lisp/progmodes/ruby-mode.el
+++ b/lisp/progmodes/ruby-mode.el
@@ -2125,7 +2125,7 @@ ruby-find-library-file
                    "or" "not" "&&" "||"))
      ;; Method name from the list.
      "\\|\\_<"
-     (regexp-opt ruby-syntax-methods-before-regexp)
+     (regexp-opt ruby-syntax-methods-before-regexp t)
      "\\)\\s *")
     "Regexp to match text that can be followed by a regular expression."))
 
@@ -2183,14 +2183,18 @@ ruby-syntax-propertize
         (when (save-excursion
                 (forward-char -1)
                 (cl-evenp (skip-chars-backward "\\\\")))
-          (let ((state (save-excursion (syntax-ppss (match-beginning 1)))))
+          (let ((state (save-excursion (syntax-ppss (match-beginning 1))))
+                space-after-slash)
             (when (or
                    ;; Beginning of a regexp.
                    (and (null (nth 8 state))
                         (save-excursion
+                          (setq space-after-slash (eql (char-after) ?\s))
                           (forward-char -1)
                           (looking-back ruby-syntax-before-regexp-re
-                                        (line-beginning-position))))
+                                        (line-beginning-position)))
+                        (or (not space-after-slash)
+                            (not (match-beginning 2))))
                    ;; End of regexp.  We don't match the whole
                    ;; regexp at once because it can have
                    ;; string interpolation inside, or span