bug#16907: 24.3.50; [ruby-mode] %w/W literals have incorrect indentation

bug#16907: 24.3.50; [ruby-mode] %w/W literals have incorrect indentation

[Bozhidar Batsov]

Very briefly put, the following code:

x = %w(
  one
  two
  three
)

is indented like this:

x = %w(
one
two
three
)

%w/W should be indented like other array literals.



In GNU Emacs 24.3.50.1 (x86_64-apple-darwin13.0.0, NS apple-appkit-1265.00)
 of 2014-01-27 on Bozhidars-MacBook-Pro.local
Windowing system distributor `Apple', version 10.3.1265
Configured using:
 `configure --prefix=/usr/local/Cellar/emacs/HEAD --without-dbus
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs/HEAD/share/info/emacs
 --without-gnutls --with-ns --disable-ns-self-contained'

Important settings:
  locale-coding-system: utf-8-unix

Major mode: Ruby

Minor modes in effect:
  ruby-tools-mode: t
  inf-ruby-minor-mode: t
  diff-auto-refine-mode: t
  subword-mode: t
  guru-mode: t
  erc-truncate-mode: t
  erc-spelling-mode: t
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-match-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-netsplit-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  which-function-mode: t
  flx-ido-mode: t
  ido-ubiquitous-mode: t
  winner-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  whitespace-mode: t
  global-anzu-mode: t
  anzu-mode: t
  projectile-global-mode: t
  projectile-mode: t
  flyspell-mode: t
  volatile-highlights-mode: t
  global-hl-line-mode: t
  shell-dirtrack-mode: t
  recentf-mode: t
  savehist-mode: t
  show-smartparens-global-mode: t
  show-smartparens-mode: t
  smartparens-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  prelude-global-mode: t
  prelude-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
p d a t e SPC R b x <backspace> <backspace> u b i n 
i o <backspace> u s SPC t r a v i s c i SPC c o n f 
i g <return> <return> S e e SPC C-y <return> f o r 
SPC d e t a i l s . C-c C-c <return> <down-mouse-1> 
<mouse-1> q P P l l q <down-mouse-1> <mouse-1> C-x 
C-f p o w <backspace> <backspace> <backspace> <backspace> 
p o w e r <return> t r a <return> C-x g F F C-x o C-n 
C-n C-n C-n C-n C-e <backspace> <backspace> <backspace> 
<backspace> C-x C-s C-x C-f g e m <return> C-n C-n 
C-n C-n C-n C-k C-k C-k C-k C-k <backspace> <backspace> 
C-x C-s C-x g C-n C-n C-n C-n C-n C-n s s c c M-p M-n 
C-p C-p C-e C-c C-c P P C-x o C-x b <return> C-p C-p 
<return> - SPC 2 . 1 . 0 C-x C-s C-x g C-n s c c R 
u n SPC s p e c s SPC o n SPC R u b y SPC 2 . 1 SPC 
a s SPC w e l l C-c C-c P P l l q <down-mouse-1> <mouse-1> 
C-x b c i d e r C-g s-g C-g s-p c i d e r <return> 
i n t e r a <return> M-< s-g d o c - h a n d l e r 
<return> <help-echo> <down-mouse-2> <mouse-1> <help-echo> 
<down-mouse-2> <mouse-1> <help-echo> C-l <help-echo> 
<down-mouse-1> <mouse-1> C-x b t e s t . <return> C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-a C-k C-k C-k C-k 
C-k C-k C-k C-n C-SPC C-n C-n C-n C-n C-n M-w <help-echo> 
<help-echo> M-x r e p o r t <return>

Recent messages:
Saving file /Users/bozhidar/projects/powerpack/.git/COMMIT_EDITMSG...
Wrote /Users/bozhidar/projects/powerpack/.git/COMMIT_EDITMSG
Git finished
Type a prefix key to toggle it. Run 'actions' with their prefixes. '?' for more help.
Running git push -v origin master:refs/heads/master
Git finished
Type a prefix key to toggle it. Run 'actions' with their prefixes. '?' for more help.
Quit [2 times]
Mark set
Grep finished (matches found)

Load-path shadows:
/Users/bozhidar/.emacs.d/elpa/tabulated-list-20120406.2251/tabulated-list hides /usr/local/Cellar/emacs/HEAD/share/emacs/24.3.50/lisp/emacs-lisp/tabulated-list

Features:
(shadow sort mail-extr emacsbug vc-annotate vc vc-dispatcher nxml-uchnm
rng-xsd xsd-regexp rng-cmpct edebug crm ielm hippie-exp markdown-mode
noutline outline gitignore-mode conf-mode yaml-mode ace-jump-mode ffap
url-parse url-vars ruby-tools inf-ruby ruby-mode-expansions
smartparens-ruby ruby-mode smie misearch multi-isearch jka-compr
eieio-opt speedbar sb-image ezimage dframe mule-util find-dired
magit-key-mode magit view epa derived epg epg-config diff-mode
git-rebase-mode git-commit-mode server log-edit message sendmail rfc822
mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 ietf-drums mailabbrev mail-utils gmm-utils mailheader pcvs-util
add-log executable vc-git lisp-mnt network-stream starttls tls cider
cider-mode cider-repl cider-eldoc clojure-test-mode cider-interaction
arc-mode archive-mode cider-client nrepl-client cider-util ewoc
superword subword clojure-mode-expansions clojure-mode inf-lisp
rainbow-mode color rainbow-delimiters elisp-slime-nav guru-mode
prelude-key-chord key-chord prelude-xml nxml-mode-expansions
html-mode-expansions sgml-mode smartparens-html rng-nxml rng-valid
rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn
nxml-ns nxml-mode nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc
xmltok prelude-web prelude-scss prelude-scheme prelude-ruby prelude-perl
prelude-org prelude-js prelude-erc erc-truncate erc-autoaway
erc-spelling erc-notify erc-log erc-list erc-menu erc-join erc-ring
erc-networks erc-pcomplete erc-track erc-match erc-button erc-fill
erc-stamp erc-netsplit erc-goodies erc erc-backend erc-compat
prelude-emacs-lisp prelude-css prelude-common-lisp slime-autoloads
prelude-clojure prelude-lisp prelude-c prelude-programming flycheck
help-mode rx f which-func imenu prelude-ido smex flx-ido flx
ido-ubiquitous warnings ido prelude-osx exec-path-from-shell
prelude-global-keybindings prelude-editor winner undo-tree diff esh-var
esh-io esh-cmd esh-opt esh-ext esh-proc esh-arg eldoc esh-groups eshell
esh-module esh-mode esh-util re-builder whitespace browse-kill-ring
midnight ediff-merg ediff-wind ediff-diff ediff-mult ediff-help
ediff-init ediff-util ediff dired-x dired anzu projectile pkg-info
find-func grep compile s bookmark pp expand-region text-mode-expansions
er-basic-expansions expand-region-core expand-region-custom flyspell
ispell etags volatile-highlights hl-line windmove tramp-cache tramp-sh
tramp tramp-compat auth-source gnus-util mm-util mail-prsvr
password-cache tramp-loaddefs trampver shell pcomplete comint ansi-color
ring format-spec recentf tree-widget wid-edit savehist saveplace
diminish smartparens-config smartparens autorevert filenotify delsel
prelude-mode easy-mmode edmacro kmacro prelude-core epl advice help-fns
dash thingatpt prelude-ui zenburn-theme prelude-packages finder-inf
ack-and-a-half-autoloads diminish-autoloads erlang-autoloads
flx-ido-autoloads eieio byte-opt bytecomp byte-compile cconv eieio-core
grizzl-autoloads key-chord-autoloads logito-autoloads info easymenu
cl-macs gv move-text-autoloads pkg-info-autoloads puppet-mode-autoloads
rainbow-delimiters-autoloads request-autoloads rubocop-autoloads
volatile-highlights-autoloads yaml-mode-autoloads package cl cl-loaddefs
cl-lib time-date tooltip electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel ns-win tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process cocoa ns
multi-tty emacs)

bug#16907: 24.3.50; [ruby-mode] %w/W literals have incorrect indentation

[Dmitry Gutov]

(This is probably too late for 24.4)

Bozhidar Batsov <bozhidar@batsov.com> writes:

> %w/W should be indented like other array literals.

And %i/I too, I guess.

Looks like this'll have to be done in two parts:

1. syntax-propertize the above cases differently from the rest of the
percent literals (which are treated like strings).

Don't know exactly what syntax to use. The values can contain any
unpaired characters, except those delimiting them. Put syntax "symbol"
on everything inside them, except whitespace?

And still handle interpolations somehow.

2. Tokenize the elements inside the above literals without regard to
characters used in them. `ruby-smie--backward-token' and its counterpart
would have to be able to easily know when it's inside such literal.

Alternatively, still propertize them as "general strings", but replace
`smie-indent-inside-string' in `smie-indent-functions' with a
specialized version that would look at what the string begins with.

The change in syntax highlighting would have to be handled separately.

bug#16907: Fw: bug#16907: 24.3.50; [ruby-mode] %w/W literals have incorrect indentation

[Bozhidar Batsov]

[-- Attachment #1: Type: text/plain, Size: 2024 bytes --]

>  
> From: Bozhidar Batsov <bozhidar@batsov.com>
> To: Dmitry Gutov <dgutov@yandex.ru>
> Date: Saturday, March 1, 2014 at 1:00:52 PM
> Subject: Re: bug#16907: 24.3.50; [ruby-mode] %w/W literals have incorrect indentation
>  
>  
>  
> --  
> Cheers,
> Bozhidar
>  
>  
> On Saturday, March 1, 2014 at 3:55 AM, Dmitry Gutov wrote:
>  
> > (This is probably too late for 24.4)
> >  
> >  
> >  
>  
> Hope it’s not :-) (although it’s not a big deal obviously).  
> >  
> > Bozhidar Batsov <bozhidar@batsov.com (mailto:bozhidar@batsov.com)> writes:
> >  
> > > %w/W should be indented like other array literals.
> >  
> > And %i/I too, I guess.
> Indeed. I forgot about them.  
> >  
> > Looks like this'll have to be done in two parts:
> >  
> > 1. syntax-propertize the above cases differently from the rest of the
> > percent literals (which are treated like strings).
> >  
> >  
> >  
>  
> I was thinking more or less the same.   
> >  
> > Don't know exactly what syntax to use. The values can contain any
> > unpaired characters, except those delimiting them. Put syntax "symbol"
> > on everything inside them, except whitespace?
> >  
> > And still handle interpolations somehow.
> Sounds reasonable. Not sure how hard it would be to implement.  
> >  
> > 2. Tokenize the elements inside the above literals without regard to
> > characters used in them. `ruby-smie--backward-token' and its counterpart
> > would have to be able to easily know when it's inside such literal.
> >  
> > Alternatively, still propertize them as "general strings", but replace
> > `smie-indent-inside-string' in `smie-indent-functions' with a
> > specialized version that would look at what the string begins with.
> >  
> >  
> >  
>  
> Option 1 seems better/cleaner to me, but obviously I’m not as familiar as you with the existing code, so
> it’s ultimately your call.
>   
> >  
> > The change in syntax highlighting would have to be handled separately.  
>  


[-- Attachment #2: Type: text/html, Size: 3953 bytes --]

bug#16907: 24.3.50; [ruby-mode] %w/W literals have incorrect indentation

[Dmitry Gutov]

(Please keep it in Cc).

On 01.03.2014 13:00, Bozhidar Batsov wrote:
>> 2. Tokenize the elements inside the above literals without regard to
>> characters used in them. `ruby-smie--backward-token' and its counterpart
>> would have to be able to easily know when it's inside such literal.
>>
>> Alternatively, still propertize them as "general strings", but replace
>> `smie-indent-inside-string' in `smie-indent-functions' with a
>> specialized version that would look at what the string begins with.
> Option 1 seems better/cleaner to me,

Option 1 is everything before "Alternatively". I'm not sure how to 
implement the item 2 in it.

bug#16907: 24.3.50; [ruby-mode] %w/W literals have incorrect indentation

[Bozhidar Batsov]

[-- Attachment #1: Type: text/plain, Size: 853 bytes --]

On Saturday, March 1, 2014 at 3:57 PM, Dmitry Gutov wrote:
> (Please keep it in Cc).
> 
> 

I know. Sorry about that. 
> 
> On 01.03.2014 13:00, Bozhidar Batsov wrote:
> > > 2. Tokenize the elements inside the above literals without regard to
> > > characters used in them. `ruby-smie--backward-token' and its counterpart
> > > would have to be able to easily know when it's inside such literal.
> > > 
> > > Alternatively, still propertize them as "general strings", but replace
> > > `smie-indent-inside-string' in `smie-indent-functions' with a
> > > specialized version that would look at what the string begins with.
> > > 
> > 
> > Option 1 seems better/cleaner to me,
> > 
> 
> 
> Option 1 is everything before "Alternatively". I'm not sure how to 
> implement the item 2 in it.
> 
> 


Oops. I should have read your comments more carefully.   


[-- Attachment #2: Type: text/html, Size: 1656 bytes --]