Re: A vision for multiple major modes [was: Re: [Emacs-diffs] widen-limits c331b66:]

[Dmitry Gutov <dgutov@yandex.ru>]

On 03/29/2016 03:07 AM, Alan Mackenzie wrote:

>> E.g. (goto-char (match-end n)) is a common
>> idiom for syntax highlighting and indentation code.
>
> Yes, but that match is going to be in the same island, or at the very
> least, in the chain of islands containing the current one.

Would looking-at skip over intermediate islands? Basically, I wonder if

  (= (+ (match-beginning 0) (length (match-string 0))
     (match-end 0))

is going to always hold in that new world.

> Are you thinking more that `forward-char' should move from the end of an
> island to the beginning of the next island in a chain?

As one option, yes. Let's call it option A, and it entails renumerating 
buffer positions to avoid gaps.

>> True. I'm more worried about gaps. But they could be treated like
>> whitespace, I suppose.
>
> For example, by `forward-char'?  What primitives were you thinking of,
> here?

This would be option B. forward-char doesn't care if the characters are 
whitespace or not. I don't know if e.g. search-forward would see the 
contents of the intermediate islands as a bunch of actual space 
characters (it's something to consider).

Most importantly, the contents of intermediate islands would match 
"\\s-", and parse-partial-sexp would similarly skip over them.

>> Right. So then, some yet-unknown body of code has to become
>> island-aware, and the improvement is not that seamless anymore.
>
> The way I see it, the super modes would have to be totally aware of the
> island mechanism, but the major modes would be largely, it not totally,
> unaware of it.  `syntax-ppss' seems more part of the super mode
> mechanism.

At the very least, that does sound somewhat complicated.

>> What if var has an island-local binding? And my function has this:
>
>> (setq var 1)
>> (goto-char xx) ; where xx is in a different island
>
>> will its value change mid-program? What if xx is in the same island?
>> Will the value change then?
>
> A different island local binding would become current, so yes its value
> would thereby change.  Much like if var had a buffer local binding, and
> you do (set-buffer "foo"), the value of var you'd just set would no
> longer be in the current binding.  if xx is in the same island, the
> binding just setq'd would remain the same.

That might be something to look out for. Changing a buffer is an 
operation with big consequences, we know that already, but using 
goto-char didn't have too many implications until now (aside from 
point-entered hooks, I guess, which are being phased out).

>> I don't think either should be true. Then, the "adequate" model would
>> amount to changing them in post-command-hook anyway.
>
> What sort of variables are you thinking about here?  [ Some time later:]
> Could it be that it might be better to have "island chain local"
> variables rather than merely "island local" ones?

mmm-mode has both kinds.

> So that the three ruby
> lines in your example below would be three islands in a chain, and would
> thus all share a set of "island chain local" bindings rather than each
> line having its own binding?

Yes, probably. But that doesn't solve the above concern.

>> So making it seem, to the Lisp code, like the multiple related islands
>> don't have anything (for some definition of "anything") between them is
>> not a priority?
>
> By "related", I think you mean the same thing I meaan by "chained" - the
> three ruby mode lines enclosed in <%...%> in your example below, would be
> chained together by the super mode.

Yes, we can call them chained. As long as it doesn't imply anything 
about any presence of islands of other types between them.

>> Here's an example of ERB code Vitalie brought up recently:
>
>> <% if foo %>
>>    <%= bar(boo) %>
>> <% end %>
>
>> The parts of the buffer between %'s would be Ruby islands. The
>> suggestion was to use the value returned by the indentation function in
>> the second island (second line) to indent the "real" buffer contents.
>
> The question seems to be, are we indenting in the super mode or in each
> island?  The answer seems to be you'd want to indent in the super mode
> here, I think.

Indentation in the super mode would look like this:

   <% if foo %>
   <%= bar(boo) %>
   <% end %>

which is not what we'd really want. But yes, a good solution would take 
html-mode's indentation logic as a base, and modify it with the 
indentation offsets provided by the islands.

>> But: neither of the islands contains a newline. If they are combined in
>> the most straightforward way, the Ruby line will be 'if foo bar(boo)',
>> and its indentation would be not what we're looking for. I think the
>> current ways to look for a newline,
>
>>    (progn (skip-syntax-backward " ")
>>           (eq (char-before ?\n))
>
>> or
>
>>    (looking-at "\\s-$")
>
>> will fail to work. Also note that neither of the two islands in this
>> example contains the newline in question.
>
> Two?  Don't you mean three, here?  Is this a minor mistake in your
> counting or am I missing something important because I don't grok ruby?

The mistake would be mine, but really, I'm just talking about the first 
two. We're discussing the indentation of the second line. The third one 
is just here to make the example look more complete.

>> The "not chained" idea would require some explanation.
>
> For example, islands in a shell script which were individual AWK scripts
> would not be chained together.  But the three (?two) lines of ruby in the
> example above would be chained together.  I think we need to work this
> out more precisely.

Makes sense.

> For example, in the three ruby line scenario above, is there any variable
> you can think of for which it would be advantageous to have a binding
> for each island rather than a shared binding for the set of three (?two)?

Not off the top of my head. Maybe there's none.

> I'm kind of envisioning successive `forward-char's moving from the end
> of "<% if foo %>" to the beginning of "<%= bar(boo) %>", or even from
> the end of "if foo" to the beginning of "bar(boo)".  How does this sound?

Sounds like what I've calling option A above. Basically, my vague 
concerns are:

- Performance (checking islands boundaries in each primitive must incur 
overhead, even when there are no islands).
- Code complexity: new code paths that might be exercised not very often 
in the future. Hence, they could be prone to breakage. A dedicated test 
suite would help with that, though.

Also, think back to the problem of the absence of newlines in the Ruby 
islands in the above example. The indentation engine needs them, and it 
might be harder to make newlines appear both inside and outside of the 
Ruby islands (the html-mode indentation code needs them too, I'd wager). 
This is where option B has another advantage, aside from (probably) 
being easier to implement.