Re: A vision for multiple major modes [was: Re: [Emacs-diffs] widen-limits c331b66:]

[Dmitry Gutov <dgutov@yandex.ru>]

On 03/24/2016 08:38 PM, Alan Mackenzie wrote:

>> Each multi-mode package implements something like this already. It
>> doesn't work well e.g. because font-lock rules of each particular
>> language, indentation code, etc, are free to widen.
>
> With islands it should work well, regardless of whether any major mode
> widens or narrows.  That's because primitives (such as regexp search)
> would constrain themselves to the island.

It might work well. Without so much as a proof of concept, there's no 
way to tell, really.

>>> For this we will need a new type of local variable, an "island-local" or
>>> "span-local" variable, or whatever you want to call it.  Values of these
>>> variables will vary according to where point is.
>
>> That part is already doable (and done), for most practical purposes.
>
> Except you said above that it doesn't work very well.

Submode-local, and chunk-local, variables work fine. But you don't get 
functioning facilities just by having variables.

>> You'll have to present the total list of facilities, decide how the
>> islands would be applied, and other issues will likely come up from
>> unexpected places.
>
> That will require more investigation.  But syntax based searching and
> regexp based searching will certainly be amongst them.

What about looking-at? char-after? syntax-after? forward-char? 
goto-char? Speaking of the last one, will the buffer positions, as 
visible to the submode code, be renumerated, or will they have gaps?

The above are just some prominent functions that are frequently used in 
indentation code.

> The island-local variables would stay with the island, so that when
> somebody inserts or removes text the right thing would be done.  If
> somebody deletes the island, those variables would disappear (just as
> buffer local ones do when a buffer is deleted).

Depending on your answer to the previous question, even simply inserting 
text inside one of the preceding islands might make syntax-ppss-cache 
out of date in the current island.

>> On the one hand, we'd probably want to retain some variables, in order
>> not to rerun the major mode functions over and over again. On the
>> other hand, if we were to put e.g.  syntax-ppss-last into an
>> island-local variable (and it's a logical continuation of this idea),
>> after island boundaries change it should what... become unbound? Nil?
>
> That's for the application to decide.  The island local binding would
> countinue to exist for as long as the island exists, and the application
> would be free to use it.

Something would have to create and maintain the island identities, as 
users add and remove text, including island delimiters, it's not like 
Emacs could do that automagically. My point is, implementing significant 
part in Elisp is unavoidable anyway.

>> Next, at which points exactly would Emacs look at the island boundaries
>> and change the island-local variables to the values set in the current
>> island? Probably not after each point movement. In post-command-hook?
>> That's also already done.
>
> It wouldn't use any high level facility like post-command-hook.  The
> mechanism would be more like a buffer local variable, entirely handled
> by the C level, so that such a binding would simply exist.

You didn't answer my question, though. When will the bindings apply?

>>> Although the above vision implies a lot of development work, there is
>>> nothing there which is beyond our abilities to implement readily.  It
>>> would give us a true multi major mode capability, yet the impact on
>>> individual major modes would be minimal.
>
> I'm worried by this.  Already narrowing/widening which is currently
> simple, straightforward, and rigorously defined, is looking like
> becoming complicated.  Major modes are going to be constrained in what
> they can do.

A lot of code is already constrained. That's why we most often use 
(point-min) instead of 1. The only unconstrained code we currently have 
is the one that starts with (widen).

> I get the impression that major modes are going to be
> restricted in how they can use parse-partial-sexp.  These are serious
> matters, but I haven't seen any widespread debate on emacs-devel about
> them.

Most parse-partial-sexp uses don't call `widen' in advance.

>> On the other hand, using narrowing for multi-mode purpose is a familiar
>> ground already, and the changes in Emacs core required to do so are
>> minimal. And most of the code written for Emacs has been taught to
>> respect narrowing bounds (even if only by the virtue of always using
>> (point-min) instead of 1), so we can utilize that.
>
> What is "(narrow-to-region 1 (point-max))" going to become?  It seems
> there will be a need for "the bounds of the island", which will impose
> complexities in major mode code.

Replacing '1' with (point-min) everywhere sounds like a minor change all 
modes can bear.

I don't know if we'd need the island-beginnig/island-end accessors in 
your proposal.

> How will this be done when narrowing is the tool used?  It you narrow to
> an island, none of the other islands in a chain are visible.  If you
> narrow to the smallest region which spans those islands, you leave a lot
> of stuff visible which shouldn't be.

a) You can assign whitespace syntax to that stuff (which should cover 
many use cases).

b) You can use a temporary buffer.

>> Doing that using an islands framework limits it to a
>> predefined set of semantics (e.g. all Ruby islands see all other Ruby
>> islands).
>
> I don't see why.  There's no reason why islands couldn't be linked to
> and unlinked from eachother freely at runtime.

Very well.

>> That's not to say that being able to make parse-partial-sexp to skip
>> over certain intervals wouldn't be valuable. But you can do that, sort
>> of, already, by applying existing text properties to those intervals
>> (like beginning-of-comment/end-of-comment, or just "whitespace" over the
>> whole of it), and then removing them at the end of an operation.
>
> Not really.  If you "whitespace" a region out, you've got to remember
> the text properties that were there originally, and restore them
> afterwards.  There's no easy way to do that.

I think it's doable using char-property-alias-alist. The multi-mode will 
define its own property as an alias of `syntax-table', and then put it 
on and remove it from text at will.

> The comment-fence text
> properties won't work either, because, in the general case, there will
> already be comment-fence properties in your region which will foul
> things up.

Yeah, that's a plausible problem. So at least your proposal could pivot 
into something like new kind of high-priority, generic comment fences. 
Or maybe just being able to assign priorities to the "generic comment" 
syntax-table values.

> I'm worried that the multi-mode projects will suborn these
> text properties, making them unavailable for major modes to use.

See the "alias" suggestion.

>> But the end benefits might not be high enough to justify the necessary
>> work and the increase in complexity in internals.
>
> They might not.  They might.  Basically, nobody else really seams
> interested in my idea, so it doesn't look like it will happen.

It sounds somewhat attractive to me, but we should understand more 
clearly the goals we'll reach with it, as well as the semantics of the 
new feature.

In order to deliver on the promise of seamless-ness, I think the islands 
should be able to believe that every one of them starts with position 1, 
and that they all have no gaps. I.e. every primitive would have to 
behave that way, at least as long as the relevant global variable is 
non-nil.