Re: [Emacs-diffs] scratch/widen-less a4ba846: Replace prog-widen with consolidating widen calls

[Dmitry Gutov <dgutov@yandex.ru>]

On 12/3/17 3:28 PM, Eli Zaretskii wrote:

> Because every development environment should first support its own
> development well.  If it doesn't do that, it's incomplete.

OK, I see. But!

This feature is *not* targeted at Emacs development.

Please don't make me fly out to Scotland and chase Alan down with a 
broomstick to make this development come through.

> Indeed; and C-like languages are quite popular among them.

Not in comparison to JS, CSS and HTML.

> How will we look if we support JS and Ruby embedded in HTML, but not C
> code embedded in Yacc grammars nor Awk snippets embedded in shell
> scripts?  Both are quite frequent out there.  We will be laughed at.

We'll be fine. And the actual support is still going to reside in 
third-party packages for now. This change just makes them possible (or 
their codebases saner, at least).

Except for antlr-mode. It will be able to use it right away (the version 
inside Emacs, at least).

> Convince Alan to do what?

To adhere to the current proposal (avoid widening in 
indent-line-function and font-lock-keywords, to start with).

> Do we even understand well enough what are
> the problems in CC Mode that get in the way?

Who do you think is most qualified to study that issue? We'd probably 
have to convince Alan to do that as well, unfortunately.

I have a rough understanding of the issue, but since I haven't reached a 
working state, I don't know how many pitfalls there are left.

> About the only thing
> spelled out in this discussion was the need not to widen.  Personally,
> I think this is not a grave issue, just some technical problem that is
> certainly solvable (you proposed one such solution).

Indeed.

> But is that all?

I imagine the process itself might be trickier than expected. Various 
primitives use caches that save context information. What is such 
primitive to do if the cache contains "beginning of nesting" outside of 
the current restriction, and the logic of said primitive says "go to the 
beginning of the current function and do such and such"? The answer 
isn't obvious to me.

But it's going to need to be answered anyway, no matter which MMM 
approach we choose in the end, I think.

> You mentioned some other problems, but never described them.  What are
> they?  Are they grave enough to prevent your proposal from ever
> supporting CC Mode, even if your proposal is amended?  I don't know.
> Does anyone know?  If so, would they please speak up?

Yes, please.

> I also don't understand well the essence of your proposal.  Yes, I've
> (tried to) read the discussions you pointed to, but what I'm looking
> for is buried there under many layers of secondary stuff. 

The proposal itself is very small, and there's not much to explain. Just 
look at the changes in the manual, in this branch.

It simply facilitates what mmm-mode (and other modes, including 
mhtml-mode) has been doing for years, with varying success. And does 
that in a way that requires minimal changes from major modes (except for 
CC Mode, it seems), and would let us easily change our mind later.

> Why do you
> want to abandon the prog-widen stuff, which you supported just 2 years
> ago?

I never actually supported it, just stopped arguing because I didn't 
have a good alternative idea. Now I do, I think. And there is some 
agreement from Stefan.

> How do you address the issues which prog-indentation-context did
> (e.g., if the embedded chunk of code is incomplete, and perhaps even
> syntactically invalid)?

prog-indentation-context never addressed that issue, and we don't 
either. The major mode code in such situation will either have to 
swallow errors, or, well, they won't work. Or a MMM package will have to 
do error-swallowing, that's also possible.

In practice, a lot of such situations don't actually manifest, though.

> All these questions and considerations need
> to be understood, because you are explicitly proposing an incomplete
> solution, and asking us to agree with its deficiencies, but without
> providing a clear picture of what are we going to give up on.

That's the thing: we're not giving up much (I'd argue we're giving up 
nothing, but Alan obviously disagrees).

Consolidating the 'widen' calls is simply good engineering, even aside 
from making life easier for MMM packages: it's much better to do that in 
several well-defined places, instead of having every helper function do 
it as well, like python-mode CC Mode do.

> And
> last, but not least, what is different in your proposal that will not
> cause it to end up like all the previous ones: either committed to
> Emacs and basically unused, or, worse, collecting dust in long
> forgotten patches or branches?

Like mentioned, the narrowing-based approach is already used in mmm-mode 
and mhtml-mode (and polymode too, I think).

And it works fine in many situations, because many major modes already 
never call widen in their indentation functions (SMIE-based modes, Ruby, 
HTML and JS don't). So this proposal is, to a large extent, codifying an 
existing practice. And at the same time makes the aforementioned modes 
work *better* with interactive narrowing.

> We desperately need these details spelled out to make the discussion
> of this to become technical again, not emotional.

Please feel free to ask any further questions.

> You
> are still debating its design and implementation, even if we disregard
> the CC Mode issue.

Not really. There's a minor issue of whether to make prog-first-column a 
variable, or a hook right away, but the importance of that choice isn't big.

> That code is in Emacs for more than 2 years.  It was admitted with
> Stefan's full support, and at least ANTLR needs it in conjunction with
> Python.

Transitioning from prog-indentation-context to the new approach is very 
easy. And I would show you the patch, but unfortunately, the antlr-mode 
in Emacs doesn't use prog-indentation-context.

So really, it's been here for 2 years, and virtually nobody's using it, 
or improving it.

> Removing it without having some alternative again makes no
> sense to me. 

You have the alternative, though.

> We should discuss this when the incompatible code lands;

How about we discuss it now?

> at that time, we will see how to remove/replace prog-widen etc. with
> minimal pain for its users.  For now, there's no incompatibility that
> requires us to remove the code.  And anyway, making such changes when
> we are in pretest is against our practices.

That seems very unwise to me. Not to mention discouraging.

Take a look at any third-party packages. I'm willing to bet none use 
prog-indentation-context yet. Maybe the standalone version of 
antlr-mode, if only.