Taming some chaos

Taming some chaos

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 5122 bytes --]

On Sat, Oct 10, 2015 at 2:31 PM, John Wiegley <johnw@newartisans.com> wrote:

> We should define what we want from a "more IDE" Emacs. For example, I do
> not
> want window-layout functionality. That's a presentation aspect of IDEs
> that's
> entirely separate from what I want from them.
>
...

> To reiterate: I think Emacs becomes more of an IDE when it provides the
> backbone of an IDE, and not when it looks like one. We have some pieces of
> that backbone already, which have avoided fragmentation in those areas,
> but we
> need more. A standardized way to do tooltips, popups, visual completion
> selection (or at least the structure), REPLs that interact with buffer
> contents, etc., would give us a foundation to move to the next step.


Apart from any IDE functionality there are general weaknesses in Emacs'
current collection of packages that I think represents a significant
barrier to
bringing in new recruits, both users and developers.  I want to argue that
most
packages dabble in too many areas and are coded in too imperative a manner.
Such shortcoming can be understood in light of the era and technological
setting in which many Emacs conventions and paradigms were first  developed.
But that does not alter the fact that the state of computing / UX /
what-have-you
has advanced significantly.

From a user's POV too many Emacs packages violate the Unix paradigm
of simple composable tools that perform a single task well.  Instead of
being
supported by a framework or an environment in which a user registers global
policies and preferences and wherein an executing package requests services
in terms of high-level abstractions too many Emacs packages implement
large swashes of concrete, detailed UI logic.  Here are two examples:

Window management

Too many packages explicitly choreograph how windows are managed,
where and when they should be split, where buffers should be displayed
and whether or not any previous window configuration should be restored.
Such logic is articulated in terms of detailed manipulation of a simplistic
 "current-window / other-window" model no matter how poorly it maps to
the user's actual configuration.  In essence such packages have taken on
managing the presentation layer even when that is not their main focus.

I have been developing a small package to maintain a persistent horizontal
window (PHW) akin to ECB's persistent shell window.  My default layout on
a 30" 2560x1600 screen now includes a shallow full width PHW plus two,
three or four additional, balanced vertical "user" or "edit" windows.  I am
struck by how many packages behave poorly in this setting.  When I am
lucky a package I want to use provides sufficient config settings and hooks
that I can achieve satisfactory behavior in my less typical environment.
When I am unlucky I have to add ad-hoc logic into my window manager
to accommodate a number of packages.  Predictably the more window
manipulation a package performs the more likely I am to have to resort to
such ad-hoc solutions.

It does not have to be that way.  In the X world there are any number of
window manages (both tiling and overlapping).  The overwhelming majority
of applications run correctly irrespective of the user's choice of Wm.  In
fact
users have a sufficient expectation of correct inter-operation that when it
breaks they do not accept that breakage as "just the way things are", as
we might do in the Emacs world.  Instead it is viewed as an outright bug,
be it in the WM or in the application (most likely its toolkit or
framework).

Fonts and colors schemes

The "angry salad" meme is only a way of brushing off a truly awful aspect
of adopting a new Emacs package.  Here again we suffer from a lack of
abstractions.  In the realm of text markup technologies we all recognize the
merits of declarative ("this is a first level heading", "this is an element
of an
numbered list", "emphasize this text") over imperative (render this text in
a specific font at a particular point size and weight, prefix this paragraph
with '3.' and indent it by 1/2", use italics).  Surely given Emacs' support
for
inheritance within the faces framework we should be able to introduce a
vocabulary of basic concept faces from which packages then would be
encouraged to inherit.

That said I am not so naive as to think that a finite set of base faces
mapped to semantic concepts will cover the needs of the rich packages
we have.  Ediff has nearly 20 faces and the current version of magit has
over 90.  Trying to wrestle that many faces into a coherent color scheme
and trying to achieve any degree of commonality across packages is
excruciating.  I have to believe there are members of our community who
are familiar with color palette UX strategies and technologies (e.g. iPhone
or Android Material style guidelines).  Hopefully someone will come forward
and propose a model.  In my vision an Emacs user would choose a palette
(likely along with a mapping of palette items to visual roles).  Then an
Emacs
provided service could generate faces on demand based on application
requested attributes / properties.

/john

[-- Attachment #2: Type: text/html, Size: 6594 bytes --]

Re: Taming some chaos

[Dmitry Gutov]

On 10/14/2015 02:47 PM, John Yates wrote:

> Window management
>
> Too many packages explicitly choreograph how windows are managed,
> where and when they should be split, where buffers should be displayed
> and whether or not any previous window configuration should be restored.
> Such logic is articulated in terms of detailed manipulation of a simplistic
>   "current-window / other-window" model no matter how poorly it maps to
> the user's actual configuration.  In essence such packages have taken on
> managing the presentation layer even when that is not their main focus.

I agree. One could point to display-buffer-alist, a third-party packages 
like shackle or popwin, but even then the user has to operate 
lower-level primitives of window management.

Some higher abstraction and UI would do us some good. But it's not like 
it's trivial to even design that.

> It does not have to be that way.  In the X world there are any number of
> window manages (both tiling and overlapping).  The overwhelming majority
> of applications run correctly irrespective of the user's choice of Wm.

X has it easier: each application usually gets its own frame. And they 
can have modal windows (there's nothing really corresponding to that in 
Emacs).

> Fonts and colors schemes
> ...

> Surely given Emacs' support for
> inheritance within the faces framework we should be able to introduce a
> vocabulary of basic concept faces from which packages then would be
> encouraged to inherit.

font-lock-*-face? And faces in lisp/faces.el, the list of which can be 
extended.

> Trying to wrestle that many faces into a coherent color scheme
> and trying to achieve any degree of commonality across packages is
> excruciating.  I have to believe there are members of our community who
> are familiar with color palette UX strategies and technologies (e.g. iPhone
> or Android Material style guidelines).  Hopefully someone will come forward
> and propose a model.  In my vision an Emacs user would choose a palette
> (likely along with a mapping of palette items to visual roles).

That would be nice as well.

Re: Taming some chaos

[John Wiegley]

>>>>> John Yates <john@yates-sheets.org> writes:

> Apart from any IDE functionality there are general weaknesses in Emacs'
> current collection of packages that I think represents a significant barrier
> to bringing in new recruits, both users and developers. I want to argue that
> most packages dabble in too many areas and are coded in too imperative a
> manner. Such shortcoming can be understood in light of the era and
> technological setting in which many Emacs conventions and paradigms were
> first developed. But that does not alter the fact that the state of
> computing / UX / what-have-you has advanced significantly.

I can't really argue with your assessment, John.

However, I think that ship has sailed. The changes you propose in your message
would require a ground-up redesign of many aspects of Emacs, would they not?
We have so much extant Emacs Lisp code, and lot of Emacs' value lies in that
code, and not merely in its fundamentals.

At this point, we can design new abstractions to be used by new code; but we
can't change how fundamental things are done in a way that suddenly makes
Emacs far less capable by breaking reams of existing code.

John

Re: Taming some chaos

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 2557 bytes --]

On Fri, Oct 16, 2015 at 7:20 PM, John Wiegley <johnw@newartisans.com> wrote:

> However, I think that ship has sailed. The changes you propose in your
> message
> would require a ground-up redesign of many aspects of Emacs, would they
> not?
> We have so much extant Emacs Lisp code, and lot of Emacs' value lies in
> that
> code, and not merely in its fundamentals.
>

John,

I fear my opening paragraphs were too inflammatory.  At heart I am an
incrementalist.  It was never my intention to suggest tossing aside our
current code base and package collection.

In leading projects I find that my greatest successes have come when I
have been able to bridge an embraceable critique of the current state of
the system and an appealing vision of a future state with a credible set
of stepping stones to get us there.  Since I have always worked in the
commercial space stepping stones have meant incremental deliverables
that regularly added functionality or performance to a shipping product.

The Emacs community has long felt like a collection of bright, polite
coders pursuing individual visions and scratching personal itches.  In
your postings to the ML it seemed you were interested in stepping
forward to provide the leadership needed to get us to rally to vision
of  an enticing Emacs future.  Articulating such a vision inherently
involves contrasting it with the current state of affairs.  I had hoped
that my note would be taken as a contribution to a discussion about
things could be improved.


> At this point, we can design new abstractions to be used by new code; but
> we
> can't change how fundamental things are done


I could not agree more.  Dmitry responded: "Some higher abstraction
and UI would do us some good. But it's not like it's trivial to even design
that."  That is my point.  To move forward in that dimension we need
some consensus that such changes are part of where we want to go.
Once such consensus emerges I have no doubt that design discussions
would arise naturally.

... in a way that suddenly makes
> Emacs far less capable by breaking reams of existing code.
>

Given the personalities of the old-timers in this community I have no
fear that Emacs will end up any less capable.  It would be entirely in
character to provide new abstractions capable of being configured to
reproduce current behavior (probably the default for users having yet
to register an specific choices).  A measure of success for any new
abstraction would be the rate at which actively maintained packages
would port over to using it.

/john

[-- Attachment #2: Type: text/html, Size: 4491 bytes --]

Re: Taming some chaos

[John Wiegley]

>>>>> John Yates <john@yates-sheets.org> writes:

> The Emacs community has long felt like a collection of bright, polite coders
> pursuing individual visions and scratching personal itches. In your postings
> to the ML it seemed you were interested in stepping forward to provide the
> leadership needed to get us to rally to vision of an enticing Emacs future.
> Articulating such a vision inherently involves contrasting it with the
> current state of affairs. I had hoped that my note would be taken as a
> contribution to a discussion about things could be improved.

Hi John!

I apologize for apparently having misread your intent completely. Reading this
last message, I couldn't agree with you more, both in sentiment and approach.
I think we want almost exactly the same things.

Please allow me take back any criticism of your ideas as being too wild, and
encourage you to rephrase them with an eye to "maintaining existing value".
Can we correct past decisions in the light of new advances, in a way that
carries forward the merits of that past?

I find myself in the strange position of wanting change -- even radical change
-- while equally wanting to respect the tremendous value and experience we've
built up over the decades. Striking this balance at the right points is key.

John