Strategies for Composable Modular Configuration?

Strategies for Composable Modular Configuration?

[Psionic K]

AFAIK, if we compare Emacs configuration to NixOS configuration, Elisp
lacks the composable module system.

A consequence is that users cannot easily delegate out the declaration of
configuration in whole or in part.  If I like someone's org mode defaults,
they cannot provide and I cannot consume that configuration except manually
because infrastructure for declaring its boundaries and how to compose it
with my own modifications doesn't exist.

What we would do with a composable module system is, for example, install
and configure a coherent set of minibuffer focused packages but without
getting in the way of the user providing extra configuration on top or
overriding attributes of the module either in a first-class or adhoc way.

1.  What prior art can I look at
2.  What implementation strategies appear viable
3.  Did DOOM and Spacemacs get some things right?

Thanks
-- 

남백호
대표 겸 공동 창업자
포지트론

Re: Strategies for Composable Modular Configuration?

[Nikolay Kudryavtsev]

This topic gets discussed here and there sometimes.

I think the first issue is configuration discoverability. You can run 
someone's init for org and see that you're having a much better 
experience, because of settings A,B,C. But there may also be setting D 
that you don't need. Setting E that actually makes it worse for you. 
Then for you there's actually an even better value for setting B.

Then there's the learning curve problem. Emacs already has quite a 
curve, but any third party configurations almost double it, because you 
now have multiple behavior providers - a thing may be a part of core 
Emacs functionality and the default. While some other thing is provided 
by a third party package X, bundled in your starter kit. Then there's 
the starter kit configuration layers on top of it.

Now onto your questions:

2. Considering your minibuffer package zoo example, I've been thinking 
that something akin to protocols from Clojure would work for this case 
as that extra layer of indirection that can solve those kinds of 
problems. Clojure protocols are its basic are a set of function hooks 
unified under one name. So, for example I decide to use the flight sim 
rudder pedals in my Emacs configuration. Obviously I want them to 
iterate over items. So lets say Emacs had 
item-iteration-binding-protocol. Which has next-item and previous-item 
hooks. Then every minibuffer completion package can implement that and I 
just need to bind my pedals to that next and previous item bindings for 
it to work everywhere. Very obviously this is the best case scenario and 
there would be numerous practical problems when it comes to implementation.

3. I have never played with Doom, but from my limited experience with 
it, Spacemacs seems to be doing as good of a job as possible with its 
layers system, considering the complexity of the problem.

Re: Strategies for Composable Modular Configuration?

[Psionic K]

If abstracting settings by grouping and naming them, yes, the result is
indirection.  I agree that this approach fails to deliver ROI.  IIRC both
Doom and Spacemacs style configuration involves named abstraction of groups
of settings and also cascading some settings into other settings
(composition, but without nearly the discipline of a Nix style solution).

So, for example I decide to use the flight sim
> rudder pedals in my Emacs configuration. Obviously I want them to
> iterate over items. So lets say Emacs had
> item-iteration-binding-protocol.
>

This is also an abstraction by naming style solution.  While it does make
declarations more generic, such declarations must be concurrently
implemented with broad consensus or else it will become  inconsistently
applied indirection.  The ecosystem cannot move in this way except for
problems of near ubiquitous frequency.

Superimposing configuration works differently.  Instead of the packages you
consume providing a generic interface, you would use a "pedals" declaration
that can produce every variation of just the additional contents that each
minibuffer configuration would need.  When adding all the declarations, if
the set doesn't exist, the contents are never calculated and the addition
is no-op.  The minibuffer packages don't care, and maintenance is
completely centralized into the one "pedals" declaration.

Because each group of users who wants a "pedals" can create and maintain it
without any cooperation with the packages they depend on, it's adding
opt-in cooperative gains without organizational complexity for maintainers
in the ecosystem.  It is not easier for the individual pedals users.  It is
easier for groups of users that are larger than individuals but much much
smaller than those with near ubiquitous problems.

Superimposing is a simple union of declarations.  For some strict subset of
possible use-package expressions (ones written with one preferred way for
everything instead of doing things however one pleases), it's trivial to
implement, can express everything, and is transparent to debug.  Discipline
in the Nix community with the introduction of Flakes has been very good to
the ecosystem.

We can leave composition alone.  Composition relies on a fixed point
calculation, a recursive lazy merge where declarations thunk on each other
until everything resolves or a loop is found.  The value add is not
incredible compared to the complexity.  For Emacs users, 90% of the
maintenance overhead and new user setup can be handled by simple union of
declarations.


On Wed, Nov 22, 2023 at 9:59 PM Nikolay Kudryavtsev <
nikolay.kudryavtsev@gmail.com> wrote:

> This topic gets discussed here and there sometimes.
>
> I think the first issue is configuration discoverability. You can run
> someone's init for org and see that you're having a much better
> experience, because of settings A,B,C. But there may also be setting D
> that you don't need. Setting E that actually makes it worse for you.
> Then for you there's actually an even better value for setting B.
>
> Then there's the learning curve problem. Emacs already has quite a
> curve, but any third party configurations almost double it, because you
> now have multiple behavior providers - a thing may be a part of core
> Emacs functionality and the default. While some other thing is provided
> by a third party package X, bundled in your starter kit. Then there's
> the starter kit configuration layers on top of it.
>
> Now onto your questions:
>
> 2. Considering your minibuffer package zoo example, I've been thinking
> that something akin to protocols from Clojure would work for this case
> as that extra layer of indirection that can solve those kinds of
> problems. Clojure protocols are its basic are a set of function hooks
> unified under one name. So, for example I decide to use the flight sim
> rudder pedals in my Emacs configuration. Obviously I want them to
> iterate over items. So lets say Emacs had
> item-iteration-binding-protocol. Which has next-item and previous-item
> hooks. Then every minibuffer completion package can implement that and I
> just need to bind my pedals to that next and previous item bindings for
> it to work everywhere. Very obviously this is the best case scenario and
> there would be numerous practical problems when it comes to implementation.
>
> 3. I have never played with Doom, but from my limited experience with
> it, Spacemacs seems to be doing as good of a job as possible with its
> layers system, considering the complexity of the problem.
>
>

-- 

남백호
대표 겸 공동 창업자
포지트론

Re: Strategies for Composable Modular Configuration?

[Nikolay Kudryavtsev]

I'm not sure that I fully understand what you're suggesting... I've been 
a casual Nixos user for years and even contribute from time to time, but 
nothing made me really care about flakes. So to try and explain this in 
a more practical manner, it's something like this:

There's a portal lets say emacs-tweaks.gnu.org. It operates much the 
same way as a package repository, but the contents of it are not 
features(packages), but tweaks(flakes, pedals, whatever). The difference 
between a feature and a tweak is roughly that features implement new 
behavior, while tweaks just modify existing features to some requirement.

Your definition of tweaks also does not allow for composition. Which 
means that most of them are limited to a single feature, which IMHO is 
not going to provide much ROI either. Because most of those tweaks are 
going to be isolated single variable tweaks and that's not where the 
problem lies. It's also not going to fully take over most of the 
configuration needs because you can't re-implement Spacemacs(on Doom, 
whatever) using such a limited system.

Correct me if I misunderstood anything.

Re: Strategies for Composable Modular Configuration?

[Psionic K]

My rough sketch of a solution is a more strict subset of use-package
expressions that are merged by union.  Detect conflicts.  Support automatic
and hands-on resolution.  Track changes and notify the user when changes
are available.

If use-package were not eager and just collected declarations until the
user said, "Okay that's all my declarations, now execute the ones that
aren't no-op."  We would be free of the 1:1 correlation between use-package
expressions and packages configured for loading.

For configurations that are made across several packages, their declaration
would be untangled.  Can we do this now?  I don't know if some use of defer
enables this behavior, but I have never seen an example or evidence of
support for use in this way.

Composition, the calculation of one set on another set, I don't think is
necessary to do away with 80% of boring configuration, where the defaults
are at odds with what's popular among users.  Any composition will build on
top of things that are better for union aka superimposition, so union must
be supported before composition anyway.

If a users only agrees with 75% of the 80% of popular settings, they still
have only 40% of the configuration overhead.  This works in maintenance
because if changes made over time are only being rejected at a 25% rate,
the you end up doing 40% of the work, and most importantly, you would see
those changes as new conflicts rather than having to keep track of each
package's evolution.  The market rate can adjust to fit the ROI.  For
instance, if you agree with 99% of a small 20% of extremely unpopular
values, depending on them instead of setting and tracking them yourself is
almost a pure time save.

Regarding Flakes and ecosystem health, to understand the kinds of mistakes
that were being made by the uninitiated masses, you can't look at good
organizations, but instead must look at middling organizations and bad
organizations.  The free-for-all approach to creating expressions without
any structure lead to such oddities as operating Nix impurely and
articulating CI and even deployments.  It was a case of using something
within the limits of what worked, and there were no limits because it's a
programming language.  Flakes made these extreme misadventures obviously
wrong enough for them to stop happening.

Being strict about whether each symbol is loaded removes implicit
dependency, and then side-effects that are order-dependent don't rely on
flukes of the loading order and even user behavior to work every time.  The
subset of really nice use-package declarations users do already fits well
with the subset that is trivial to merge in union.

On Thu, Nov 23, 2023 at 8:19 PM Nikolay Kudryavtsev <
nikolay.kudryavtsev@gmail.com> wrote:

> I'm not sure that I fully understand what you're suggesting... I've been
> a casual Nixos user for years and even contribute from time to time, but
> nothing made me really care about flakes. So to try and explain this in
> a more practical manner, it's something like this:
>
> There's a portal lets say emacs-tweaks.gnu.org. It operates much the
> same way as a package repository, but the contents of it are not
> features(packages), but tweaks(flakes, pedals, whatever). The difference
> between a feature and a tweak is roughly that features implement new
> behavior, while tweaks just modify existing features to some requirement.
>
> Your definition of tweaks also does not allow for composition. Which
> means that most of them are limited to a single feature, which IMHO is
> not going to provide much ROI either. Because most of those tweaks are
> going to be isolated single variable tweaks and that's not where the
> problem lies. It's also not going to fully take over most of the
> configuration needs because you can't re-implement Spacemacs(on Doom,
> whatever) using such a limited system.
>
> Correct me if I misunderstood anything.
>
>

-- 

남백호
대표 겸 공동 창업자
포지트론