bug#49860: 28.0.50; add IRCv3 building blocks to ERC

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Let's try this again (previous attempt vanished down the memory hole).

-------------------- Start of forwarded message --------------------
From: "J.P." <jp@neverwas.me>
To: bug-gnu-emacs@gnu.org
Subject: 28.0.50; add IRCv3 building blocks to ERC
Date: Tue, 03 Aug 2021 02:22:59 -0700
Cc: emacs-erc@gnu.org

Hi,

It's high time we explored bringing IRCv3 to ERC. I say we approach this
by focusing first on those innovations most likely to alleviate the more
pernicious problems endangering this client, such as

  * text processing bottlenecks
  * buffer association / session management / connection bookkeeping
  * maintenance debt

and leave the bells and whistles (fun stuff) for the next generation of
contributors [1].


Text processing

Perhaps most evident during history playback, sluggish message handling
looks poised to become a persistent and pervasive drag. This will likely
only worsen as the average length, complexity, and frequency of messages
increases. In the short term, I'd like to offer UI feedback indicating
the relative progress of remaining work by leveraging the heads-up that
batch provides [2].

Of course, this won't improve raw performance or relieve any I/O
pressure [3], but it will enrich the user experience overall. Also worth
prioritizing (even if it prolongs wait times, IMO) would be ensuring
some allowance for minimal interactivity (like basic navigation) while
processing is ongoing.


Bookkeeping

IRCv3 provides a much needed solution for determining and tracking
session ownership and uniqueness, and that's account awareness. While
useful for keeping tabs on other users, it also offers standardized,
real-time knowledge of a user's own authentication status [4]. This is
critical for laying to rest long festering issues [5] widely felt during
the recent move from Freenode to Libera.


Codebase

The flexibility and granularity demanded by the spec (different sets of
extensions for different sessions) forces us to make ERC more limber and
session-focused. This means ensuring the right seams and machinery exist
for adapting to context/environment at runtime. The CLOS dispatch
facility may be the obvious choice, but any combination of solutions
providing comparable flexibility would be a marked improvement over the
status quo.


A call to action
~~~~~~~~~~~~~~~~

To kick things off, I'm asking for a seasoned contributor to volunteer
their experience and elbow grease and to roll up their sleeves and get
stuck in with me down here in debbugstan. I vow to do whatever it takes
to shoulder most of the burden, whether that means cobbling together a
blueprint to get the ball rolling and/or doing the lion's share of the
intel legwork [6]. Please find it in yourself to step forward and answer
the call.

Thanks,
J.P.


Notes
~~~~~

[1] Even the traditional set of building blocks I hope to introduce
    should open the door to a wealth of opportunities. For example, by
    caching and tracking things like away statuses, idle times, and
    account IDs, we can dynamically update buffers to have nicks go dim
    or italic and otherwise react as updates arrive. (For anyone
    saying "please no": as with all things ERC, such enhancements would
    be optional/opt-in. The point is the possibilities are many.)

[2] https://ircv3.net/specs/extensions/batch

[3] Eventually, it may be nice to actually shoot for performance gains,
    perhaps by spawning child processes that ingest raw batched text and
    return structured data nearly ready for insertion. Also along these
    lines would be an optional "agent" subprocess to buffer I/O and
    wrangle PINGs while Emacs is otherwise preoccupied. (Some of the
    server-initiated message types introduced by IRCv3 are pretty
    chatty.)

[4] https://ircv3.net/specs/extensions/account-notify

[5] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=48598

[6] Actually, I've been engaged in the latter (note taking, knocking on
    doors) for the better part of a year, now. As for the former, a
    provisional/experimental (but usable) implementation may soon find
    its way to this thread, if only to kick start the conversation.


In GNU Emacs 28.0.50 (build 1, x86_64-redhat-linux-gnu, GTK+ Version 3.24.30, cairo version 1.17.4)
 of 2021-07-20 built on localhost
Repository revision: 1b251ed4e8550c889d17fe7d88f58aa2fbc95fe0
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Fedora 34 (Workstation Edition)

Configured using:
 'configure --enable-check-lisp-object-type --enable-checking=yes,glyphs
 --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu
 --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin
 --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share
 --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec
 --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --with-dbus --with-gif --with-jpeg --with-png
 --with-rsvg --with-tiff --with-xft --with-xpm --with-x-toolkit=gtk3
 --with-gpm=no --with-xwidgets --with-modules --with-harfbuzz
 --with-cairo --with-json build_alias=x86_64-redhat-linux-gnu
 host_alias=x86_64-redhat-linux-gnu CC=gcc 'CFLAGS=-O0 -g3'
 LDFLAGS=-Wl,-z,relro
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
X11 XDBE XIM XPM XWIDGETS GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads
xwidget-internal dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit
x multi-tty make-network-process emacs)

Memory information:
((conses 16 51359 6165)
 (symbols 48 6620 1)
 (strings 32 18257 1521)
 (string-bytes 1 615936)
 (vectors 16 13438)
 (vector-slots 8 179083 10089)
 (floats 8 21 47)
 (intervals 56 260 0)
 (buffers 992 10))

-------------------- End of forwarded message --------------------

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Hi,

I've gone ahead and started laying some groundwork [1]. Although parts
may look rather cemented in place, please don't let that deter anyone
from proposing a new direction, even if that means a complete overhaul.

In the meantime, I'm offering a usable POC-turned-WIP [2] that I'll
continue to update and report on until told otherwise. The basic
approach is rather conservative, with compatibility driving most
decisions. As such, external packages like erc-hl-nicks appear to hold
up just fine, though that's merely a happy side effect. (BTW, I've been
using some form of this as a daily driver for some time now, not that
anyone should care.)

Since waiting for collaborators to emerge from the woodwork may take
forever, I've decided to start shaving off small pieces and submitting
them as separate bugs. Some of these changes won't make much sense
without the larger context, but so be it. And while it may appear like
prevailing attitudes toward bold changes in ERC country would render
such an exercise absurd and quixotic, I think the sheer presence of lots
of little crumbs out on the dance floor leading back here can only help
long term.

Thanks,
J.P.

P.S. Perhaps this bug's severity should be reconsidered because some v3
extensions, like SASL, may soon be de facto required by major networks.


Notes
~~~~~

[1] Latest: https://jpneverwas.gitlab.io/erc-tools/49860/patches.tar.gz
            https://jpneverwas.gitlab.io/erc-tools/49860/logs.tar.gz

    Snapshots for refuseniks:

    https://debbugs.gnu.org/cgi/bugreport.cgi?att=1;msg=6;filename=patches.tar.gz;bug=49860
    https://debbugs.gnu.org/cgi/bugreport.cgi?msg=6;bug=49860;filename=logs.tar.gz;att=2

[2] To try this thing without patching/building, just install as usual:

      (require 'package)

      (push '("erc-tools"
              . "https://jpneverwas.gitlab.io/erc-tools/archive/")
            package-archives)

    Then M-x list-packages RET, and find the bottom-most entry for this
    bug, which should look something like:

      erc 49860.20210805.5 available An Emacs Internet Relay Chat ...

    And hit [Install] in the popup. After that, just add:

      (require 'erc-v3)
      (push 'v3 erc-modules)

      ;; Optionally, add this demo module showing some v3 features in action
      (push 'eldoc erc-modules)

    And connect as you normally would. (If you need SASL, see the
    commentary in erc-v3-sasl.el).

Re: bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[Olivier Certner]

Hi J.P.,

Just a small word to say that I didn't have time to review your work (this you 
have noticed...) and probably will not for the next 2 months. I may have a 
small window to try catching up at beginning of September, that's all I can 
hope for at this point. In the meantime, good luck and keep up with this work.

Thanks,
Olivier

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Olivier Certner <ocert.dev@free.fr> writes:

> Just a small word ...

Thanks Olivier. Small words mean a lot. I appreciate the encouragement
and the forecast and look forward to hearing your thoughts down the
road.

bug#49860: Status of IRCv3 support?

[Alexis]

i'm using the soju bouncer, which supports the IRCv3 'chathistory'  extension. From what i can tell, this isn't supported in the ERC that ships with 29.3? If that's so, has any work been done on adding support in this regard?

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Hi Alexis,

Apologies for the late reply.

Alexis <flexibeast@gmail.com> writes:

> i'm using the soju bouncer, which supports the IRCv3 'chathistory' extension.
> From what i can tell, this isn't supported in the ERC that ships with
> 29.3?

That's correct.

> If that's so, has any work been done on adding support in this regard?

In terms of basic, foundational IRCv3, a number of design decisions
still have to be made before we can arrive at something installable. I
will try to summarize the work that's been done so far and propose a
rough roadmap/checklist soon. If you're curious or able to contribute,
please say so, and I'll make sure to Cc you.

For now, there are some WIP patches [1] for the various foundational
extensions as well as POCs for fancier ones like chathistory. On 29, you
can try them as an ELPA package by doing

  (require 'package)
  (push '("emacs-erc" . "https://emacs-erc.gitlab.io/bugs/archive/")
        package-archives)

followed by C-u M-x package-install RET erc-49860 RET. When that's done,
connect from emacs -Q with something like

  (require 'erc-v3)
  (erc-toggle-debug-irc-protocol)
  (setopt erc-modules `(eldoc fill-wrap nicks scrolltobottom v3
                        ,@erc-modules)
          erc-v3-extensions `(draft/multiline
                              draft/chathistory draft/event-playback
                              labeled-response ,@erc-v3-extensions)
          erc-scrolltobottom-all t)
  (erc-tls
   :server "bouncer.alexis-vps.el"
   :port 6670
   :nick "alexis"
   :user "alexis@laptop/testnet"
   :password "hunter2"
   :full-name "alexis")

Please be aware that many things simply won't work and that seemingly
random errors will occasionally occur. You'll also be met with many
annoying inconveniences, like having to hit C-v at the top of a target
buffer to fetch non-catch-up history (infinite scroll). Similarly,
buffers for all query targets you've conversed with recently will show
up for no good reason on connect because we don't (yet) persist any
state to disk. Another pain point is query buffers not being renamed
when users change nicks while you're disconnected. Additionally, expect
to see duplicate messages near the bounding "bookend" indicators,
which are themselves unsightly and yet visible by default.

If you do end up trying these builds for any sustained period, please
occasionally check for buffers named "*erc-foo-error*" and, if possible,
send their contents to me along with those of the "*erc-protocol*"
buffer. Remember, if a build breaks for whatever reason, you can always
"roll back" to a less broken one.

Anyway, if not already obvious, ERC needs contributors to make this
initiative happen. So if you know or hear of anyone willing to work on
the most frivolous corner of Emacs, please give them a nudge.

Thanks,
J.P.

[1] https://emacs-erc.gitlab.io/bugs/49860/patches.tar.gz 

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[Alexis]

Thanks for the update, and for explaining how to try out current 
work! i'm certainly happy to help with testing things out and 
reporting what i find, although unfortunately i'm not in a 
position to be able to contribute beyond that ....


Alexis.

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Here's an update on the current state of the proposed implementation. As
of now, the target version remains ERC 5.7, which will coincide with
Emacs 31 at the earliest. To get involved in this initiative, please
comment in this thread or in the channel.


Modeling an IRCv3 extension
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Modules are the primary means of extending ERC. They're basically
ERC-managed minor modes and customization groups. IRCv3 extensions are
likewise modular and also provide functionality that's activated in a
declarative manner. In many ways, it would make sense to model
extensions as local modules. However, extensions are protocol driven and
depend on coordination with the server. This means successful activation
depends on details discovered from the logical connection long after
modules are typically initialized. Extensions also generally describe
and affect a lower level of functionality than do traditional modules.

For the sake of simplicity, I believe we should frame extensions as
supplementary in nature and as providing additional functionality to
modules. For example, `nickbar' might display account names and away
statuses when extensions providing such awareness become active. And
even when an extension's functionality seems inherently coupled to a
module's feature set, I think we should resist the urge to let
extensions and modules activate or otherwise control one another [1].
For example, the `read-marker' extension seems naturally suited for
integration with `keep-place-indicator', so much so that a hypothetical
implementation would likely be a no-op when the module's disabled. In
these situations, I'd actually prefer silently dropping functionality
over engaging in active feedback, e.g., by enabling the module on the
user's behalf or issuing a didactic warning. Instead, we can explain
these nuanced relationships in the manual and mention in doc strings
that modules can exhibit alternative functionality when various
extensions are active.

With this in mind, I'm proposing we adopt a mostly traditional,
object-oriented approach to modeling extensions, specifically one that
marries a new struct-based type with the convenience and familiarity of
minor modes, all without exposing either as part of the public
interface. Users will instead mostly interact with extensions
indirectly, through a new `v3' module. The subset of extensions slated
for activation will itself be a user option. As for a serious library
API, I'd rather wait a release or two.

Under the proposed scheme, extensions will be actual minor modes with an
associated mode variable and a non-interactive toggle function (rather
than a traditional mode command). These modes will be kept internal and
modified slightly to meet the demands of your typical extension. Most
importantly, instead of being t, a mode variable's enabled value will be
an instance of a new `extension' "type," a hierarchical data structure
to be shared among all buffers of a session as a first-class citizen.
Its purpose: to describe the extension's health and lifecycle stage and
its relationship to other extensions. It may also contain arbitrary
application state relevant to the session.

The bulk of the technical challenges arising from this design are well
understood and thus solvable by prior art. For example, an extension's
definition includes its dependencies, which will be resolved and loaded
in topological order. The more novel challenges mostly involve making
the design play nice with ERC's existing architecture. For example,
modules persist state between IRC sessions by inspecting and possibly
assuming ownership of assets they manage from the prior session. This
ritual normally occurs just after major-mode activation and before
dialing. As mentioned previously, extensions are subject to discovery
and possibly negotiation, meaning if they're to manage modules, they'll
need the ritual to be postponed or prolonged so they can participate.
Complicating matters slightly is the proposed means of presenting users
access to extensions and IRCv3 functionality as a whole: being ERC, this
interface must be compatibility focused and optional by default. History
would thus dictate we do this by encapsulating it all behind a single
`v3' library and an accompanying local module.

Here's an example of an extension's definition:

  (erc-v3--define-capability spam
    :depends '(batch)
    :supports '(labeled-response multiline)
    :aliases '(draft/spam)
    :enablep #'erc-spam--extract-cap-values
    :slots ((foo 0 :type integer)
            (bar nil :type list))
    :keymap erc-v3--spam-mode-map
    (if erc-v3--spam
        (do-init-stuff)
      (undo-init-stuff)))

This can be thought of as a quasi "class" definition for a so-called
`capability', which is a type of `extension' that has additional methods
and attributes relevant to IRCv3 capability negotiation. Under the hood,
this defines a minor mode named `erc-v3--spam' whose activation function
and local mode variable share the same name. The lack of a "-mode"
suffix is an obfuscation tactic to dissuade users and package authors
from discovering and handling it directly. We can provide analogs for
user-defined extensions, if necessary.

Continuing with the example, the `:depends' and `:supports' items
declare hard and soft dependencies respectively, which are guaranteed to
precede this extension if selected for activation. If a hard dependency
is missing, ERC silently skips activation. Everything after the final
keyword pair becomes the body of the mode's toggle function. As
mentioned earlier, the non-nil (enabled) value of the mode variable is
actually an instance of a `capability' (subtype) and is instantiated via
the mode's activation toggle, which doubles as a constructor.


Historical insertions and deletions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A few key extensions pretty much require some minimally invasive means
of inserting and deleting messages at various historical points in a
buffer (rather than just appending). I've heard various IRC experts
refer to this as the "mutable buffer" requirement. These operations
don't demand "random access" in the constant-time sense. In our case,
such insertions and deletions will be reserved for relatively rare
occasions, so we can likely afford to scan for a single UUID or a
timestamp when necessary. And of those operations stemming from a single
application action, all but the first will be sequential and won't need
scanning. This should afford us the liberty of not having to retain and
associate a marker with every inserted message but instead only track
important ones demarcating interesting regions, such as datelines and
history playback intervals.

The end goal of adding such an infrastructure is a richer, somewhat more
web-like and less terminal-like experience, in which a buffer's visible
contents update dynamically in response to arriving messages. By far the
biggest challenge to providing this functionality is accommodating
so-called "stateful" features, namely, those that treat messages as
quasi "recurrences" where the appearance and behavior of a message
depends on the appearance and behavior of the one preceding it. Modules
providing such features need a way to hook into supported splicing and
excising operations to run integrating code of a healing or "mending"
nature. In practice, this sort of manual interpolation will likely rely
on crude approximations and heuristics, although having a persistent
message store may help minimize any visible scarring.

Here are some examples of stateful features that require nuanced
mending:

- Invisibility. Many modules add their own `invisible' text-property
  tokens for controlling the visibility of affected portions of the
  buffer. For example, timestamp and fool visibility can overlap yet be
  toggled independently. This is made possible by the meticulous merging
  and teasing apart of adjacent regions of invisible properties so that
  intervening newlines and neutral regions also abide.

- Smart folding. A hypothetical future module might provide dynamic
  folding and hiding of successive "JOIN" and "QUIT" messages from the
  same user. When enabled, certain sequences of alternating message
  types would be automatically hidden but could later be revealed
  (unfolded) for inspection.

- Amalgamated reactions. Unseen reactions can show up as normal
  messages, e.g., "* bob reacted with a :thumbs up: to alice saying 'ok,
  you?' 30 min ago", but then vanish when marked as read or scrolled off
  screen (or clicked on), at which point their reaction will be
  aggregated in the summary displayed on the referenced message,
  down-buffer.

The current proposal for an infrastructure supporting such mending
operations boils down to intercepting the execution of insertion-hook
members in `erc-insert-line'. Basically, we'll have a function-valued
variable that modules can decorate as needed with local advice in order
to exert influence over insertion-hook members as they're visited by
`run-hook-wrapped'. In most cases, a wrapper will inspect surrounding
messages before and/or after insertion and take care to alter the
behavior of its own hook members as needed. An analogous and somewhat
simpler interface will be offered for deletions.

In general, the proposed approach is messy and inelegant, with plenty of
unwanted cross pollination and implementation leakage. While a tidier
and more abstraction-preserving pattern would be much preferred, we can
at least find some solace in knowing that these ungainly interfaces will
all be kept internal.


A backing store
~~~~~~~~~~~~~~~

An efficient storage mechanism for structured message data will allow us
to "repaint" portions of a buffer for various needs. This means we can
discard similar info currently retained in text properties and
buffer-local variables, often for relatively rare uses. For example, we
might want to retain the account name of a speaker on all their messages
in order to know whether they were logged in at the time a certain
message attributed to their nick was inserted (not so much for forensics
as UI enhancement). But if we can query that information at will, we can
instead dispense with keeping it in-buffer. We'll also be free to
destructively modify inserted messages in order to display them in an
abbreviated or idealized way instead of dressing them up in a veneer of
`display' props merely to ensure their underlying text remains
unadulterated (for faithful logging and killing, etc.).

It's worth emphasizing that this proposal does not currently advocate
for a meaningfully persistent nonvolatile storage solution spanning
Emacs sessions. Although the store should be resilient enough to survive
reconnects, its primary purpose will be to facilitate the lessening and
simplification of in-buffer, per-message data. It's also worth noting
that this addition won't really help with historical insertions and
deletions, which describe an orthogonal concern. And although the
current WIP implementation has yet to be fleshed out and wired in, it's
pretty much a given it'll rely on SQLite as a back end, meaning we'll be
needing a fallback solution for older Emacsen.


Generic response handlers
~~~~~~~~~~~~~~~~~~~~~~~~~

One key to minimizing the maintenance footprint of this initiative is
somehow finding a way to override long-established response-processing
behavior. Most of it originates from default response handlers, like
`erc-server-PRIVSMG', which run on abnormal response hooks, like
`erc-server-PRIVMSG-functions'. The traditional way of doing this
involves preempting the default handler (e.g., `erc-server-PRIVMSG') by
adding an overriding hook member that returns non-nil. I'm proposing we
add another, internal means of overriding such behavior, namely, by
converting a small subset of default handlers to generic functions.
Going this route should reduce the presence of response-hook members
managed by ERC while also sparing third-party members likely churn (as
well as the hassle of learning about hook depth, which currently only
affects insertion hooks).

Folks who worry about generics proliferation are usually referring to
public functions designed to be overridden by users. In our case, only a
handful of implementations for a given handler will ever exist, and
they'll all be internal, so the "polymorphic" dispatch penalty should be
kept relatively negligible. This penalty is usually at most a minor
concern for high-level code like ours that runs relatively infrequently.
For example, in Python 3.9, the penalty is roughly n log(n) when doing a
"BINARY_ADD" on two lists, which is why Pythonistas use "LIST_EXTEND"
(star syntax) when dealing with hot code paths because its complexity is
linear on account of not needing any dispatch.

One potential complication to be mindful of with these generic handlers
is how they'll intersect with handler aliases. For the purpose of code
reuse, some default handlers have aliases, like `erc-server-NOTICE' for
`erc-server-PRIVMSG'. When converted to generics, these handlers will
still always share the same code as their referent. To put it another
way, barring some terrible abuse of `&context' specializers, generics
can't help us override only one among a set of aliased handlers
(something that's occasionally desirable). For example, if we only want
to override "PRIVMSG" handling, we must code that into the method's
implementation, e.g., by running `cl-call-next-method' on receiving a
"NOTICE", so the message gets the default treatment. This may seem
obvious, but a historical quirk makes it easy to confuse with related
hook behavior because ERC doesn't `defvar-alias' them, so modifying one
never modifies others.


Reusable response handling
~~~~~~~~~~~~~~~~~~~~~~~~~~

A common gripe regarding ERC's response handling API is that there's no
way to pass refined, processed data down the line to other handlers.
Although annoying, it's only meaningful to the extent suitable library
functions exist to process such data. This proposal includes a plan to
address both deficiencies, but only in service of the use case explained
in the previous section about overriding default handlers. The idea is
to leverage a common message-handling paradigm that preserves work
artifacts derived from a raw message and other inputs. A shared message
object retains these products for the remainder of its life. The object
typically offers a set of methods and properties for handlers to perform
common operations on inputs, often repeatedly, without being wasteful
or fussing over complicated implementation details.

At present, ERC's main message type is the `erc-response', which at face
value is inadequate for this purpose. However, if we indulge the notion
of its "substitutability" in existing infrastructure and pretend that
functions and variables expecting a traditional `erc-response' won't
balk when handed a subtype, a wealth of possibilities emerge. (I suggest
we do this.) Ignoring whatever performance gains this reuse-focused
scheme may provide, the main win here, from a maintenance standpoint, is
that a module can override some or all of a default handler's duties
without additional upkeep. IOW, the "downstream" library no longer has
to study the default handler and replicate choice bits of copy pasta.
Rather, the library merely wires together whatever combination of
getters and properties it desires, a la carte.

The proposed implementation demonstrated below may seem a bit heavy on
magic, but it makes adapting existing code to these new, more specific
response objects relatively seamless and transparent (aside from the
requisite symbol renaming). The variant being proposed doesn't actually
use traditional methods but rather slot accessors themselves as caching
getters. Regardless, the key takeaway is that it introduces a set of
`erc-response' subtypes for commonly overridden responses, each with
relevant slots that it initializes lazily, on first use. Pros include
code reuse and encapsulation to better isolate concerns as well as
(likely infinitesimal) performance gains. Cons include additional
onboarding overhead for new contributors and a slightly elevated risk of
misuse due to faulty assumptions about its nonstandard struct behavior.

Here's an example response definition specific to a "PRIVMSG":

  (erc--define-zresponse (PRIVMSG NOTICE)
    :include erc--zstatused
    ( buffer (erc-get-buffer (if (erc--zPRIVMSG-query-p parsed)
                                 (car (erc--zresponse-nuh parsed))
                               (erc--zstatused-target parsed))
                             erc-server-process)
      :type buffer)
    ( query-p (equal (erc-downcase (erc--zstatused-target parsed))
                     (erc--zresponse-mynick-d parsed))
      :type boolean)
    ( speaker (car (erc--zPRIVMSG-nuh parsed))
      :type string)
    ( notice-p (string= (erc-response.command parsed) "NOTICE")
      :type boolean)
    ( input-p (string= (erc--zPRIVMSG-mynick-d parsed)
                       (car (erc--zPRIVMSG-nuh parsed)))
      :type boolean))

In the definition above, the init forms are basically single-use method
bodies for the generated accessors (subsequent calls return cached
results). The init forms themselves are free to reference other
accessors generated by this definition, which in turn initialize _their_
slots, if necessary, in a cascading fashion. The variable `parsed' is a
reference to the `zresponse' instance, i.e., "this"/"self". Some care
has been taken to replicate the inlining benefits and gv-place awareness
provided by definitions normally generated by `cl-defstruct' (although
review by an expert in this area would be most welcome).


Notes
~~~~~

[1] IMO, allowing services to "pull in" one another will only lead to
    unwanted complications. Systems that allow for this already have a
    sophisticated foundation in place to manage intricate interactions
    between producers and consumers.

    A note on terminology: I used to make a point of distinguishing
    between "active" and "enabled" when referring to modules: "enabled"
    meant present in `erc-modules' and "active" meant activated for the
    session as a minor mode. I've since abandoned trying to advocate for
    this usage or any such distinction and am fully resigned to the fact
    that others will always use them interchangeably.

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

"J.P." <jp@neverwas.me> writes:

> Under the proposed scheme, extensions will be actual minor modes with an
> associated mode variable and a non-interactive toggle function (rather
> than a traditional mode command). These modes will be kept internal and
> modified slightly to meet the demands of your typical extension. Most
> importantly, instead of being t, a mode variable's enabled value will be
> an instance of a new `extension' "type," a hierarchical data structure
> to be shared among all buffers of a session as a first-class citizen.
> Its purpose: to describe the extension's health and lifecycle stage and
> its relationship to other extensions. It may also contain arbitrary
> application state relevant to the session.

How would IRv3 extension be implement as minor mode would work?
For example deactivating them does not work without reconnecting.
Further they are also interconnected you need some to activate others.

I think there are some where deactivating them doesn't make sense
to me such as for example the server-time extension where the client
will know when a message was send as opposed to not knowing it and
assuming that the time the message was send is the time the client
receives the message.

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Hi Björn,

Björn Bidar <bjorn.bidar@thaodan.de> writes:

> How would IRv3 extension be implement as minor mode would work?
> For example deactivating them does not work without reconnecting.
> Further they are also interconnected you need some to activate others.

It's true that minor modes tend to be user-facing with user-initiated
activation: the user pulls a lever to toggle it on and off. With this
design, the protocol pulls the lever, sometimes by way of other
extensions when "interconnected," as you say. However, the user still
controls which advertised subset gets activated. What _won't_ work is
trying to run

  (erc-v3--foo +1)

out of the blue and expecting ERC to activate `foo' somehow. That must
be done via the protocol (and ultimately via user configuration).

My intention in describing these extensions as conforming to the
minor-mode interface (to whatever extent such a thing exists, even as a
loosely defined set of requirements) was merely as shorthand to imply
they provide setup and teardown code, a mode variable, etc. affecting
functionality layered atop a major mode. But while this _does_ satisfy
those fundamental requirements (and thus "implement" the interface),
perhaps that's more of a technicality, and stressing the point will only
cause confusion. Therefore, in the future, I will relegate all such
mention of minor modes to a mere footnote. Thanks for raising this
concern.

> I think there are some where deactivating them doesn't make sense
> to me such as for example the server-time extension where the client
> will know when a message was send as opposed to not knowing it and
> assuming that the time the message was send is the time the client
> receives the message.

Well, some module attempting to deactivate them itself via

  (erc-v3--foo -1)

definitely won't work. But manually coaxing the protocol to do so on
your behalf, i.e.,

  /CAP REQ -server-time RET

is indeed supported, albeit probably nonsensical and not recommended.

Deactivating them because they've been DEL'd, however, is an actual
requirement and obviously protocol-driven. Being asked to do that for
`server-time' in particular is admittedly unlikely. I suppose in
pathological cases, like a replication failover or a clock-sync/skew
emergency, the network may need to disable stamps temporarily so they're
not relayed to users (where they could cause havoc in logs).

Anyway, ERC has been offering a mostly functional POC for users to try
for years now. You can find the info elsewhere in this bug thread, just
in case you're interested or are desperate to read some questionable
code. However, I'm guessing "not very" since my records indicate you're
a Circe user! (But I appreciate your input regardless.)

Cheers,
J.P.

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

"J.P." <jp@neverwas.me> writes:

> Hi Björn,
>
> Björn Bidar <bjorn.bidar@thaodan.de> writes:
>
>> How would IRv3 extension be implement as minor mode would work?
>> For example deactivating them does not work without reconnecting.
>> Further they are also interconnected you need some to activate others.
>
> My intention in describing these extensions as conforming to the
> minor-mode interface (to whatever extent such a thing exists, even as a
> loosely defined set of requirements) was merely as shorthand to imply
> they provide setup and teardown code, a mode variable, etc. affecting
> functionality layered atop a major mode. But while this _does_ satisfy
> those fundamental requirements (and thus "implement" the interface),
> perhaps that's more of a technicality, and stressing the point will only
> cause confusion. Therefore, in the future, I will relegate all such
> mention of minor modes to a mere footnote. Thanks for raising this
> concern.
>

I think I was taking the word minor-mode to literal. My concerns
came from the point that in theory a user could call the minor mode
function while they are connected and cause issues.
Thank you for explaining further.

Having each extension be module can be nice however it also increases
the complexity and overhead that comes with two many small modules.

>
> Anyway, ERC has been offering a mostly functional POC for users to try
> for years now. You can find the info elsewhere in this bug thread, just
> in case you're interested or are desperate to read some questionable
> code. However, I'm guessing "not very" since my records indicate you're
> a Circe user! (But I appreciate your input regardless.)
>

I'm indeed a Circe user however I'm interested in working IRCv3 in Emacs
and was thinking on how to implement in Emacs in the past.
At the end of the day working IRCv3 is very important, more important
than which mode to use, although ERC is a bit messy compared to Circe
from my point of view. But that's off-topic anyhow.