bug#49860: 28.0.50; add IRCv3 building blocks to ERC

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Let's try this again (previous attempt vanished down the memory hole).

-------------------- Start of forwarded message --------------------
From: "J.P." <jp@neverwas.me>
To: bug-gnu-emacs@gnu.org
Subject: 28.0.50; add IRCv3 building blocks to ERC
Date: Tue, 03 Aug 2021 02:22:59 -0700
Cc: emacs-erc@gnu.org

Hi,

It's high time we explored bringing IRCv3 to ERC. I say we approach this
by focusing first on those innovations most likely to alleviate the more
pernicious problems endangering this client, such as

  * text processing bottlenecks
  * buffer association / session management / connection bookkeeping
  * maintenance debt

and leave the bells and whistles (fun stuff) for the next generation of
contributors [1].


Text processing

Perhaps most evident during history playback, sluggish message handling
looks poised to become a persistent and pervasive drag. This will likely
only worsen as the average length, complexity, and frequency of messages
increases. In the short term, I'd like to offer UI feedback indicating
the relative progress of remaining work by leveraging the heads-up that
batch provides [2].

Of course, this won't improve raw performance or relieve any I/O
pressure [3], but it will enrich the user experience overall. Also worth
prioritizing (even if it prolongs wait times, IMO) would be ensuring
some allowance for minimal interactivity (like basic navigation) while
processing is ongoing.


Bookkeeping

IRCv3 provides a much needed solution for determining and tracking
session ownership and uniqueness, and that's account awareness. While
useful for keeping tabs on other users, it also offers standardized,
real-time knowledge of a user's own authentication status [4]. This is
critical for laying to rest long festering issues [5] widely felt during
the recent move from Freenode to Libera.


Codebase

The flexibility and granularity demanded by the spec (different sets of
extensions for different sessions) forces us to make ERC more limber and
session-focused. This means ensuring the right seams and machinery exist
for adapting to context/environment at runtime. The CLOS dispatch
facility may be the obvious choice, but any combination of solutions
providing comparable flexibility would be a marked improvement over the
status quo.


A call to action
~~~~~~~~~~~~~~~~

To kick things off, I'm asking for a seasoned contributor to volunteer
their experience and elbow grease and to roll up their sleeves and get
stuck in with me down here in debbugstan. I vow to do whatever it takes
to shoulder most of the burden, whether that means cobbling together a
blueprint to get the ball rolling and/or doing the lion's share of the
intel legwork [6]. Please find it in yourself to step forward and answer
the call.

Thanks,
J.P.


Notes
~~~~~

[1] Even the traditional set of building blocks I hope to introduce
    should open the door to a wealth of opportunities. For example, by
    caching and tracking things like away statuses, idle times, and
    account IDs, we can dynamically update buffers to have nicks go dim
    or italic and otherwise react as updates arrive. (For anyone
    saying "please no": as with all things ERC, such enhancements would
    be optional/opt-in. The point is the possibilities are many.)

[2] https://ircv3.net/specs/extensions/batch

[3] Eventually, it may be nice to actually shoot for performance gains,
    perhaps by spawning child processes that ingest raw batched text and
    return structured data nearly ready for insertion. Also along these
    lines would be an optional "agent" subprocess to buffer I/O and
    wrangle PINGs while Emacs is otherwise preoccupied. (Some of the
    server-initiated message types introduced by IRCv3 are pretty
    chatty.)

[4] https://ircv3.net/specs/extensions/account-notify

[5] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=48598

[6] Actually, I've been engaged in the latter (note taking, knocking on
    doors) for the better part of a year, now. As for the former, a
    provisional/experimental (but usable) implementation may soon find
    its way to this thread, if only to kick start the conversation.


In GNU Emacs 28.0.50 (build 1, x86_64-redhat-linux-gnu, GTK+ Version 3.24.30, cairo version 1.17.4)
 of 2021-07-20 built on localhost
Repository revision: 1b251ed4e8550c889d17fe7d88f58aa2fbc95fe0
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Fedora 34 (Workstation Edition)

Configured using:
 'configure --enable-check-lisp-object-type --enable-checking=yes,glyphs
 --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu
 --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin
 --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share
 --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec
 --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --with-dbus --with-gif --with-jpeg --with-png
 --with-rsvg --with-tiff --with-xft --with-xpm --with-x-toolkit=gtk3
 --with-gpm=no --with-xwidgets --with-modules --with-harfbuzz
 --with-cairo --with-json build_alias=x86_64-redhat-linux-gnu
 host_alias=x86_64-redhat-linux-gnu CC=gcc 'CFLAGS=-O0 -g3'
 LDFLAGS=-Wl,-z,relro
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
X11 XDBE XIM XPM XWIDGETS GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads
xwidget-internal dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit
x multi-tty make-network-process emacs)

Memory information:
((conses 16 51359 6165)
 (symbols 48 6620 1)
 (strings 32 18257 1521)
 (string-bytes 1 615936)
 (vectors 16 13438)
 (vector-slots 8 179083 10089)
 (floats 8 21 47)
 (intervals 56 260 0)
 (buffers 992 10))

-------------------- End of forwarded message --------------------

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Hi,

I've gone ahead and started laying some groundwork [1]. Although parts
may look rather cemented in place, please don't let that deter anyone
from proposing a new direction, even if that means a complete overhaul.

In the meantime, I'm offering a usable POC-turned-WIP [2] that I'll
continue to update and report on until told otherwise. The basic
approach is rather conservative, with compatibility driving most
decisions. As such, external packages like erc-hl-nicks appear to hold
up just fine, though that's merely a happy side effect. (BTW, I've been
using some form of this as a daily driver for some time now, not that
anyone should care.)

Since waiting for collaborators to emerge from the woodwork may take
forever, I've decided to start shaving off small pieces and submitting
them as separate bugs. Some of these changes won't make much sense
without the larger context, but so be it. And while it may appear like
prevailing attitudes toward bold changes in ERC country would render
such an exercise absurd and quixotic, I think the sheer presence of lots
of little crumbs out on the dance floor leading back here can only help
long term.

Thanks,
J.P.

P.S. Perhaps this bug's severity should be reconsidered because some v3
extensions, like SASL, may soon be de facto required by major networks.


Notes
~~~~~

[1] Latest: https://jpneverwas.gitlab.io/erc-tools/49860/patches.tar.gz
            https://jpneverwas.gitlab.io/erc-tools/49860/logs.tar.gz

    Snapshots for refuseniks:

    https://debbugs.gnu.org/cgi/bugreport.cgi?att=1;msg=6;filename=patches.tar.gz;bug=49860
    https://debbugs.gnu.org/cgi/bugreport.cgi?msg=6;bug=49860;filename=logs.tar.gz;att=2

[2] To try this thing without patching/building, just install as usual:

      (require 'package)

      (push '("erc-tools"
              . "https://jpneverwas.gitlab.io/erc-tools/archive/")
            package-archives)

    Then M-x list-packages RET, and find the bottom-most entry for this
    bug, which should look something like:

      erc 49860.20210805.5 available An Emacs Internet Relay Chat ...

    And hit [Install] in the popup. After that, just add:

      (require 'erc-v3)
      (push 'v3 erc-modules)

      ;; Optionally, add this demo module showing some v3 features in action
      (push 'eldoc erc-modules)

    And connect as you normally would. (If you need SASL, see the
    commentary in erc-v3-sasl.el).

Re: bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[Olivier Certner]

Hi J.P.,

Just a small word to say that I didn't have time to review your work (this you 
have noticed...) and probably will not for the next 2 months. I may have a 
small window to try catching up at beginning of September, that's all I can 
hope for at this point. In the meantime, good luck and keep up with this work.

Thanks,
Olivier

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Olivier Certner <ocert.dev@free.fr> writes:

> Just a small word ...

Thanks Olivier. Small words mean a lot. I appreciate the encouragement
and the forecast and look forward to hearing your thoughts down the
road.

bug#49860: Status of IRCv3 support?

[Alexis]

i'm using the soju bouncer, which supports the IRCv3 'chathistory'  extension. From what i can tell, this isn't supported in the ERC that ships with 29.3? If that's so, has any work been done on adding support in this regard?

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[J.P.]

Hi Alexis,

Apologies for the late reply.

Alexis <flexibeast@gmail.com> writes:

> i'm using the soju bouncer, which supports the IRCv3 'chathistory' extension.
> From what i can tell, this isn't supported in the ERC that ships with
> 29.3?

That's correct.

> If that's so, has any work been done on adding support in this regard?

In terms of basic, foundational IRCv3, a number of design decisions
still have to be made before we can arrive at something installable. I
will try to summarize the work that's been done so far and propose a
rough roadmap/checklist soon. If you're curious or able to contribute,
please say so, and I'll make sure to Cc you.

For now, there are some WIP patches [1] for the various foundational
extensions as well as POCs for fancier ones like chathistory. On 29, you
can try them as an ELPA package by doing

  (require 'package)
  (push '("emacs-erc" . "https://emacs-erc.gitlab.io/bugs/archive/")
        package-archives)

followed by C-u M-x package-install RET erc-49860 RET. When that's done,
connect from emacs -Q with something like

  (require 'erc-v3)
  (erc-toggle-debug-irc-protocol)
  (setopt erc-modules `(eldoc fill-wrap nicks scrolltobottom v3
                        ,@erc-modules)
          erc-v3-extensions `(draft/multiline
                              draft/chathistory draft/event-playback
                              labeled-response ,@erc-v3-extensions)
          erc-scrolltobottom-all t)
  (erc-tls
   :server "bouncer.alexis-vps.el"
   :port 6670
   :nick "alexis"
   :user "alexis@laptop/testnet"
   :password "hunter2"
   :full-name "alexis")

Please be aware that many things simply won't work and that seemingly
random errors will occasionally occur. You'll also be met with many
annoying inconveniences, like having to hit C-v at the top of a target
buffer to fetch non-catch-up history (infinite scroll). Similarly,
buffers for all query targets you've conversed with recently will show
up for no good reason on connect because we don't (yet) persist any
state to disk. Another pain point is query buffers not being renamed
when users change nicks while you're disconnected. Additionally, expect
to see duplicate messages near the bounding "bookend" indicators,
which are themselves unsightly and yet visible by default.

If you do end up trying these builds for any sustained period, please
occasionally check for buffers named "*erc-foo-error*" and, if possible,
send their contents to me along with those of the "*erc-protocol*"
buffer. Remember, if a build breaks for whatever reason, you can always
"roll back" to a less broken one.

Anyway, if not already obvious, ERC needs contributors to make this
initiative happen. So if you know or hear of anyone willing to work on
the most frivolous corner of Emacs, please give them a nudge.

Thanks,
J.P.

[1] https://emacs-erc.gitlab.io/bugs/49860/patches.tar.gz 

bug#49860: 28.0.50; add IRCv3 building blocks to ERC

[Alexis]

Thanks for the update, and for explaining how to try out current 
work! i'm certainly happy to help with testing things out and 
reporting what i find, although unfortunately i'm not in a 
position to be able to contribute beyond that ....


Alexis.