bug#58797: 29.0.50; Revise format of stored message tags in ERC

bug#58797: 29.0.50; Revise format of stored message tags in ERC

[J.P.]

[-- Attachment #1: Type: text/plain, Size: 5624 bytes --]

Tags: patch

I'm proposing we change the format (type) of the "tags" field in the
`erc-response' struct from

  (STRING . LIST)

where LIST contains at most one (possibly empty) string, to

  (SYMBOL . OPT-STRING)

where OPT-STRING, when non-nil, is a nonempty string. For ERC 5.5 and
Emacs 29, this change would merely manifest as a warning when first
encountered. In subsequent releases, it would still be reserved only for
users who opt in by activating optional modules. Rationale below.

Thanks,
J.P.


Imagined FAQ:

  Why bother?

    In the existing implementation, all code accessing a processed tag
    value must check that it's both non-nil and nonempty. This stands to
    become a common occurrence with the arrival of IRCv3 in future
    releases. The majority of instances will only contain one item (a
    timestamp) and will persist beyond the parsing stage as text
    properties in ERC buffers. And since we need to redo the parsing
    anyway to abide by all the escaping rules and other spec minutiae,
    we might as well make the processed goods as easy to manage as
    possible, no?

  Isn't this a breaking change?

    Yes. But I believe preserving backward compatibility (by adding an
    additional field and keeping the old one around) is less important
    in this case because the field currently goes unused in ERC's client
    code and is likely underused in user code (based on a cursory survey
    of Melpa packages, popular shared configs, and wiki snippets). While
    message tags *are* currently parsed when present, the field is only
    ever written to and never again accessed (on account of ERC's lack
    of general IRCv3 support). That said, the library function as well
    as the `erc-response' structure are indeed public (exported).

    There is also the matter of authoritative intent to consider. The
    doc string for the function `erc-parse-tags' describes its return
    type as being an alist of "(tag . value)". For anyone willing to
    entertain "value" as meaning "logical tag value," this change
    becomes a belated bug fix.

  Why now? Why not wait?

    It's true that the main benefit of doing this won't be felt for a
    couple more ERC releases. As stated above, the immediate effect of
    this change will merely be as a warning for any user code that
    depends on this field. This approach seems somewhat in keeping with
    the overall Emacs policy for function and variable deprecation.


In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.34, cairo version 1.17.6) of 2022-10-26 built on localhost
Repository revision: 832bd2dfe51b2b9ea5e0e56e0e6ee60b2d8ae2ed
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12014000
System Description: Fedora Linux 36 (Workstation Edition)

Configured using:
 'configure --enable-check-lisp-object-type --enable-checking=yes,glyphs
 'CFLAGS=-O0 -g3'
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
emacs)

Memory information:
((conses 16 36089 6500)
 (symbols 48 5118 0)
 (strings 32 13124 1756)
 (string-bytes 1 372990)
 (vectors 16 9304)
 (vector-slots 8 147959 9387)
 (floats 8 21 29)
 (intervals 56 220 0)
 (buffers 1000 10))

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Warn-of-future-breaking-change-to-erc-response.tags.patch --]
[-- Type: text/x-patch, Size: 4619 bytes --]

From 44cb69befc28ae4e14499521fb2f8a09d7acaf5c Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@neverwas.me>
Date: Mon, 24 Oct 2022 22:58:13 -0700
Subject: [PATCH] Warn of future breaking change to erc-response.tags

* lisp/erc/erc-backend.el (erc-parse-tags-format): New option to
determine type of the `erc-response' "tags" field.
(erc-parse-tags): Defer to internal generic function.
(erc--parse-tags): New function to hold original `erc-parse-tags'
implementation.
(erc--parse-message-tags): New generic function that conditionally
calls `erc--parse-tags', perhaps emitting a warning beforehand.
(erc-parse-server-response): Call `erc--parse-message-tags'.
---
 lisp/erc/erc-backend.el | 59 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 56 insertions(+), 3 deletions(-)

diff --git a/lisp/erc/erc-backend.el b/lisp/erc/erc-backend.el
index df9efe4b0c..9854e863c3 100644
--- a/lisp/erc/erc-backend.el
+++ b/lisp/erc/erc-backend.el
@@ -992,8 +992,39 @@ erc-send-ctcp-notice
 
 ;;;; Handling responses
 
+(defcustom erc-tags-format 'overridable
+  "Shape of the `tags' alist in `erc-response' objects.
+When set to `legacy', pre-5.5 parsing behavior takes effect for
+the tags portion of every message.  The resulting alist contains
+conses of the form (STRING . LIST), in which LIST is comprised of
+at most one (possibly empty) string.
+
+When nil, ERC only parses tags if an active module defines an
+implementation.  It otherwise ignores them.  In such cases, each
+alist element is a cons of a symbol and an optional, non-empty
+string.
+
+With the default value of `overridable', ERC behaves as it does
+with `legacy' except that it emits a warning whenever first
+encountering a message containing tags in a given Emacs session.
+But it only does so when a module implementing overriding,
+non-legacy behavior isn't already active in the current network
+context.
+
+Note that future bundled modules providing IRCv3 functionality
+may not be compatible with the legacy format.  User code should
+eventually transition to expecting this \"5.5+ variant\" and set
+this option to nil."
+  :package-version '(ERC . "5.4.1") ; FIXME increment on next release
+  :type '(choice (const nil)
+                 (const legacy)
+                 (const overridable)))
+
 (defun erc-parse-tags (string)
   "Parse IRCv3 tags list in STRING to a (tag . value) alist."
+  (erc--parse-message-tags string))
+
+(defun erc--parse-tags (string)
   (let ((tags)
         (tag-strings (split-string string ";")))
     (dolist (tag-string tag-strings tags)
@@ -1003,6 +1034,28 @@ erc-parse-tags
                 `(,pair))
               tags)))))
 
+;; One benefit of this function being internal is to avoid having to
+;; define a separate method just to ensure an `erc-tags-format' value
+;; of `legacy' always wins.  The downside is that module code must
+;; take care to preserve that promise manually.
+
+(cl-defgeneric erc--parse-message-tags (string)
+  "Parse STRING into an alist of (TAG . VALUE) conses.
+TAG is a symbol.  VALUE is nil or a non-empty string.  Composite
+raw input values containing commas are never split but always
+left as a single string."
+  (when erc-tags-format
+    (unless (or (eq erc-tags-format 'legacy)
+                (get 'erc-parse-tags 'erc-v3-warned-p))
+      (put 'erc-parse-tags 'erc-v3-warned-p t)
+      (display-warning
+       'ERC
+       (concat
+        "Legacy ERC tags behavior is currently in effect, but other modules,"
+        " including those bundled with ERC, may override this in future"
+        " releases.  See `erc-tags-format' for more info.")))
+    (erc--parse-tags string)))
+
 (defun erc-parse-server-response (proc string)
   "Parse and act upon a complete line from an IRC server.
 PROC is the process (connection) from which STRING was received.
@@ -1012,9 +1065,9 @@ erc-parse-server-response
       (let* ((tag-list (when (eq (aref string 0) ?@)
                          (substring string 1
                                     (string-search " " string))))
-             (msg (make-erc-response :unparsed string :tags (when tag-list
-                                                              (erc-parse-tags
-                                                               tag-list))))
+             (msg (make-erc-response :unparsed string :tags
+                                     (when tag-list
+                                       (erc--parse-message-tags tag-list))))
              (string (if tag-list
                          (substring string (+ 1 (string-search " " string)))
                        string))
-- 
2.37.3

bug#58797: 29.0.50; Revise format of stored message tags in ERC

[Emanuel Berg]

J.P. wrote:

> Tags: patch
>
> I'm proposing we change the format (type) of the "tags"
> field in the `erc-response'

What is that, it's not a variable and it's not a function ...

> struct from
>
>   (STRING . LIST)
>
> where LIST contains at most one (possibly empty) string, to
>
>   (SYMBOL . OPT-STRING)

Well, as you know, symbol and string are object types in
Emacs, the use of the "OPT" prefix OTOH signals it's a name
and the purpose is to hold options, this mix isn't good IMO.

You can change OPT to OPTS perhaps, SYMBOL I don't know what
to change to since I don't know what symbols are intended to
be stored there ...

Also IMO the dotted list data structure is passé but if code
relies on it I don't suggest changing that just for the sake
of it ...

> For ERC 5.5 and Emacs 29

Okay, but isn't ERC built-in only or can you get "future"
version of ERC from GNU ELPA? Okay, that's it then, I see that
5.4.1 is avaliable there, I'm on

  ERC 5.4.1 (IRC client for GNU Emacs 29.0.50)

But I use, as you see already, a very fresh Emacs, namely

  GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, cairo
  version 1.16.0) of 2022-10-09 [commit
  a2dd9d683a02525183b5e692e42622c65639dda8]

so I guess I'm on the edge here :)

> Why bother?

To improve the software ...

> Isn't this a breaking change?

You can't make an omelet without hatching eggs ...

> Why now? Why not wait?

Do it today, in a different way!

-- 
underground experts united
https://dataswamp.org/~incal

Re: bug#58797: 29.0.50; Revise format of stored message tags in ERC

[J.P.]

Emanuel Berg <incal@dataswamp.org> writes:

> J.P. wrote:
>
>> Tags: patch
>>
>> I'm proposing we change the format (type) of the "tags"
>> field in the `erc-response'
>
> What is that, it's not a variable and it's not a function ...

Um, I guess by "type" I meant something more like `:type' in the broader
"describing the shape of some field" sense. Not so much (info "(elisp)
Lisp Data Types") or what have you. Apologies for the confusion.

>> struct from
>>
>>   (STRING . LIST)
>>
>> where LIST contains at most one (possibly empty) string, to
>>
>>   (SYMBOL . OPT-STRING)
>
> Well, as you know, symbol and string are object types in
> Emacs, the use of the "OPT" prefix OTOH signals it's a name
> and the purpose is to hold options, this mix isn't good IMO.
>
> You can change OPT to OPTS perhaps, SYMBOL I don't know what
> to change to since I don't know what symbols are intended to
> be stored there ...

Poor choice of descriptive label on my part, clearly. I didn't mean to
suggest a custom user option but rather something like a
NIL-OR-NONEMPTY-STRING. IOW, whatever might satisfy a `typep' spec of

  '(cons symbol (or null (and string (not (string 0)))))

or similar (bastardized pseudo-CL notwithstanding).

>> For ERC 5.5 and Emacs 29
>
> Okay, but isn't ERC built-in only or can you get "future"
> version of ERC from GNU ELPA? Okay, that's it then, I see that
> 5.4.1 is avaliable there, I'm on
>
>   ERC 5.4.1 (IRC client for GNU Emacs 29.0.50)

The releases on ELPA are basically snapshots of lisp/erc taken whenever
the ";; Version: " header changes. So, the work you see on HEAD is still
unreleased. It's possible that incrementing the version ahead of time
(like to 5.5-git or something) would make that clearer, but AFAIK we
can't do that without triggering yet another release. Hope that makes
sense.