Lisp primitives and their calling of the change hooks

[Alan Mackenzie <acm@muc.de>]

Hello, Emacs.

I've just had an interesting few days investigating our privitives'
calling of before-change-functions and after-change-functions.

In an ideal world, each primitive would call each of those hooks exactly
once.  However, we're not in an ideal world, and there are primitives
which perform several (or many) distinct changes (e.g. transpose-regions
or subst-char-in-region) where a single pair of hook calls wouldn't make
sense.

TL;DR: There are several, but not many, primitives where
before-change-functions doesn't match after-change-functions.

Here is how I went about this:
1. Extract a list of external functions in insdel.c from the section in
lisp.h which declares them.  Form a regexp from this list, and convert it
into a grep regexp (by replacing '?' by '\?'.

2. grep -l *.c with this regexp.  This gave these files:
    buffer.c
    callproc.c
    casefiddle.c
    cmds.c
    coding.c
    decompress.c
    editfns.c
    emacs.c
    fileio.c
    fns.c
    indent.c
    insdel.c
    print.c
    process.c
    search.c
    textprop.c
    xdisp.c
    xml.c

3. Using GNU cflow, a utility which creates call graphs for C files,
create a reverse call graph (i.e. an "is called by" graph) for the 18 C
files.

4. Analyse this graph with an elisp script to find all functions which,
directly or indirectly, call signal_before_change or signal_after_change.

5. Filter this list of functions to leave only the lisp primitives (i.e.
functions starting with "F"), and convert to Lisp names.  Edit this list
by hand to remove those primitives which don't change the buffer (most of
them were removed).  This left the following list:
    (add-face-text-property)
    (add-text-properties)
    (base64-decode-region)
    (base64-encode-region)
    (capitalize-region)

    (capitalize-word)
    (delete-and-extract-region)
    (delete-char)
    (delete-field)
    (delete-region)

    (downcase-region)
    (downcase-word)
    (erase-buffer)
    (indent-to)
    (insert)

    (insert-and-inherit)
    (insert-before-markers)
    (insert-buffer-substring)
    (insert-byte)
    (insert-char)

    (insert-file-contents)
    (move-to-column)
    (princ)
    (print)
    (put-text-property)

    (remove-list-of-text-properties)
    (remove-text-properties)
    (replace-buffer-contents)
    (replace-match)
    (self-insert-command)

    (set-buffer-multibyte)
    (set-text-properties)
    (subst-char-in-region)
    (terpri)
    (translate-region-internal)

    (transpose-regions)
    (upcase-initials-region)
    (upcase-region)
    (upcase-word)
    (write-char)

    (zlib-decompress-region)

6. Write and run a script which executes each of these primitives whilst
counting the number of times it invokes before-change-hooks and
after-change-hooks.  Output messages where these numbers aren't 1 and 1.
This gave the following:
    Primitive              b-c-f calls         a-c-f calls
base64-encode-region          2                    2
base64-decode-region          2                    1
insert-file-contents          2                    2
move-to-column-1              3                    3   ; with the &optional
                                                       ; which untabifies.
replace-buffer-contents      123                  123
set-buffer-multibyte          0                    0   ; May have done nothing
tranlate-region-internal      1                   146
tranpose-regions              2                    1
upcase-initials-region        1                    0[*]
upcase-region                 1                    0[*]
zlib-decompress-region        0                    0
erase-buffer                  0                    0

[*] In upcase-... there weren't any lower case characters in the buffer
at the time.

This list is incomplete.  There were one or two other primitives which
triggered messages which I didn't write down, and now cannot reproduce.

7. Surpising is that insert-file-contents (which gave so much trouble in
summer 2016) returned balanced counts.  It seems to be mostly the special
purpose primitives, those not widely used in Lisp, which are most likely
to have strange b-c-f and a-c-f counts.  But the case-switching
primitives might be troublesome.  So might transpose-regions.

8. Possibly these things could be more accurately documented in
elisp.info.

-- 
Alan Mackenzie (Nuremberg, Germany).