bug#40704: 28.0.50; Improve and speed up (Gnus) registry saving

[Michael Heerdegen <michael_heerdegen@web.de>]

[-- Attachment #1: Type: text/plain, Size: 1333 bytes --]


Hello,

Saving the Gnus registry is quite slow currently.  I profiled a bit and
for now suggest to do something like in the attached patch.  In detail:

(1) We need to bind inhibit-modification-hooks -> t, this offers a good
speedup (~ 4 or so).

(2) Printing the registry which basically consists of huge hash tables,
causes a lot of garbage.  Most of that garbage seems to be unavoidable
(is it created by the printing primitives?).  Anyway, seems we should
temporarily increase `gc-cons-threshold' drastically, this offers
another speedup of 25% or so.  The patch attached uses the value that
works well for me and the size of my registry, and I bind it in
`gnus-registry-save', because I assume other registries outside of Gnus
can be smaller.  What would be a good value of `gc-cons-threshold', or
should it even scale with `gnus-registry-max-entries' instead of being
constant?

(3) I also decided to change `eieio-override-prin1' to print hash tables
"by hand" from Lisp.  The eieio-persistent requires to modify how
elements in the hash tables are printed, and the current way of doing
this (make a copy of the complete table, change the elements, prin1 and
re-read the result) is not only hackish but also inefficient (it does
this recursively for nested tables).

Any comments on the suggested changes?

TIA,

Michael.




[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-WIP-Try-to-improve-gnus-registry-saving.patch --]
[-- Type: text/x-diff, Size: 3406 bytes --]

From 618362e3496df18d0f60143b71185ad57501fdcb Mon Sep 17 00:00:00 2001
From: Michael Heerdegen <michael_heerdegen@web.de>
Date: Mon, 13 Apr 2020 23:33:39 +0200
Subject: [PATCH] WIP: Try to improve gnus registry saving

---
 lisp/emacs-lisp/eieio-base.el |  3 ++-
 lisp/emacs-lisp/eieio.el      | 33 +++++++++++++++++++++++++--------
 lisp/gnus/gnus-registry.el    |  1 +
 3 files changed, 28 insertions(+), 9 deletions(-)

diff --git a/lisp/emacs-lisp/eieio-base.el b/lisp/emacs-lisp/eieio-base.el
index 2cb1f614ce..010a2b673e 100644
--- a/lisp/emacs-lisp/eieio-base.el
+++ b/lisp/emacs-lisp/eieio-base.el
@@ -473,7 +473,8 @@ eieio-persistent-save
     (let* ((cfn (or file (oref this file)))
            (default-directory (file-name-directory cfn)))
       (cl-letf ((standard-output (current-buffer))
-                ((oref this file)       ;FIXME: Why change it?
+                (inhibit-modification-hooks t)
+                ((oref this file) ;FIXME: Why change it?
                  (if file
                      ;; FIXME: Makes a name relative to (oref this file),
                      ;; whereas I think it should be relative to cfn.
diff --git a/lisp/emacs-lisp/eieio.el b/lisp/emacs-lisp/eieio.el
index 9f8b639e52..43d515e1fa 100644
--- a/lisp/emacs-lisp/eieio.el
+++ b/lisp/emacs-lisp/eieio.el
@@ -934,15 +934,32 @@ eieio-override-prin1
 	((consp thing)
 	 (eieio-list-prin1 thing))
 	((hash-table-p thing)
-         (let ((copy (copy-hash-table thing)))
-	   (maphash
+         (princ "#s(hash-table size ")
+         (prin1 (hash-table-size thing))
+         (princ " test ")
+         (prin1 (hash-table-test thing))
+         (princ " weakness ")
+         (prin1 (hash-table-weakness thing))
+         (princ " rehash-size ")
+         (prin1 (hash-table-rehash-size thing))
+         (princ " rehash-threshold ")
+         (prin1 (hash-table-rehash-threshold thing))
+         (princ " data (")
+         (let* ((eieio-print-depth (if eieio-print-indentation
+                                       (1+ eieio-print-depth)
+                                     eieio-print-depth))
+                (do-indent (if eieio-print-indentation
+                               (lambda () (princ (make-string (* eieio-print-depth 2) ? )))
+                             #'ignore)))
+           (maphash
 	    (lambda (key val)
-	      (setf (gethash key copy)
-		    (read
-		     (with-output-to-string
-		       (eieio-override-prin1 val)))))
-	    copy)
-	   (prin1 copy)))
+              (princ "\n")
+              (funcall do-indent)
+              (prin1 key)
+              (princ " ")
+              (eieio-override-prin1 val))
+            thing))
+         (princ "))"))
 	((vectorp thing)
          (let ((copy (copy-sequence thing)))
 	  (dotimes (i (length copy))
diff --git a/lisp/gnus/gnus-registry.el b/lisp/gnus/gnus-registry.el
index 480ed80ef8..4ac3c84a80 100644
--- a/lisp/gnus/gnus-registry.el
+++ b/lisp/gnus/gnus-registry.el
@@ -398,6 +398,7 @@ gnus-registry-save
   (interactive)
   (let* ((file (or file gnus-registry-cache-file))
          (db (or db gnus-registry-db))
+         (gc-cons-threshold (max gc-cons-threshold (* 800000 500)))
 	 (clone (clone db)))
     (gnus-message 5 "Saving Gnus registry (%d entries) to %s..."
                   (registry-size db) file)
--
2.25.1