From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Michael Heerdegen Newsgroups: gmane.emacs.bugs Subject: bug#40704: 28.0.50; Improve and speed up (Gnus) registry saving Date: Sun, 19 Apr 2020 04:14:00 +0200 Message-ID: <87a738ql1z.fsf@web.de> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="13099"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eric Abrahamsen To: 40704@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Apr 19 04:15:55 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jPzUx-0003J6-JF for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 19 Apr 2020 04:15:55 +0200 Original-Received: from localhost ([::1]:36474 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jPzUw-0004EZ-Dy for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 18 Apr 2020 22:15:54 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:55624) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jPzU7-0004BN-7I for bug-gnu-emacs@gnu.org; Sat, 18 Apr 2020 22:15:03 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jPzU6-00085U-MI for bug-gnu-emacs@gnu.org; Sat, 18 Apr 2020 22:15:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:60537) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jPzU6-00084w-9g for bug-gnu-emacs@gnu.org; Sat, 18 Apr 2020 22:15:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jPzU5-0003wH-TR for bug-gnu-emacs@gnu.org; Sat, 18 Apr 2020 22:15:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Michael Heerdegen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 19 Apr 2020 02:15:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 40704 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.158726245515062 (code B ref -1); Sun, 19 Apr 2020 02:15:01 +0000 Original-Received: (at submit) by debbugs.gnu.org; 19 Apr 2020 02:14:15 +0000 Original-Received: from localhost ([127.0.0.1]:43850 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jPzTL-0003us-0h for submit@debbugs.gnu.org; Sat, 18 Apr 2020 22:14:15 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:38412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jPzTI-0003uk-Rw for submit@debbugs.gnu.org; Sat, 18 Apr 2020 22:14:13 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:55168) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jPzTI-0003Mc-Do for bug-gnu-emacs@gnu.org; Sat, 18 Apr 2020 22:14:12 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jPzTH-0007Xp-Qg for bug-gnu-emacs@gnu.org; Sat, 18 Apr 2020 22:14:12 -0400 Original-Received: from mout.web.de ([212.227.17.12]:48197) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jPzTH-0007Ua-4D for bug-gnu-emacs@gnu.org; Sat, 18 Apr 2020 22:14:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1587262438; bh=UYC1btuYddzqCUQaJiaUTWKZOvqd6ClUEQWO0a3KGwQ=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date; b=d3K9FBNK7Y8tFBA88192VJXNQssxgKqMY+liaGR/QJjfyAZakntT0o+22Ix7CNkxU 0jfF+zyagO4VeBOaaDxDkEPou+cHpXwrmVyeJEsDayB2VzwMc2NT/SwZOl4pWDEfIZ //gmJmu/vtKT4BkJ6ugZvt2f8exFDP8wnpodTys0= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Original-Received: from drachen.dragon ([188.98.99.7]) by smtp.web.de (mrweb102 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MWj6V-1jizDp15IJ-00XujL; Sun, 19 Apr 2020 04:13:58 +0200 X-Provags-ID: V03:K1:72DVJg2XPFw/iUlCC3EWSoOF714EPmOjXHGWnNMo8d74YjSJ9EF FizHwrVBdSf5qkoE6Odlk87bPSqLRMF7Xo/m5x/3k98sKOQlK1BUA/zNAAuZyqUwrZg768I djD8hDKYFXJtvvTFPiKQWu3B2sXllDFcaZh95iKy1nxafNgnAzoC5lK+hxjXAphQdWnDcgO W4hgXd43QamGmxgJHm01g== X-UI-Out-Filterresults: notjunk:1;V03:K0:THVUBuAJCWU=:/LBRKf1j07638O7V+daMIu 0RCS9ov8NMqXL+2TDgFpwkwrX2uES9oaaizJeZqh02z6dFmMRnFgM2YLuDq4Ca27+d+k0l73D u78A3x1kgJyROHcgnIYXQm4IHPoBEK9QAeIrEEhLyMgtaVoCT91ORk4J63AjRWXgmb3x5mFen n37SmLz9M5IQht2v9sa6tO0irjB3+b3zWF7eTrrRqgdynGKvSm7xYW4Dml4WAz/uSD3nh+OO7 j5aHvWV/wkWtUm7ug1DBzA2Hzzz/Z83eSUqyVmG8wqgWk7N7usbkkgVfZUd0v09zzZFEznA1K Ib26kRSmqXShoN+6BtgBTeq9bBHAeuFLXuoJX9ahHoF1Fa2ro1wIHopZVK9uB7rXjeisal5l7 aAHcgrcDH3ZAnDIOGulJ6sgz5Kr28CHz0ptJCEljsVoQL2BRcNTo18cQIG6DWrF9PrK/2vXQb kfBxbmH8su6MEU550lWuZ8fGP2Vf2JkLfy8nl/P5hs8Kv2Pe1VdhO2VxM8NXXJ5RA2ttUXMJy cudxeKdsrcsiwgbHH67sPGUu4Mfs9jUTQVcMwUaGvtBbhmBJ56dIYYxThdzcMPLUCT5EToTv8 JaYNgZsqTx/bEbEopbv8KUMrhU0PCoe2+GFSa3+fb6hfyQWP0UQMBppsCOC96ao/31yYqhvEr FAKBaVHo34pAT4EIBWOhpt2C/NQqP3subG7RszN81ZvcXhmcy/lK8mwdULfXS8cXSP1o4rMM3 g580FhNAPZBX3HWqnX6tLXD5lR01vU+DgF5OVpxelTil0B+0cszlR9rEzvsUVL/Wh0qPDmvY Received-SPF: pass client-ip=212.227.17.12; envelope-from=michael_heerdegen@web.de; helo=mout.web.de X-detected-operating-system: by eggs.gnu.org: Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:178617 Archived-At: --=-=-= Content-Type: text/plain Hello, Saving the Gnus registry is quite slow currently. I profiled a bit and for now suggest to do something like in the attached patch. In detail: (1) We need to bind inhibit-modification-hooks -> t, this offers a good speedup (~ 4 or so). (2) Printing the registry which basically consists of huge hash tables, causes a lot of garbage. Most of that garbage seems to be unavoidable (is it created by the printing primitives?). Anyway, seems we should temporarily increase `gc-cons-threshold' drastically, this offers another speedup of 25% or so. The patch attached uses the value that works well for me and the size of my registry, and I bind it in `gnus-registry-save', because I assume other registries outside of Gnus can be smaller. What would be a good value of `gc-cons-threshold', or should it even scale with `gnus-registry-max-entries' instead of being constant? (3) I also decided to change `eieio-override-prin1' to print hash tables "by hand" from Lisp. The eieio-persistent requires to modify how elements in the hash tables are printed, and the current way of doing this (make a copy of the complete table, change the elements, prin1 and re-read the result) is not only hackish but also inefficient (it does this recursively for nested tables). Any comments on the suggested changes? TIA, Michael. --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-WIP-Try-to-improve-gnus-registry-saving.patch Content-Transfer-Encoding: quoted-printable =46rom 618362e3496df18d0f60143b71185ad57501fdcb Mon Sep 17 00:00:00 2001 From: Michael Heerdegen Date: Mon, 13 Apr 2020 23:33:39 +0200 Subject: [PATCH] WIP: Try to improve gnus registry saving =2D-- lisp/emacs-lisp/eieio-base.el | 3 ++- lisp/emacs-lisp/eieio.el | 33 +++++++++++++++++++++++++-------- lisp/gnus/gnus-registry.el | 1 + 3 files changed, 28 insertions(+), 9 deletions(-) diff --git a/lisp/emacs-lisp/eieio-base.el b/lisp/emacs-lisp/eieio-base.el index 2cb1f614ce..010a2b673e 100644 =2D-- a/lisp/emacs-lisp/eieio-base.el +++ b/lisp/emacs-lisp/eieio-base.el @@ -473,7 +473,8 @@ eieio-persistent-save (let* ((cfn (or file (oref this file))) (default-directory (file-name-directory cfn))) (cl-letf ((standard-output (current-buffer)) - ((oref this file) ;FIXME: Why change it? + (inhibit-modification-hooks t) + ((oref this file) ;FIXME: Why change it? (if file ;; FIXME: Makes a name relative to (oref this file), ;; whereas I think it should be relative to cfn. diff --git a/lisp/emacs-lisp/eieio.el b/lisp/emacs-lisp/eieio.el index 9f8b639e52..43d515e1fa 100644 =2D-- a/lisp/emacs-lisp/eieio.el +++ b/lisp/emacs-lisp/eieio.el @@ -934,15 +934,32 @@ eieio-override-prin1 ((consp thing) (eieio-list-prin1 thing)) ((hash-table-p thing) - (let ((copy (copy-hash-table thing))) - (maphash + (princ "#s(hash-table size ") + (prin1 (hash-table-size thing)) + (princ " test ") + (prin1 (hash-table-test thing)) + (princ " weakness ") + (prin1 (hash-table-weakness thing)) + (princ " rehash-size ") + (prin1 (hash-table-rehash-size thing)) + (princ " rehash-threshold ") + (prin1 (hash-table-rehash-threshold thing)) + (princ " data (") + (let* ((eieio-print-depth (if eieio-print-indentation + (1+ eieio-print-depth) + eieio-print-depth)) + (do-indent (if eieio-print-indentation + (lambda () (princ (make-string (* eieio-pr= int-depth 2) ? ))) + #'ignore))) + (maphash (lambda (key val) - (setf (gethash key copy) - (read - (with-output-to-string - (eieio-override-prin1 val))))) - copy) - (prin1 copy))) + (princ "\n") + (funcall do-indent) + (prin1 key) + (princ " ") + (eieio-override-prin1 val)) + thing)) + (princ "))")) ((vectorp thing) (let ((copy (copy-sequence thing))) (dotimes (i (length copy)) diff --git a/lisp/gnus/gnus-registry.el b/lisp/gnus/gnus-registry.el index 480ed80ef8..4ac3c84a80 100644 =2D-- a/lisp/gnus/gnus-registry.el +++ b/lisp/gnus/gnus-registry.el @@ -398,6 +398,7 @@ gnus-registry-save (interactive) (let* ((file (or file gnus-registry-cache-file)) (db (or db gnus-registry-db)) + (gc-cons-threshold (max gc-cons-threshold (* 800000 500))) (clone (clone db))) (gnus-message 5 "Saving Gnus registry (%d entries) to %s..." (registry-size db) file) =2D- 2.25.1 --=-=-=--