From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#63861: [PATCH] pp.el: New "pretty printing" code Date: Thu, 08 Jun 2023 18:35:05 -0400 Message-ID: References: <87edmnics1.fsf@posteo.net> <87bkhpvm35.fsf@posteo.net> Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35063"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: 63861@debbugs.gnu.org To: Thierry Volpiatto Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Jun 09 00:36:08 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q7OEh-0008tW-Lf for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 09 Jun 2023 00:36:07 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q7OEc-0000Ch-RT; Thu, 08 Jun 2023 18:36:02 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q7OEc-0000CY-8b for bug-gnu-emacs@gnu.org; Thu, 08 Jun 2023 18:36:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q7OEc-0001EL-0C for bug-gnu-emacs@gnu.org; Thu, 08 Jun 2023 18:36:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1q7OEb-0008Nh-Rj for bug-gnu-emacs@gnu.org; Thu, 08 Jun 2023 18:36:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 08 Jun 2023 22:36:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 63861 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 63861-submit@debbugs.gnu.org id=B63861.168626372032169 (code B ref 63861); Thu, 08 Jun 2023 22:36:01 +0000 Original-Received: (at 63861) by debbugs.gnu.org; 8 Jun 2023 22:35:20 +0000 Original-Received: from localhost ([127.0.0.1]:57857 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q7ODv-0008Mk-UN for submit@debbugs.gnu.org; Thu, 08 Jun 2023 18:35:20 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:14945) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q7ODr-0008MU-Ku for 63861@debbugs.gnu.org; Thu, 08 Jun 2023 18:35:18 -0400 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id CC3A2442828; Thu, 8 Jun 2023 18:35:09 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 6DE1B442822; Thu, 8 Jun 2023 18:35:07 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1686263707; bh=bCsQ14NO62KPSbj1VPVjQkBTZ1PvVa0r27Y3NTb+Ruk=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=mEUhdpXMMnPWp9oqjctZ2BrLpE+Dq4/UF1QlucWzCiY6tBgi7qLoIbwIxaAK9EWzy crAydcLdilc3x9l4cF+TbfOEr2bzwkr+CyXxYsYR87LINqkF/1OaAqTjdnzDZNB35N 6AvmAbUKFYJ2Efk0PEM2MpTTsi5uIcUjGPML0WgocWhXHk9b8bnDcwM2eawo5FFGZZ 9Zn/rPM8yJDtMFnfwp/pHKucWOxbpH+0mipfZSqjOeN71OJWD+5bYxpWCWjKnmznVZ HPLU1IcMj0GP+EaY8yDhKAAX3F2UEhtD133DtVKd2Z3F8mbhnJQwzmRIkYnM+BTeHD ah+kGIBW3xV7w== Original-Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 4E2581207FF; Thu, 8 Jun 2023 18:35:07 -0400 (EDT) In-Reply-To: <87bkhpvm35.fsf@posteo.net> (Thierry Volpiatto's message of "Thu, 08 Jun 2023 18:08:18 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:263144 Archived-At: >> I find these results (mine) quite odd: they suggest that my `pp-region` >> is *faster* than the old `pp-buffer` for `load-history` and `bookmark-alist` >> data, which I definitely did not expect (and don't know how to explain >> either). I've just redone my tests a bit differently, added `pp-emacs-lisp-code`, and also introduced a var to control whether to activate the `lisp-ppss` patch or not. I also fixed my `foo.el` file: its content was accidentally already pretty printed rather than being on a single line, which totally changes the behavior of `pp-region` and `pp-buffer`). For reference: % (cd ~/tmp; l foo.el test*.el) -rw------- 1 monnier monnier 1125154 8 jun 11:20 test-load-history.el -rw------- 1 monnier monnier 163258 8 jun 11:20 test-bookmark-alist.el -rw-r--r-- 1 monnier monnier 77101 8 jun 17:20 foo.el % Here's the code I used to run the test: for f in ~/tmp/foo.el ~/tmp/test-bookmark-alist.el ~/tmp/test-load-history.el; do for ppss in nil t; do for v in '(pp-buffer)' '(pp-region (point-min) (point-max))' '(tv/pp-region (point-min) (point-max))' '(let ((s (read (current-buffer)))) (erase-buffer) (pp-emacs-lisp-code s))'; do src/emacs -Q --batch -l ~/tmp/describe-variable --eval "(with-temp-buffer (emacs-lisp-mode) (insert-file-contents \"$f\") (setq pp-buffer-use-pp-region nil lisp--faster-ppss $ppss) (message \"%S %S %S %S\" (file-name-nondirectory \"$f\") (benchmark-run $v) '$v '$ppss))"&; done; done; done So, by file, from fastest to slowest: foo.el (0.859482743 0 0.0) (pp-buffer) t foo.el (0.890402623 0 0.0) (pp-buffer) nil foo.el (4.62344853 4 1.7225397670000002) (tv/pp-region (point-min) (point-max)) t foo.el (4.687414465 4 1.7116580980000002) (tv/pp-region (point-min) (point-max)) nil foo.el (7.932661181 1 0.3435169600000001) (pp-region (point-min) (point-max)) t foo.el (196.183345212 1 0.618591124) (pp-region (point-min) (point-max)) nil foo.el (2997.739238575 505 105.82851685700001) (let ((s (read (current-buffer)))) (erase-buffer) (pp-emacs-lisp-code s)) t Here we see that my `pp-region` code is slower than `pp-buffer` by a factor ~10x: I'm not very happy about it, but this `foo.el` file was selected because it was the worst case I had come across (tho that was before I found the `lisp-ppss` patch). The last element in each line is whether we activated the `lisp-ppss` patch. As we can see here, the `lisp-ppss` patch makes an enormous difference (~24x) for my code, but not for `pp-buffer` (because it relies on `lisp-indent-region` rather than `lisp-indent-line`) and not for `tv/pp-region` either (because it doesn't indent at all). We also see that `pp-emacs-lisp-code` is *much* slower. I don't include other results for this function in this email because they're still running :-) test-bookmark-alist.el (13.237991019999999 6 2.403892035) (tv/pp-region (point-min) (point-max)) nil test-bookmark-alist.el (14.853056353 6 2.705511935) (tv/pp-region (point-min) (point-max)) t test-bookmark-alist.el (28.059658418 5 2.005039257) (pp-region (point-min) (point-max)) t test-bookmark-alist.el (180.390204026 5 2.1182066760000002) (pp-region (point-min) (point-max)) nil test-bookmark-alist.el (265.95429676599997 10 4.445954908) (pp-buffer) t test-bookmark-alist.el (268.975666886 10 3.6774180120000004) (pp-buffer) nil Here we see that my `pp-region` code can be faster (even significantly so) than `pp-buffer`. I'm not sure why, but I'll take the win :-) We also see that the faster `lisp-ppss` again makes an important difference in the performance of `pp-region` (~8x), tho the effect is not as drastic as in the case of `foo.el`. test-load-history.el (235.134082197 8 4.440112806999999) (tv/pp-region (point-min) (point-max)) nil test-load-history.el (235.873981253 8 4.416064476) (tv/pp-region (point-min) (point-max)) t test-load-history.el (506.770548196 31 9.706665932) (pp-region (point-min) (point-max)) t test-load-history.el (701.991875274 43 12.390212449) (pp-buffer) t test-load-history.el (710.843618646 43 12.156289354) (pp-buffer) nil test-load-history.el (1419.268184021 36 9.260999640000001) (pp-region (point-min) (point-max)) nil Here again, we see that `pp-region` is competitive with `pp-buffer` and the `lisp-ppss` patch speeds it up significantly (~3x). Another thing we see in those tests is that `pp-region` (with the `lisp-ppss` patch) is ~2x slower than `tv/pp-region`, whereas the performance differential with `pp-buffer` varies a lot more. Also if we compare the time it takes to the size of the file, we see: 77101B / 7.932661181s = 9719 B/s 163258B / 28.059658418s = 5818 B/s 1125154B / 506.770548196s = 2220 B/s `pp-region`s performance is not quite linear in the size of the file :-( Interestingly, the same holds for `tv/pp-region`: 77101B / 4.62344853s = 16676 B/s 163258B / 13.237991019s = 12332 B/s 1125154B / 235.134082197s = 4785 B/s even though it works in a fundamentally very different way (which, to my naive eye should result in a linear performance), so maybe the slowdown here is due to something external (such as the fact that various operations on buffers get slower as the buffer gets bigger). > hmm, don't know, I ran pp-buffer with M-: from the test-load-history.el or the > test-bookmark-alist.el buffer. May be using emacs --batch makes a > difference? I don't see any significant performance difference between batch and non-batch :-( > is the data really printed in such case? Yes, definitely. > More or less the code using pp-region takes between 42 to 48s and the one > with old pp-buffer around 6s. I wonder why we see such wildly different performance. In my tests on your `test-bookmark-alist.el` I basically see the reverse ratio! > Also sorry about your last patch I tested it too fast, it is indeed > slightly faster, but not much: 42 vs 46s. This is also perplexing. In my tests, the patch has a very significant impact on the performance of `pp-region`. Are you sure the patch is used (`lisp-mode.el` is preloaded, so you need to re-dump Emacs, or otherwise manually force-reload `lisp-mode.elc` into your Emacs session)? FWIW, I'm running my tests on Emacs's `master` branch with the native ELisp compiler enabled (tho I don't see much difference in performance on these tests when running my other Emacs build without the native compiler) on an i386 Debian testing system. >> And do I understand correctly that `tv/pp-region` does not indent its >> output? > No, it does indent, see function tv/pp which use pp-to-string which use pp-buffer > and pp-buffer indent the whole sexp at the end. AFAICT `tv/pp` uses `pp-to-string` only on "atomic" values (i.e. not lists, vectors, or hash-tables), so there's usually not much to indent in there. What I see in the output after calling `tv/pp-region` are non-indented lists. >> What was the reason for this choice? > Because indentation is done at the end by pp-buffer. When I use `describe-variable` with your code, the printed value is indeed indented, but that code uses only `pp-buffer` and not `tv/pp-region` (i.e. `tv/describe-variable` does not call `tv/pp-region`, neither directly nor indirectly). > PS (unrelated to pp-region): About the old pp-buffer, there is > a difficult to find bug where the same operation is running twice > (newline is added, then removed, then added again and then the loop > continue) > > You can see it by edebugging pp-buffer on a simple alist like this: > > (defvar bar '((1 . "un") (2 . "deux") (3 . "trois") (4 . "quatre") (5 . "cinq") (6 . "six"))) That might be part of the poor performance we see on `test-bookmark-alist.el`? Stefan