From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Raffael Stocker Newsgroups: gmane.emacs.bugs Subject: bug#67536: 29.1; Calc mode's math-read-preprocess-string conses unnecessarily Date: Wed, 29 Nov 2023 22:29:38 +0100 Message-ID: Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="454"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.10.8; emacs 29.1 To: 67536@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Nov 29 22:32:12 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1r8SAG-000Aaz-3c for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 29 Nov 2023 22:32:12 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r8SA0-0001R2-VH; Wed, 29 Nov 2023 16:31:56 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r8S9y-0001Qn-Vj for bug-gnu-emacs@gnu.org; Wed, 29 Nov 2023 16:31:54 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r8S9y-0006CM-N0 for bug-gnu-emacs@gnu.org; Wed, 29 Nov 2023 16:31:54 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1r8SA6-0003RM-Bd for bug-gnu-emacs@gnu.org; Wed, 29 Nov 2023 16:32:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Raffael Stocker Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 29 Nov 2023 21:32:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 67536 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.170129347313141 (code B ref -1); Wed, 29 Nov 2023 21:32:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 29 Nov 2023 21:31:13 +0000 Original-Received: from localhost ([127.0.0.1]:51624 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8S9J-0003Pt-Co for submit@debbugs.gnu.org; Wed, 29 Nov 2023 16:31:13 -0500 Original-Received: from lists.gnu.org ([2001:470:142::17]:44922) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r8S9G-0003KJ-Uu for submit@debbugs.gnu.org; Wed, 29 Nov 2023 16:31:11 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r8S90-0001Af-UO for bug-gnu-emacs@gnu.org; Wed, 29 Nov 2023 16:30:56 -0500 Original-Received: from mail-out.m-online.net ([212.18.0.9]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r8S8y-0005wG-Nq for bug-gnu-emacs@gnu.org; Wed, 29 Nov 2023 16:30:54 -0500 Original-Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4SgXY75BPbz1qsP0 for ; Wed, 29 Nov 2023 22:30:19 +0100 (CET) Original-Received: from localhost (dynscan1.mnet-online.de [192.168.6.68]) by mail.m-online.net (Postfix) with ESMTP id 4SgXY74DmBz1qqlS for ; Wed, 29 Nov 2023 22:30:19 +0100 (CET) X-Virus-Scanned: amavis at mnet-online.de Original-Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.68]) (amavis, port 10024) with ESMTP id PW3vS2HFt88T for ; Wed, 29 Nov 2023 22:30:18 +0100 (CET) X-Auth-Info: 8TwOdn/C5t4VAAYPUyY3SkDRDhI0qO1AfYuFiJpGgGyjfCoAIfhAAcD8PNQt4Ssm Original-Received: from Whiteflame (ppp-212-114-182-133.dynamic.mnet-online.de [212.114.182.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA for ; Wed, 29 Nov 2023 22:30:18 +0100 (CET) Received-SPF: pass client-ip=212.18.0.9; envelope-from=r.stocker@mnet-mail.de; helo=mail-out.m-online.net X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:275260 Archived-At: Org table re-calculation is very slow, partly due to math-read-preprocess-string of calc mode consing unnecessarily. For example, in one (large) table, I get the following memory usage from the profiler: ... 60,252,646 96% - org-ctrl-c-ctrl-c 60,248,166 96% - org-table-calc-current-TBLFM 60,216,431 96% - funcall-interactively 60,205,119 96% - org-table-recalculate 49,094,651 78% - org-table-eval-formula 32,624,631 52% - calc-eval 32,624,631 52% - calc-do-calc-eval 32,620,487 52% - calc-do-calc-eval 32,611,151 52% - math-read-exprs 29,388,838 47% + math-read-preprocess-string 2,343,257 3% + math-read-expr-list ... The reason for the slow-down seems to be that math-read-preprocess-string conses a lot, keeping the GC busy. This is due to heavy use of replace-regexp-in-string in this function: (defun math-read-preprocess-string (str) "Replace some substrings of STR by Calc equivalents." (setq str (replace-regexp-in-string (concat "[" math-read-superscripts "]+") "^(\\&)" str)) (setq str (replace-regexp-in-string (concat "[" math-read-subscripts "]+") "_(\\&)" str)) (let ((rep-list math-read-replacement-list)) (while rep-list ;; consing like a mad-man here: (setq str (replace-regexp-in-string (nth 0 (car rep-list)) (nth 1 (car rep-list)) str)) (setq rep-list (cdr rep-list)))) str) I would like to propose using a temp buffer instead of kneading the string into submission: (defun math-read-preprocess-string (str) "Replace some substrings of STR by Calc equivalents." (with-temp-buffer (insert str) (goto-char 0) (while (re-search-forward (concat "[" math-read-superscripts "]+") nil t) (replace-match "^(\\&)")) (goto-char 0) (while (re-search-forward (concat "[" math-read-subscripts "]+") nil t) (replace-match "_(\\&)")) (goto-char 0) (let ((rep-list math-read-replacement-list)) (while rep-list (while (re-search-forward (nth 0 (car rep-list)) nil t) (replace-match (nth 1 (car rep-list)))) (goto-char 0) (setq rep-list (cdr rep-list)))) (buffer-string))) With this replacement, the profiler shows much less memory usage on the same org table: ... 30,411,804 91% - org-ctrl-c-ctrl-c 30,407,324 91% - org-table-calc-current-TBLFM 30,363,932 91% - funcall-interactively 30,331,900 91% - org-table-recalculate 20,430,223 61% - org-table-eval-formula 6,751,852 20% + org-table-justify-field-maybe 4,598,523 13% - calc-eval 4,586,091 13% - calc-do-calc-eval 4,569,619 13% - calc-do-calc-eval 4,547,971 13% - math-read-exprs 2,297,453 6% + math-read-expr-list 1,347,377 4% + math-read-preprocess-string 7,296 0% math-read-token ... Only I am not sure the replacement is correct in all possible edge cases (or whether it uses so much less memory because I have overlooked something). What do you think? Can a calc-mode expert weigh in here? Regards, Raffael