From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: mail@daniel-mendler.de Newsgroups: gmane.emacs.bugs Subject: bug#46326: 27.1.50; Excessive memory allocations with minibuffer-with-setup-hook Date: Fri, 05 Feb 2021 13:51:41 +0100 Message-ID: <62c490ed0d8d24d8b259ac1ba55ea79e@mendler.net> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33705"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Roundcube Webmail/1.2-git To: 46326@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Feb 05 14:44:18 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1l81PE-0008c6-Hq for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 05 Feb 2021 14:44:16 +0100 Original-Received: from localhost ([::1]:38400 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l81PD-00026v-JQ for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 05 Feb 2021 08:44:15 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:58540) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l81P0-00024C-1n for bug-gnu-emacs@gnu.org; Fri, 05 Feb 2021 08:44:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:59893) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l81Oz-0000e1-Qm for bug-gnu-emacs@gnu.org; Fri, 05 Feb 2021 08:44:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1l81Oz-0004Gb-PK for bug-gnu-emacs@gnu.org; Fri, 05 Feb 2021 08:44:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: mail@daniel-mendler.de Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 05 Feb 2021 13:44:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 46326 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.161253259316327 (code B ref -1); Fri, 05 Feb 2021 13:44:01 +0000 Original-Received: (at submit) by debbugs.gnu.org; 5 Feb 2021 13:43:13 +0000 Original-Received: from localhost ([127.0.0.1]:43203 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l81OC-0004FF-OZ for submit@debbugs.gnu.org; Fri, 05 Feb 2021 08:43:13 -0500 Original-Received: from lists.gnu.org ([209.51.188.17]:54878) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l80ah-0004zg-8t for submit@debbugs.gnu.org; Fri, 05 Feb 2021 07:52:03 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:48264) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l80ag-0000v1-WC for bug-gnu-emacs@gnu.org; Fri, 05 Feb 2021 07:52:03 -0500 Original-Received: from server.qxqx.de ([2a01:4f8:121:346::180]:34612 helo=mail.qxqx.de) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l80ad-0002ow-Lp for bug-gnu-emacs@gnu.org; Fri, 05 Feb 2021 07:52:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Message-ID:Subject:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=eM7GNSWg9+40Hfnm9VXYFiK0z1bUjpEji/Aj6finu7Q=; b=gE7hrcPWvMPW4uc+oRJDEdeg9tREVfseHWdxUI1QZ19PTacsjmSjexHThl71xoy6nIlqz8uyTUE8B1qVYT+CJjP3w+51TS2xqI0njMtEmDAr3n6e47tC5W26vArZNyq59rwSInE30F1hIrUr6TI2eVBTLkfw6fYQWpqkXCeSWaA=; X-Sender: mail@daniel-mendler.de Received-SPF: pass client-ip=2a01:4f8:121:346::180; envelope-from=mail@daniel-mendler.de; helo=mail.qxqx.de X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 05 Feb 2021 08:43:11 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:199379 Archived-At: Dear Emacs developers, I have an issue on 27.1.50 with excessive memory allocations when using minibuffer-with-setup-hook with large closures and :append. This issue has been observed in my Consult package, which is similar to Counsel and which provides commands based on completing-read. Naturally I have to perform some setup in the minibuffer-setup-hook. Most of the memory allocations are due to add-hook and in particular the sort function. The sort functions is called because of :append which sets a hook priority. The reason seems to be that large closures are copied, but I didn't fully investigate the reasons for the issue. Example profile: 140,068,687 92% - consult-buffer 140,068,687 92% - consult--buffer 140,068,687 92% - consult--multi 138,933,527 91% - consult--read 138,933,527 91% - apply 138,933,527 91% - # 99,916,347 65% - consult--with-async-1 99,916,347 65% - # 99,912,203 65% - consult--with-preview-1 80,735,735 53% - # 80,735,735 53% - completing-read 80,735,735 53% - selectrum-completing-read 62,909,765 41% - selectrum-read >>> 58,826,659 38% + add-hook 1,077,206 0% + selectrum--minibuffer-post-command-hook .... >>> 19,176,468 12% + add-hook >>> 39,017,180 25% + add-hook The issue can be mitigated by using a modified version of minibuffer-with-setup-hook, where I am creating a symbol and fsetting instead of adding a lambda directly via add-hook. This technique is also used in set-transient-map in order to avoid problems with letrec-closures and add-hook/remove-hook. (defmacro consult--minibuffer-with-setup-hook (fun &rest body) (declare (indent 1) (debug t)) (let ((hook (make-symbol "hook")) (append)) (when (eq (car-safe fun) :append) (setq append '(t) fun (cadr fun))) `(let ((,hook (make-symbol "consult--minibuffer-setup"))) (fset ,hook (lambda () (remove-hook 'minibuffer-setup-hook ,hook) (funcall ,fun))) (unwind-protect (progn (add-hook 'minibuffer-setup-hook ,hook ,@append) ,@body) (remove-hook 'minibuffer-setup-hook ,hook))))) See the issue https://github.com/minad/consult/issues/193 for reference. Maybe there exists an alternative solution to the problem and the problem should be fixed on a different level than by replacing the minibuffer-with-setup-hook macro? Daniel Mendler In GNU Emacs 27.1.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.5, cairo version 1.16.0) of 2020-10-24 Repository revision: c847d5998f588dbf3eca5ea1ec573a2d64a97607 Repository branch: emacs-27 Windowing system distributor 'The X.Org Foundation', version 11.0.12004000 System Description: Debian GNU/Linux 10 (buster)