From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrew Cohen Newsgroups: gmane.emacs.bugs Subject: bug#64391: 30.0.50; buffer narrowing slowdown regression in emacs 29 Date: Sat, 01 Jul 2023 08:15:25 +0800 Message-ID: <877crkcwrm.fsf@ust.hk> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="20232"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Gregory Heytings To: 64391@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Jul 01 02:21:39 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qFOMr-00053f-Ky for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 01 Jul 2023 02:21:38 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qFOMV-0003SJ-BS; Fri, 30 Jun 2023 20:21:15 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qFOMJ-0003Nq-37 for bug-gnu-emacs@gnu.org; Fri, 30 Jun 2023 20:21:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qFOMI-0004x1-PO for bug-gnu-emacs@gnu.org; Fri, 30 Jun 2023 20:21:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qFOMI-0006Wh-EV; Fri, 30 Jun 2023 20:21:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Andrew Cohen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org, bugs@gnus.org Resent-Date: Sat, 01 Jul 2023 00:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 64391 X-GNU-PR-Package: emacs,gnus X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.168817085225055 (code B ref -1); Sat, 01 Jul 2023 00:21:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 1 Jul 2023 00:20:52 +0000 Original-Received: from localhost ([127.0.0.1]:55990 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qFOM7-0006W2-SK for submit@debbugs.gnu.org; Fri, 30 Jun 2023 20:20:52 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:45134) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qFOM5-0006Vp-0T for submit@debbugs.gnu.org; Fri, 30 Jun 2023 20:20:50 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qFOLu-0003FB-5p for bug-gnu-emacs@gnu.org; Fri, 30 Jun 2023 20:20:39 -0400 Original-Received: from mail-tycjpn01on20718.outbound.protection.outlook.com ([2a01:111:f403:7010::718] helo=JPN01-TYC-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qFOLr-0004jG-OW for bug-gnu-emacs@gnu.org; Fri, 30 Jun 2023 20:20:37 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X7DaTMMktfQXG50wmiteM4MWshiclsekFgj1V14izVkeKywmEqpiREwUa7XTrx4EB1pxj3gX/Z4IP0IhXgVZU3Jx7WTCjqTnlD8/i8W0sCHF0mYhE4vInm9HRHrZKtONZh46Ah6bVDryu1B+0e9xSdUhDtHmt00O40iMfUpOc1ndGHaSDObWSW6VyGHX5cfiDJJi6o7EURAxLvIFn3YU9j/adj8epGX7lT5AiL6xghNgrBKP9VHvAKE/VDbIs7jX2W6uQZTxBhihwgat5zBku9vGrcikhXbPAD8KnopPPOIs+uArsBGoLxk/aPc/fsMmT1iK4Yk80hM6xQvjYhjsVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XWlBs6nHF4bbgSsIGbncIRe7MbtENRoWDQWDVECFKkM=; b=kaFutaVVg/681/ju1IpkGThXPTr2r5LgdU0P6WdMmmPoBXDktjD7JbLfzaSX2CKeMqxRGbrOjNz66/fkXiCSvok2o7SXO30AruBKnFjVBxgosZgIXVj68YgyZC6GkcyaTONwdDCq1tb+ey+A/WEyV/Uc07uYryA+rDdG4YhTqhgXuy9fs3hwRRddBlfV9LTFl9Xc09og8y8t+2jx4lNcrBg8XsH98XYU6xRvHKxI8V2t700NL6xWIHY/JxhvTxY/Z975hEyyd9kUC4c//p3IbgA+JSrGCQ1qm3FQtumshFA3PSGGEeUjvMYd4eDt647KUgyLidl13oxQ73jrxV88dA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ust.hk; dmarc=pass action=none header.from=ust.hk; dkim=pass header.d=ust.hk; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ust.hk; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XWlBs6nHF4bbgSsIGbncIRe7MbtENRoWDQWDVECFKkM=; b=auuMXpYfi88baCPw5/hoIx879LP9OvJROfP2OLKmbNuJP+LKphJFHVJr1btro/GfgNwzVkc90Ieo1HTce6j+VSiCDAJSJbIH4t7STitPdu94FlAekYx8Ls3izhlB8JQZs3wQql3O56dDUBL+nxxJsLxxb2Cp+OJqo4RNPRgm51tuiFWGTOrFymYTk2e1DhQOFKPaCjgzko2lsp6ZI3f1sD4/nmF/QgQDSndnbAH138UcGFyQvQNUry66VOepADgZT2gSAgIDiCKTrlhqo6IAhqL1MrsaQ4mwxWmsLJvzru/cKJUwr4HPBwKwx3FknnwEKpMTrlC2stYPIcqJzLZ5hQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ust.hk; Original-Received: from OS3P286MB1877.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:1bf::11) by TYWP286MB2037.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:162::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.19; Sat, 1 Jul 2023 00:15:28 +0000 Original-Received: from OS3P286MB1877.JPNP286.PROD.OUTLOOK.COM ([fe80::f9dd:5f78:1a2e:5b72]) by OS3P286MB1877.JPNP286.PROD.OUTLOOK.COM ([fe80::f9dd:5f78:1a2e:5b72%6]) with mapi id 15.20.6544.024; Sat, 1 Jul 2023 00:15:28 +0000 X-ClientProxiedBy: PS2PR03CA0016.apcprd03.prod.outlook.com (2603:1096:300:5b::28) To OS3P286MB1877.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:1bf::11) X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: OS3P286MB1877:EE_|TYWP286MB2037:EE_ X-MS-Office365-Filtering-Correlation-Id: cb94249f-8a94-4904-6896-08db79c84613 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MbqAgZDYoaBX3kMe9B71apd39BFnuq8+5b9yLJ8Wou/6hoAANdYG44dBZhcufc941uLimE4ZhWmZotg2cG+LbDofpopw8c4mknAYvPDeTN5+hp4NiJX9xl2pETNpI6VufhdD06fuNhjQwHTEZisAgU23K3etukfxMg44ba/B4NSDF2CDjSef/m2CFE3970qvqNojT9iFSe/7AYwkoeQrR82YQ0k8FiCr4Zl5WHQCjo5shY5mAK47YzqhXac9VoCSdqDQu3x+m+8G6uYGssal8tL343EbE7DXDZ41w3eacTnVV5PEftNre9gh6jtX4Ls5g8fET7Zslc9mDG4qlwv0dI7nCh3GjtEl/2HRH8jlXaJpcMbIHmifWizCDeirahonbeOKOTebOnezUUawCVe2HsRQyfAJr9RR/p1xUpDpUO+oqtH5Wg3my4i7HQqqyEdVLmo8oRRjrRkCWleCmUtZ7UzUSDjsw14lIERf5NSMDnTUPJoDq1Mdu0KEIoYRc8mvVDCoVfHCa6TXz7/NAdIWNkLI4n3fVVKemxcqw1TnsMlKMx7errzGTRR8e4lTz1is X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:OS3P286MB1877.JPNP286.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230028)(4636009)(376002)(346002)(39860400002)(366004)(396003)(136003)(451199021)(2616005)(38100700002)(6666004)(86362001)(6512007)(2906002)(786003)(316002)(6486002)(33964004)(36756003)(66556008)(66476007)(66946007)(4326008)(6916009)(478600001)(186003)(6506007)(83380400001)(41300700001)(5660300002)(8936002)(8676002)(235185007)(26005)(66574015); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: n2mcpWQ3xycioONmn4KnNnJyYCIF82/xJPo6rQOS3PGPf3d53xKZu3mLl22OBO/Sf4UclrIy4yIMHowBQzI9HmTAy+TObLsMXMkvvGqVKusb7aI5J6DME3SH8iMbQ7Ivjm0K6GYJzeV4iZUbL3EGR430t4k4OCmUTOB7TtuvA6A2drNsMA2bjyd8BbmUU4g2f9usWXwYkikB9sBGTCPWuVjZokPywduHgxelrma/7FLP9XQ8hqZ9fNLKPmOjRWKP6d24Sg/R5SeYJhX4h1WjDod2zmC+6VvffEbnWyj3n3oA7toGEYVwBqRWf1lktovfnaewmPwVERk/SXx67xhHIb7+DXew25WM6Y7k7MLFp65dDPAILvEPk60ba/rE2ltEBNSvwW2SmXsRJ3J38wWpFAH7yDRmmWdQ1AIG2QiUlMZWSJh1eJkMDEyFNltucXrCJdfG3JVZ7xYih62tIjQuYNLtIA9eN1f0DaOy5TsFQHQAxHpgfqiZowJCsBDQi+d/uCCPf71z4nGsKSRyyd3+s9y/e/liwN68N0RGL7sOnhh1xU8S2JCt7L84nPFM6WJXH7OHDZR8ShbMcQjSd3CuK4MbfsVdPWFY78C/8yKO74/rVS3qN4Tv4v9HmbH5dsG1IlUL1EqAUs6JgmpSTX2udmdXQK1n7AvJl2T0PRIItn0qgEX+12dTIuOsHxoFsrPm2bzSTPRUvVx14zDMis+mD+6vJoKN5pnB17NPo6IvBsGJfk4La1XPIhnJlz PHfRMGmOKpEIoJACOt+InqNpqGKAZ4AXLAbh0qcQy5HvzSB+GjF4omMQbE14RLpfXCvm7dGa3zKUxje/Xe35wVQfhPr0WNJ8ql X-OriginatorOrg: ust.hk X-MS-Exchange-CrossTenant-Network-Message-Id: cb94249f-8a94-4904-6896-08db79c84613 X-MS-Exchange-CrossTenant-AuthSource: OS3P286MB1877.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2023 00:15:28.2154 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: c917f3e2-9322-4926-9bb3-daca730413ca X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1npYbivs82lmOS58TSJmzOKybSyYaUbn8Nj6g5tRM5Oy3xiy3ZznfHGrXSnbMdJf X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYWP286MB2037 Received-SPF: pass client-ip=2a01:111:f403:7010::718; envelope-from=acohen@ust.hk; helo=JPN01-TYC-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:264344 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I have noticed a huge slowdown in parsing email headers in gnus. After some debugging with help from Mattias Engdeg=C3=A5rd, the problem has been traced to the narrowing code introduced in commit ba9315b1641b483f2bf843c38dcdba0cd1643a55 (HEAD) Merge: aef803d6c3d a3b654e069e Author: Gregory Heytings Date: Thu Nov 24 14:21:30 2022 +0100 Merge master into feature/improved-locked-narrowing. Gnus populates a buffer with headers from a set of email messages and then parses them by: narrowing the buffer to the headers for an individual message; parsing the headers; widening; and then repeating for the next message. As part of the parsing the headers are "unfolded" so that each header doesn't include line breaks. I noticed that for a long list of messages (10,000) this takes between one and two orders of magnitude more time in Emacs 30 than in Emacs 28. Unfolding all the headers in the full buffer before the parsing process removes most of the slowdown. The slowdown seems to grow quadratically with the size of the buffer. The problem seems fairly general and Mattias has produced a simple test case to demonstrate the issue (code at the end of this message): 1. Create a buffer with 100,000 lines each containing two characters "ab". 2. Loop through the buffer narrowing to each line, and immediately widening back to the full buffer (so no change is made to the buffer contents). 3. Loop through the buffer removing the first character of each line. This takes a very long time compared with reversing the order of 2. and 3. Emacs 28.2 Modify after narrowing: 0.173 s elapsed, 0 GCs, 0.000 s GC, 0.173 s non-GC Modify before narrowing: 0.160 s elapsed, 0 GCs, 0.000 s GC, 0.160 s non-GC Emacs 29.0.92 Modify after narrowing: 4.454 s elapsed, 9 GCs, 0.115 s GC, 4.339 s non-GC Modify before narrowing: 0.291 s elapsed, 9 GCs, 0.114 s GC, 0.177 s non-GC The problem may be the creation of markers in the narrowing process that slows down further mutation. (Also note that the new code has introduced some significant GC that was previously absent). --=-=-= Content-Type: application/emacs-lisp Content-Disposition: attachment; filename=abench2.el Content-Transfer-Encoding: quoted-printable Content-Description: Demonstrate narrowing slowdown ;;; -*- lexical-binding: t -*- (require 'cl-lib) (defconst abench-lines 100000) (defun abench-run (modify-before) (let ((buffer-undo-list t)) (when modify-before ;; Modify all lines in one sweep without narrowing (goto-char (point-min)) (while (not (eobp)) (delete-char 1) (forward-line))) ;; Narrow around each line but do not modify (goto-char (point-min)) (while (not (eobp)) (narrow-to-region (point) (line-end-position)) (widen) (forward-line)) (unless modify-before ;; Modify all lines in one sweep without narrowing (goto-char (point-min)) (while (not (eobp)) (delete-char 1) (forward-line))))) (defun abench (modify-before) (with-temp-buffer ;; Populate buffer (insert (mapconcat #'identity (make-list abench-lines "ab\n") nil)) ;; Run operation (garbage-collect) (let ((gcs0 gcs-done) (gc-e0 gc-elapsed) (t0 (float-time))) (abench-run modify-before) (let ((dt (- (float-time) t0)) (gcs (- gcs-done gcs0)) (gc-time (- gc-elapsed gc-e0))) (cl-assert (equal (buffer-string) (mapconcat #'identity (make-list abench-lines "b\n") nil))) (format "%.3f s elapsed, %d GCs, %.3f s GC, %.3f s non-GC\n" dt gcs gc-time (- dt gc-time)))))) (princ (format "Emacs %s\n" emacs-version)) (princ "Modify after narrowing: ") (princ (abench nil)) (princ "Modify before narrowing: ") (princ (abench t)) --=-=-= Content-Type: text/plain -- Andrew Cohen --=-=-=--