From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.devel Subject: Re: Async IO and queing process sentinels (was: Concurrency via isolated process/thread) Date: Tue, 18 Jul 2023 12:49:51 +0000 Message-ID: <87cz0p2xlc.fsf@localhost> References: <871qhnr4ty.fsf@localhost> <87pm57p57m.fsf@yahoo.com> <83mt0bjdh5.fsf@gnu.org> <03724893-5b8e-2df5-6001-f2616c8f5c1c@hugot.nl> <87r0p53i58.fsf@localhost> <87fs5l2y37.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36438"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org, Michael Albinus To: Hugo Thunnissen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Jul 18 14:50:35 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qLk9z-0009Eb-Hi for ged-emacs-devel@m.gmane-mx.org; Tue, 18 Jul 2023 14:50:35 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qLk9E-0000i1-Fd; Tue, 18 Jul 2023 08:49:48 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qLk9D-0000hk-8A for emacs-devel@gnu.org; Tue, 18 Jul 2023 08:49:47 -0400 Original-Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qLk9A-0007IX-UM for emacs-devel@gnu.org; Tue, 18 Jul 2023 08:49:46 -0400 Original-Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 0EFB3240029 for ; Tue, 18 Jul 2023 14:49:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689684582; bh=Qxnd+4vfblnvKbOMJ7V8W5TC/QqXeqg++7f76O3jWjo=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=DOazLSzkE3aGMFvXMRdC69PgLQlPtxXBZaddkyqNCxqF0cvjyMp4rOsOCwny1iYa4 GrTXGp2MScnTo1WKHrKRcmLwz4pz9TwVSZSM85VPjpWVtCGzLoodHfTjOjf6Yoxx6+ gnkYaPagDBAEI2hEkstyXP9gcXUi0Nb6gY2QdsyK1GXJGot5f1lv7Z48SGwpZvTTnW 86WUr01j1lTnT0s0VW8WAt2g/hWvlxHCfeqasPyhFsNeBbTjfdxBRO2GlCp7V5TCrb 4hT0/6tJWh89BkvfbWVzAeJkRe/eO/0Ujz93O3m2knEGS+hERpRvXyd4aqNbzeOW30 8qPAfZcmYHT7A== Original-Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R4zLF1CY1z9rxL; Tue, 18 Jul 2023 14:49:40 +0200 (CEST) In-Reply-To: <87fs5l2y37.fsf@localhost> Received-SPF: pass client-ip=185.67.36.65; envelope-from=yantar92@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:307954 Archived-At: Ihor Radchenko writes: > I also did testing locally, on my $HOME dir, and got similar results > with recursive calls taking most of the CPU time: > > 463 29% - directory-files-recursively > 434 27% - directory-files-recursively > 305 19% - directory-files-recursively > ... > > If you want to improve performance here, you likely need to rewrite > `directory-files-recursively' without recursion. Doable, and nothing to > do with IO slowness. Not recursion, actually. I did a bit more elaborate profiling with perf and it looks like most of the time is spent matching regexp against file names: The actual tested command was (ignore (directory-files-recursively "/home/yantar92/.data" ".+")) 33.82% emacs emacs [.] re_match_2_internal 14.87% emacs emacs [.] process_mark_stack 8.47% emacs emacs [.] Fnconc 6.07% emacs emacs [.] re_search_2 2.36% emacs emacs [.] unbind_to 2.08% emacs emacs [.] sweep_strings 1.98% emacs emacs [.] compile_pattern 1.64% emacs emacs [.] execute_charset 1.64% emacs emacs [.] assq_no_quit 1.40% emacs emacs [.] sweep_conses 1.01% emacs emacs [.] plist_get 0.97% emacs emacs [.] set_buffer_internal_2 0.86% emacs emacs [.] RE_SETUP_SYNTAX_TABLE_FOR_OBJECT 0.84% emacs emacs [.] mark_interval_tree_1 0.75% emacs emacs [.] internal_equal 0.73% emacs emacs [.] Ffind_file_name_handler There was quite a bit of GC (not included into Elisp profile), that shows up as *mark* and *sweep* calls. No IO at all shows up in the backtrace. Even Ffind_file_name_handler is simply matching filename against `file-name-handler-alist', calling `insert-directory-program' somewhere in the process (the call does not show up anywhere high in the profile). Most of the time is spent doing regexp matching in various places and building the actual (long) list of files. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at