From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Arthur Miller Newsgroups: gmane.emacs.devel Subject: Experiment with threads - no concurrency? Date: Mon, 06 Sep 2021 16:35:04 +0200 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="839"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Sep 06 16:36:37 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mNFjg-000AYM-Cp for ged-emacs-devel@m.gmane-mx.org; Mon, 06 Sep 2021 16:36:37 +0200 Original-Received: from localhost ([::1]:48674 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mNFjf-0003NE-4l for ged-emacs-devel@m.gmane-mx.org; Mon, 06 Sep 2021 10:36:35 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:47754) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mNFiL-0002GL-Dm for emacs-devel@gnu.org; Mon, 06 Sep 2021 10:35:13 -0400 Original-Received: from mail-oln040092072066.outbound.protection.outlook.com ([40.92.72.66]:32833 helo=EUR03-VE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mNFiG-0003RJ-MD for emacs-devel@gnu.org; Mon, 06 Sep 2021 10:35:13 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kJlPpNTf/RFG4RRNTp2ieyDFHYLm1nP+gSFVtCmWGTsj7z0jYkWsrOO9Vfq//dSIA0zFzUXZIKXrSbcialbWIj3OAwA3NJhjYQjJ4Udzf+b2oygwGVvEmLNFz3j9kbAKekwFz0+JKEJ+DR3xrobZD0W8Bgws+4xdR6SYKuiclV44oPNfxp0AeTavLdK+Bn3aHnHJ87WL5bstEcUrf9de9ELtUt1oJH7d78dT0ox5upPc5pNwgXkNtRLYlt8Qt1iOYSu21yr2HAswo9JMK8ryf3XKF2HsMcSYjmRWjGUE6RT7VM4huRni00ae/eQzStLFKvNayJgbpKDdJOvHsWvIug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=h/Ip9S9WLba04TfF3naufT7G9dWGuGQHUjvB4vMB3Zk=; b=MSJ3H2NGI660z7XVfv45xGWCeq04twUWjFZ7c3G5eBKMDIFJyy4CnT94IOThzwB12nQyqBekdY9u/FeLQygCu9Xhmnh3brO5XBDAxyDY5AIQ4oH23MdxqAG+K+mdQI45zjBKxEZzACkv4n5KVEvWjwB5+1zJ52tq9zcxR1a8HWDqyLk26aNcWVAPY7bNqb304mzHwgvFwFCxONnlDqnfalk1qD9phIkm6fVRlG+m5p3IuHMSCR4cisqTqxLDldw76FuIsurzLmHjy2lhoySMqRHzvtbXighhkZNgftYt6b/0PboxE7poulZ+KTwsMVan4eOQihIu6067eWBo5Z+ODQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=live.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h/Ip9S9WLba04TfF3naufT7G9dWGuGQHUjvB4vMB3Zk=; b=KZJ/0pcCqK0/XPDA3oLe+ju9t/LyH5pUtCX3LYHurGtMl0TPh1tsI7sstmX3J698O/6HM8mQgjLPcjKb/liLZ8F3OVPjp9xgDYpve1pIEaotWoCKngOHS4UVSOGkp50Bvbg8j5hMjplrws+2iJWQTLNFlbt/D/uosHRFvUFdVS5NuPS0JRKbZkxKu1mxOMjHUQo0Asv+zKhhdUSogjclR9QLbRRQ/KXHaFkADfu4k2ZleJOgu+ZhW2j1WZy+2IunlyzNjTVXqsZvc3OsUtuGUvrHuqiu2sKyf5RKan4IMmvCEqvnmWzF5mMHeDNguBZ972K6ArmMi58X1MLSYUOCtg== Original-Received: from DB9PR09MB4986.eurprd09.prod.outlook.com (2603:10a6:10:2a9::19) by DB6PR0902MB1766.eurprd09.prod.outlook.com (2603:10a6:4:b2::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19; Mon, 6 Sep 2021 14:35:05 +0000 Original-Received: from DB9PR09MB4986.eurprd09.prod.outlook.com ([fe80::80be:d528:d357:5d3a]) by DB9PR09MB4986.eurprd09.prod.outlook.com ([fe80::80be:d528:d357:5d3a%7]) with mapi id 15.20.4478.025; Mon, 6 Sep 2021 14:35:05 +0000 X-TMN: [DNmiDDda8KEnRgx7nf3loayvl2uXVm0s] X-ClientProxiedBy: AM5PR0301CA0014.eurprd03.prod.outlook.com (2603:10a6:206:14::27) To DB9PR09MB4986.eurprd09.prod.outlook.com (2603:10a6:10:2a9::19) X-Microsoft-Original-Message-ID: <877dftvkkn.fsf@live.com> X-MS-Exchange-MessageSentRepresentingType: 1 Original-Received: from pascal.homepc (81.232.177.30) by AM5PR0301CA0014.eurprd03.prod.outlook.com (2603:10a6:206:14::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 14:35:04 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 45a196a3-86c2-49ed-31f6-08d97143845d X-MS-TrafficTypeDiagnostic: DB6PR0902MB1766: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0FF2ddOJyuq+I0eNxRU+hBZv3tBmXRGoqN4GyAmwBStzlFSHD5nU8mI6X2xSd2vRpsUA6VjLqcjRM0i73yczA1HSj++DnNfNrQgvgaZuTJULoZzO2UvFtUfTIMx7nY8zvr6Fal4+7blT1bxI+v4Q5kI7xi5EAa2ijfkFSN1SKU6aA3AlWulDM2IUQxHndAqQ5Q0sHiKa9xfjzhYUP+FQ7D8gwHBDhJmQcPOe15i3vyqmwg47gnG76wGtl2Minm6MbLSx7vVqjecw8mVCrhIHQJcOGAvlnmmw9AHm56uhjhi7FVmbzBD3bIP9/DsFBE/l1nZMFUY7i0ggg/ptLrvOAH6zFZh/O8k/bTz+x1gXOVgc8bSHlanYqMkN6ny97hH3HA3TuVuu93Y6nb8ifq0Pbdx28DT9A9bvzXCFzkkzhl3f98B3aDPysTyvTg8M0qsM X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: j3fni63N3EKg/Lcxq6QpwCwkbKrg5Vd1DJI46lC+obw7rmCsqo6waTd3XW8lIlNdaziRJ077qMNUHxhknbwSioArlOXvtHR3MdDMps6hWjbRSvF0A6QWTSj+YVSbAPWVUpVigJ+Od1n+jSahGotyLw== X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-72e6e.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: 45a196a3-86c2-49ed-31f6-08d97143845d X-MS-Exchange-CrossTenant-AuthSource: DB9PR09MB4986.eurprd09.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 14:35:05.0528 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0902MB1766 Received-SPF: pass client-ip=40.92.72.66; envelope-from=arthur.miller@live.com; helo=EUR03-VE1-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:274128 Archived-At: --=-=-= Content-Type: text/plain I did a little experiment today. I wanted to count frequency of Emacs functions usage. So I started by indexing all functions and macros. Since it is kind of lot's of I/O and independent tasks to do, I thought threads could come handy. However I see no difference from sequential code. I tried different versions, one where I did a single call to directory-files-recursively, and than used one thread per file, to parse each file separately, in own thread, but it was the slowest version. When I used to code in Java long time ago, I used to see improvement in this kind of jobs due to I/O. Maybe I am missunderstanding Emacs threads here? In second version, I do two calls to directory-files-recursively, each in it's own thread and use only two threads to do all the work, one per list. It is slightly faster, but just slightly and still slower than pure sequential version. I have also inserted thread-yield statements so that all threads get chance to run, but I see very little difference. Measured difference is probably due to system fluctuation, rather than truly beacuse of thread scheduling (~0.1 sec on a job of ~10 secs). So, how to use threads in Emacs correctly? Is this me or is there just no benefit of current thread implementation in Emacs? Sorry I don't mean to be rude on thread implementation, but I am in doubt, should I just forgett about threads or is there something I missunderstand about how to use them correctly? The attached program is just about parsing files in emacs lisp dir for function and macro defs (no frequency counting). --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=func-freq.el ;;; func-freq.el --- -*- lexical-binding: t; -*- ;; Copyright (C) 2021 Arthur Miller ;; Author: Arthur Miller ;; Keywords: ;; This program is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see . ;;; Commentary: ;; ;;; Code: (defun ff-print (map) (maphash (lambda (k v) (print (format "%s: %s" k v))) map)) (defun ff-collect-funcs-and-macros (src &optional keyword) (with-temp-buffer (insert-file-contents-literally src) (goto-char (point-min)) (let (sxp features) (while (setq sxp (ignore-errors (read (current-buffer)))) (when (and (listp sxp) (equal (car sxp) keyword)) (push (cadr sxp) features))) features))) (defun ff-freq-seq (dir-tree) (let ((fmap (make-hash-table :test 'equal)) (mmap (make-hash-table :test 'equal)) (srcs (directory-files-recursively dir-tree "\\.el$"))) (dolist (src srcs) (dolist (f (ff-collect-funcs-and-macros src 'defun)) (puthash f 0 fmap)) (dolist (f (ff-collect-funcs-and-macros src 'defmacro)) (puthash f 0 mmap))))) (defun ff-freq-thr (dir-tree) (let (threads (fmap (make-hash-table :test 'equal)) (mmap (make-hash-table :test 'equal)) (srcs (directory-files-recursively dir-tree "\\.el$"))) (dolist (src srcs) (push (make-thread #'(lambda () (dolist (f (ff-collect-funcs-and-macros src 'defun)) (puthash f 0 fmap) ;;(thread-yield) ) (dolist (f (ff-collect-funcs-and-macros src 'defmacro)) (puthash f 0 mmap) ;;(thread-yield) ))) threads)) (dolist (thread threads) (thread-join thread)))) (defun ff-collect-macros (dir-tree map) (let ((srcs (directory-files-recursively dir-tree "\\.el$"))) (dolist (src srcs) (dolist (f (ff-collect-funcs-and-macros src 'defmacro)) (puthash f 0 map) ;;(thread-yield) )))) (defun ff-collect-functions (dir-tree map) (let ((srcs (directory-files-recursively dir-tree "\\.el$"))) (dolist (src srcs) (dolist (f (ff-collect-funcs-and-macros src 'defun)) (puthash f 0 map) ;;(thread-yield) )))) (defun ff-freq-thr2 (dir-tree) (let (fthr mthr (fmap (make-hash-table :test 'equal)) (mmap (make-hash-table :test 'equal))) (setq mthr (make-thread #'(lambda () (ff-collect-macros dir-tree fmap)))) (setq fthr (make-thread #'(lambda () (ff-collect-functions dir-tree mmap)))) (thread-join mthr) (thread-join fthr))) (benchmark-run 3 (ff-freq-seq (expand-file-name "~/repos/emacs/lisp"))) (benchmark-run 3 (ff-freq-thr (expand-file-name "~/repos/emacs/lisp"))) (benchmark-run 3 (ff-freq-thr2 (expand-file-name "~/repos/emacs/lisp"))) (provide 'func-freq) ;;; func-freq.el ends here --=-=-=--