From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Arthur Miller Newsgroups: gmane.emacs.devel Subject: Re: Helpful in Emacs? Date: Fri, 10 Sep 2021 02:52:25 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5550"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: Emacs developers , me@wilfred.me.uk, Stefan Kangas , Daniel =?utf-8?Q?Mart=C3=ADn?= To: Dmitry Gutov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Sep 10 02:53:27 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mOUnH-0001ID-11 for ged-emacs-devel@m.gmane-mx.org; Fri, 10 Sep 2021 02:53:27 +0200 Original-Received: from localhost ([::1]:33254 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mOUnF-0005lj-VU for ged-emacs-devel@m.gmane-mx.org; Thu, 09 Sep 2021 20:53:25 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59468) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mOUmQ-00055n-H6 for emacs-devel@gnu.org; Thu, 09 Sep 2021 20:52:34 -0400 Original-Received: from mail-oln040092075065.outbound.protection.outlook.com ([40.92.75.65]:25575 helo=EUR04-VI1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mOUmL-0007qG-Nh for emacs-devel@gnu.org; Thu, 09 Sep 2021 20:52:33 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=E3HXB5dJJHDyLWT9ndZSkyGAgtZrASq3Zk1zeUizppahcJdUsDS8U4+LNZXxYzKWswi7ExJEWlm7IEucMioRjFVsFyErhT+7wWO/ga+SnxNjAdRarQGy+V8DMdwe1PmFr0IJ4jkqwsWjBWcknjRurubRM2yskZ6vdDid+2GHIxZTBZ2UWgaAYmzRDUv6cU1lJRpi/TXrFPoNY9pgH7uj0PFBgIuB+PSdFXpAjIKfekppsm/xq7q9T/N/opMQUbkIM20EORDc7DWBeyZrafOnkEgxEMCr0Mc/3tQ0pgez/UcFXqVAvLxM7YOor4+9FR42yz2z4WDpoCdJmieLPktnYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=rSVQ5yxDnlqYqHnpRHfCDl3MELNeH8QgMA50Yykc3/g=; b=OSka2vpjXpNjhZggwYf+BsMQm+tZd19H7+kS6atEMktHXs3/qy2fMWb9iLaGzGttn3ZZF1rhPDjZQygVeA3j0Me9x7n6RUVyKcnkLUUr5m0d+9qzOY0xxxYKeSfaU/E+QmdT8xHpn7ZZVjH2RsMFM3Ln92uzW/xR19W/obkt8q5fksXY1sln7bZy6Xn5UIiiXe1aYGHRwPqek83tq28Fc2YoXo1MB0k3PWXkBhmyogAu8SDVfe+1y1CfjhMraaiqyUyoZYGt0dta55iic6d7bwo8rcFeuD0wB+zXVn8nTkgnLQooZ4dCcUF0g9xdUrUgX8lqm3juv7ITAwP50NE2Pg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=live.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rSVQ5yxDnlqYqHnpRHfCDl3MELNeH8QgMA50Yykc3/g=; b=UalSBNV+7DqYKLqCoNPzB2D9VTH+l7z1kgNxvarVlUz57kLrTDuMHk4fKKLnFKaqmgADge6+XO3u9gUO99DQwf4IjvyzrGzq+w9J6Bk6b1AgslULsifQn91Lil8KLxNtzHZYaRYyaZiMSRzrFn5M1Y/mFb9MHhCzZjqEOnwCBgyL5kYcEuNM5M9NMncyd0cHGbKmXfzkdXI9awaKPInTH0cfC5HdaK2UJiUs445piNXBii7oQm66g8xbfoSrLw5stInFSR+DmJZoghPhsiR7e7o9FEgvUzk9ieHBEXtqdIomr+zd5Oza10rq7MCNLcj38KFzPxLjrGHs+E86FeekZQ== Original-Received: from AM9PR09MB4977.eurprd09.prod.outlook.com (2603:10a6:20b:304::20) by AM4PR0902MB1764.eurprd09.prod.outlook.com (2603:10a6:200:95::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Fri, 10 Sep 2021 00:52:26 +0000 Original-Received: from AM9PR09MB4977.eurprd09.prod.outlook.com ([fe80::c55c:ece5:bed2:a9dc]) by AM9PR09MB4977.eurprd09.prod.outlook.com ([fe80::c55c:ece5:bed2:a9dc%8]) with mapi id 15.20.4500.017; Fri, 10 Sep 2021 00:52:26 +0000 In-Reply-To: (Dmitry Gutov's message of "Fri, 10 Sep 2021 01:55:08 +0300") X-TMN: [f5bHzENk6mV/YiikngRzv5+dcZdo+dRt] X-ClientProxiedBy: AS8P251CA0021.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:2f2::18) To AM9PR09MB4977.eurprd09.prod.outlook.com (2603:10a6:20b:304::20) X-Microsoft-Original-Message-ID: <87ilz9ntfa.fsf@live.com> X-MS-Exchange-MessageSentRepresentingType: 1 Original-Received: from pascal.homepc (81.232.177.30) by AS8P251CA0021.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:2f2::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Fri, 10 Sep 2021 00:52:25 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e07c6514-8c4c-4a64-607e-08d973f541cf X-MS-TrafficTypeDiagnostic: AM4PR0902MB1764: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iRwC8G7tdufjqvpOU+gzPOhndh6TfIhWrLUHHMXYcA6RJimcu6/RiLi/nRet8mUxkUsDFVJedM0vkAOyDcWuukZgfbSV3cQinXD53d7SPdlVw6qRJwzbRZKat5v62FzblbtkCRRkDZVrGu+rCdPrWgX3iW4v/u5gskimqJpsanfmrXEpMk4tDBusHHvQrnFF1wTrb5LoS0j75n4h7SB0BbBhcOdb4o+wYS6fBZfPothnCHDLa6phehfgWC3R3+1J+Ojs1+9084SX/vANLfmV1+odMaF0VrOLM+02PBCWUCFidJkZJXmASGAQvw3mLzIy4EwgFHcwI0xDBW5aqumPyYBJyXwUD4SQbPVh4p2w42hmWm4dBOGFvpehA9RDgByBZ/BAziOfJMVhxzLVSPS38EBiK/rmz/ciJMxMjoOSZF3hA8pZgVJ+c83IEZ8m7UOc X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: gfpcRvoUsUR3EJWnBeBppTr/OGC7GD3zu/iuNm0Q4pvsGWO9hI0ZHVk8h6Ouh9HD3X1swW0LQnCafVqIkii4POOkCIAr639o+6p25chErXsz/4XGVKwCQHe2eGY5mSz2DmJ2IIBC0eYENkIeczdfdw== X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-72e6e.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: e07c6514-8c4c-4a64-607e-08d973f541cf X-MS-Exchange-CrossTenant-AuthSource: AM9PR09MB4977.eurprd09.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2021 00:52:26.2218 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0902MB1764 Received-SPF: pass client-ip=40.92.75.65; envelope-from=arthur.miller@live.com; helo=EUR04-VI1-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:274481 Archived-At: --=-=-= Content-Type: text/plain Dmitry Gutov writes: > On 09.09.2021 23:23, Arthur Miller wrote: >> What do you need to build xref backend you mention? As mentioned on elisp-refs >> gh page: >> "xref-find-references: This command is included in Emacs 25.1, but it's based >> on >> a text search. It is confused by comments and strings, and cannot distinguish >> between functions and variables." >> Would rebuilding this command to use read instead of text search do the thing? > > You can easily plug another search implementation in xref-find-references. You > can even make summaries multiline (though that would feel weird to me). Ok, that sounds good. > But is elisp-refs that much better at searching? Distinguishing variables from > functions sounds nice, but then you might want to rename the function -- and > you'll probably want to rename the variable with the same name as well. I can't really tell you since I am not an expert. The quote was from the elisp-refs project. Read does have semantic information about symbols readily avialable, plain string searching does not have it immidiately, but strings can always be interned and looked up, so I really don't know how it compares to xref. I do believe that the author is correct in what it says, otherwise I don't think he would be doing that work. Observe also that the comment is old, a couple of years or more. > Same about comments: if elisp-refs skips comments altogether (rather than > smartly parsing references inside), you might miss some references to your > function when doing a rename. > I also wonder how it compares in performance. I haven't benchmarked or done any deeper investigations; I haven't even had time to look at his code yet; but I don't think the performance would be much different than just search through buffer. Read might be somewhat faster since it isn't as general regex matcher as regex-search is I guess, but that would be just my wild guessing, I don't think I am qualified to speak much about performance here without benchmarking. However what I have experienced, in my own little experiment, is that searching for all symbol definitions in entire Emacs lisp directory went fast on my computer, ~2 seconds, with flushing the database to the disk. Now I don't do much, I just collect all defuns, defmacros, defvars and defcustoms, so it is not terribly lots of work, finding references is probably much more work, but at least as an experiment it indicates that it could be very fast. I attached the code for the reference. (benchmark-run (ff-build-emacs-db)) (2.037053179 0 0.0) Now I don't know how fast is xref-find-references nor how it does what it does yet either, so lots here is just speculations, most of them probably wrong :). --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=func-freq.el ;;; func-freq.el --- -*- lexical-binding: t; -*- ;; Copyright (C) 2021 Arthur Miller ;; Author: Arthur Miller ;; Keywords: ;; This program is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see . ;;; Commentary: ;; ;;; Code: (defun ff-print (map) (maphash (lambda (k v) (print (format "%s: %s" k v))) map)) (defvar ff-srcs nil "Cash list of source files.") (defvar ff-lmap nil "Load paths") (defvar ff-pmap nil "Provided features") (defvar ff-vmap nil "Variables") (defvar ff-fmap nil "Functions and macros") (defun ff-save-db (db-file) (with-temp-file db-file (prin1 ff-lmap (current-buffer)) (prin1 ff-pmap (current-buffer)) (prin1 ff-vmap (current-buffer)) (prin1 ff-fmap (current-buffer)))) (defun ff-read-db (db-file) (with-temp-buffer (insert-file-contents db-file) (goto-char (point-min)) (setq ff-lmap (read (current-buffer)) ff-pmap (read (current-buffer)) ff-vmap (read (current-buffer)) ff-fmap (read (current-buffer))))) (defun ff-collect-features (src index) (let (sxp) (with-current-buffer (get-buffer-create "*ff-buffer*") (erase-buffer) (insert-file-contents src) (goto-char (point-min)) (while (setq sxp (ignore-errors (read (current-buffer)))) (when (listp sxp) (cond ((or (equal (car sxp) 'defun) (equal (car sxp) 'defmacro)) (puthash (cadr sxp) index ff-fmap)) ((or (equal (car sxp) 'defvar) (equal (car sxp) 'defcustom)) (puthash (cadr sxp) index ff-vmap)) ((equal (car sxp) 'provide) (puthash (cadr sxp) index ff-pmap)))))))) (defun ff-build-db (dir-tree) ;;(when (or (not ff-srcs) (equal (car ff-srcs) dir-tree)) (setq ff-srcs (cons dir-tree (directory-files-recursively dir-tree "\\.el$")) ff-lmap (make-hash-table :test 'equal) ff-vmap (make-hash-table :test 'equal) ff-fmap (make-hash-table :test 'equal) ff-pmap (make-hash-table :test 'equal)) ;; ) (let ((index 0)) (dolist (src (cdr ff-srcs)) (puthash index src ff-lmap) (ff-collect-features src index) (setq index (1+ index))))) (defun ff-build-emacs-db () (ff-build-db (expand-file-name "lisp/" source-directory)) (ff-save-db (expand-file-name "ff-db-emacs" user-emacs-directory))) (defun ff-build-package-db () (ff-build-db (expand-file-name "elpa/" user-emacs-directory)) (ff-save-db (expand-file-name "ff-db-packages" user-emacs-directory))) (provide 'func-freq) ;;; func-freq.el ends here --=-=-=--