Re: Helpful in Emacs?

[Arthur Miller <arthur.miller@live.com>]

[-- Attachment #1: Type: text/plain, Size: 2742 bytes --]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 09.09.2021 23:23, Arthur Miller wrote:
>> What do you need to build  xref backend you mention? As mentioned on elisp-refs
>> gh page:
>> "xref-find-references: This command is included in Emacs 25.1, but it's based
>> on
>> a text search. It is confused by comments and strings, and cannot distinguish
>> between functions and variables."
>> Would rebuilding this command to use read instead of text search do the thing?
>
> You can easily plug another search implementation in xref-find-references. You
> can even make summaries multiline (though that would feel weird to me).
Ok, that sounds good.

> But is elisp-refs that much better at searching? Distinguishing variables from
> functions sounds nice, but then you might want to rename the function -- and
> you'll probably want to rename the variable with the same name as well.

I can't really tell you since I am not an expert. The quote was from the
elisp-refs project. Read does have semantic information about symbols readily
avialable, plain string searching does not have it immidiately, but strings can
always be interned and looked up, so I really don't know how it compares to
xref. I do believe that the author is correct in what it says, otherwise I don't
think he would be doing that work. Observe also that the comment is old, a
couple of years or more.

> Same about comments: if elisp-refs skips comments altogether (rather than
> smartly parsing references inside), you might miss some references to your
> function when doing a rename.

> I also wonder how it compares in performance.

I haven't benchmarked or done any deeper investigations; I haven't even had time
to look at his code yet; but I don't think the performance would be much
different than just search through buffer. Read might be somewhat faster since
it isn't as general regex matcher as regex-search is I guess, but that would be
just my wild guessing, I don't think I am qualified to speak much about
performance here without benchmarking.

However what I have experienced, in my own little experiment, is that searching for
all symbol definitions in entire Emacs lisp directory went fast on my computer,
~2 seconds, with flushing the database to the disk. Now I don't do much, I just
collect all defuns, defmacros, defvars and defcustoms, so it is not terribly
lots of work, finding references is probably much more work, but at least as an
experiment it indicates that it could be very fast. I attached the code for the
reference.

(benchmark-run (ff-build-emacs-db))

(2.037053179 0 0.0)

Now I don't know how fast is xref-find-references nor how it does what it does
yet either, so lots here is just speculations, most of them probably wrong :). 


[-- Attachment #2: func-freq.el --]
[-- Type: text/plain, Size: 3258 bytes --]

;;; func-freq.el ---                                 -*- lexical-binding: t; -*-

;; Copyright (C) 2021  Arthur Miller

;; Author: Arthur Miller <arthur.miller@live.com>
;; Keywords: 

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;; 

;;; Code:
(defun ff-print (map)
  (maphash (lambda (k v) (print (format "%s: %s" k v))) map))

(defvar ff-srcs nil
  "Cash list of source files.")
(defvar ff-lmap nil
  "Load paths")
(defvar ff-pmap nil
  "Provided features")
(defvar ff-vmap nil
  "Variables")
(defvar ff-fmap nil
  "Functions and macros")

(defun ff-save-db (db-file)
  (with-temp-file db-file
    (prin1 ff-lmap (current-buffer))
    (prin1 ff-pmap (current-buffer))
    (prin1 ff-vmap (current-buffer))
    (prin1 ff-fmap (current-buffer))))

(defun ff-read-db (db-file)
  (with-temp-buffer
    (insert-file-contents db-file)
    (goto-char (point-min))
    (setq ff-lmap (read (current-buffer))
          ff-pmap (read (current-buffer))
          ff-vmap (read (current-buffer))
          ff-fmap (read (current-buffer)))))

(defun ff-collect-features (src index)
  (let (sxp)
    (with-current-buffer (get-buffer-create "*ff-buffer*")
      (erase-buffer)
      (insert-file-contents src)
      (goto-char (point-min))
      (while (setq sxp (ignore-errors (read (current-buffer))))
        (when (listp sxp)
          (cond ((or (equal (car sxp) 'defun)
                     (equal (car sxp) 'defmacro))
                 (puthash (cadr sxp) index ff-fmap))
                ((or (equal (car sxp) 'defvar)
                     (equal (car sxp) 'defcustom))
                 (puthash (cadr sxp) index ff-vmap))
                ((equal (car sxp) 'provide)
                 (puthash (cadr sxp) index ff-pmap))))))))

(defun ff-build-db (dir-tree)
  ;;(when (or (not ff-srcs) (equal (car ff-srcs) dir-tree))
      (setq ff-srcs (cons dir-tree (directory-files-recursively dir-tree "\\.el$"))
            ff-lmap (make-hash-table :test 'equal)
            ff-vmap (make-hash-table :test 'equal)
            ff-fmap (make-hash-table :test 'equal)
            ff-pmap (make-hash-table :test 'equal))
      ;; )
  (let ((index 0))
    (dolist (src (cdr ff-srcs))
      (puthash index src ff-lmap)
      (ff-collect-features src index)
      (setq index (1+ index)))))

(defun ff-build-emacs-db ()
  (ff-build-db (expand-file-name "lisp/" source-directory))
  (ff-save-db (expand-file-name "ff-db-emacs" user-emacs-directory)))

(defun ff-build-package-db ()
  (ff-build-db (expand-file-name "elpa/" user-emacs-directory))
  (ff-save-db (expand-file-name "ff-db-packages" user-emacs-directory)))

(provide 'func-freq)
;;; func-freq.el ends here