From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#23223: 25.0.92; Can xref-find-references be sped up? Date: Mon, 11 Apr 2016 02:27:34 +0300 Message-ID: References: <83pou4m6h7.fsf@gnu.org> <902ac022-3a76-c363-0c77-12b1cdb8d521@yandex.ru> <8360vumzk4.fsf@gnu.org> <83mvp5lauu.fsf@gnu.org> <4424e043-31c7-0da4-213a-ee8ac31d9265@yandex.ru> <83vb3ri6q0.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------000DF9DF9ACD1293B1D3637F" X-Trace: ger.gmane.org 1460330892 1993 80.91.229.3 (10 Apr 2016 23:28:12 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 10 Apr 2016 23:28:12 +0000 (UTC) Cc: 23223@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Apr 11 01:28:12 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1apOm3-0001Yq-D8 for geb-bug-gnu-emacs@m.gmane.org; Mon, 11 Apr 2016 01:28:11 +0200 Original-Received: from localhost ([::1]:37013 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1apOm2-0001xv-Cp for geb-bug-gnu-emacs@m.gmane.org; Sun, 10 Apr 2016 19:28:10 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58537) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1apOlx-0001uX-Eu for bug-gnu-emacs@gnu.org; Sun, 10 Apr 2016 19:28:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1apOlu-00046J-6G for bug-gnu-emacs@gnu.org; Sun, 10 Apr 2016 19:28:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:45356) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1apOlu-00046D-2U for bug-gnu-emacs@gnu.org; Sun, 10 Apr 2016 19:28:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1apOlt-0002Gk-MW for bug-gnu-emacs@gnu.org; Sun, 10 Apr 2016 19:28:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 10 Apr 2016 23:28:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23223 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 23223-submit@debbugs.gnu.org id=B23223.14603308648700 (code B ref 23223); Sun, 10 Apr 2016 23:28:01 +0000 Original-Received: (at 23223) by debbugs.gnu.org; 10 Apr 2016 23:27:44 +0000 Original-Received: from localhost ([127.0.0.1]:57693 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1apOlb-0002GF-V4 for submit@debbugs.gnu.org; Sun, 10 Apr 2016 19:27:44 -0400 Original-Received: from mail-wm0-f41.google.com ([74.125.82.41]:34281) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1apOla-0002G0-QB for 23223@debbugs.gnu.org; Sun, 10 Apr 2016 19:27:43 -0400 Original-Received: by mail-wm0-f41.google.com with SMTP id l6so124512106wml.1 for <23223@debbugs.gnu.org>; Sun, 10 Apr 2016 16:27:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=eMSIQYRQWCGvNSZldVAvuXki+b19QcRtNWEAh9azv4M=; b=DRgKXru4+b1xoX8typFZ1F0HO8yQcbtoL0t6LirdAnIPrgWOI9c5TsO8w9f+TF5NLN HJlv+KzXVGiKcRDT4gWalxUrjS6FPkqFHCY1ODLgEBASjBDZk0w2602iiWDTZJ4iPvAW 2PJuGAW7K5K9Nbr7/VBVh5LmDwRGTTxViaoPPYePkEbAk4BUxx9DULtCcNQnEOsEJfFR VjF562708XLrta8xp4ixwBUf0wmocvC9VGSklB0ZUMfk4EXzYYc1RmuPkNtX9eRvCXQD mkkWEvE21s8THXtZ0TKuCwv2dj4oCYmsGVBQwaxxp8188lYbJp/RFBtIvIgbkne5gv6x 14Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to; bh=eMSIQYRQWCGvNSZldVAvuXki+b19QcRtNWEAh9azv4M=; b=kDgYTAGyzJ9fM5ta9I9X3l1lyEJCkyjG9N35ucDsecyuQijk9D/Y60/JAexQGDtSQO VFjeTEonWB8GSmsyyDf6ZPT5saaD/8bnhKrBs16q3sn5E4XuLk1zYIonczFpFtyXROdy YXnmTYwbeTG8HUPBZ17x8EFrRf/ALHG3UvNDvh8nySolG5cELe5YYGxguwD6jtKoHp/5 02H/q2tTnoz50TNr7dWoVDwa5l6NOEBhWYgpDKpNSqxOtFf1DlyR6IUCfAH+KSNFv0au hEah9d8zciaMNK63lMh+odCzTop5mbitHGz17kTR/6ta5HQHz4sBq7ib4op/OHXeJR2w dCMg== X-Gm-Message-State: AD7BkJKuA7XAisFAriP7Ai69vRYNOWZ6UbFf82zrTw5hHoZZ5wFIWIW7XjpgMsceWRfuvg== X-Received: by 10.194.57.236 with SMTP id l12mr20197528wjq.126.1460330857237; Sun, 10 Apr 2016 16:27:37 -0700 (PDT) Original-Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id a73sm14397625wme.2.2016.04.10.16.27.35 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 10 Apr 2016 16:27:36 -0700 (PDT) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.0 In-Reply-To: <83vb3ri6q0.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:116357 Archived-At: This is a multi-part message in MIME format. --------------000DF9DF9ACD1293B1D3637F Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Please try the attached patch. It cuts the time to search for 'current_buffer' from 4.5s to 0.75s here. You can comment out different parts of xref--collect-matches, to test if there are any easy bottlenecks left. The only one I see is find-buffer-visiting; replacing it with get-file-buffer yields some extra boost, but it would come with corresponding downsides in those exact cases for which find-buffer-visiting was designed (although I can add another local variable instead that would cache the previous file name and buffer used for it; but that's scraping the bottom). To measure, open src/xdisp.c and evaluate (benchmark 1 '(xref-collect-references "current_buffer" default-directory)). On 04/09/2016 10:25 AM, Eli Zaretskii wrote: > Would it help to only use the mode's syntax table, and avoid switching > on the major mode as a whole? Not for performance anymore (see above). You can't always determine the syntax table from the major mode name (there is a convention, but it's not iron-clad), it might be in an autoloaded file, etc... And like I mentioned before, we also need syntax-propertize-function, and potentially any buffer-local variables that it could use. > That problem is relevant for IDutils as well (the scanner is > determined by the file's extension only), so we already have a certain > (hopefully, small) number of misses and false positives. That affects xref-find-references (as long as you use id-utils, which is not mandatory) but not project-find-regexp. The currently discussed tuneup should improve both. > I think this > cannot be entirely avoided. So maybe we shouldn't try so hard > avoiding false positives. It seems we can't avoid ignoring the mode specification at the bottom, but that's about it. And nothing's stopping id-utils (or other tools) from using a better language detection scheme in the future. > E.g., the "M-x gid" command, which comes > with IDutils and is a trivial wrapper around lid invocation, simply > shows the output in a Grep-like buffer, through which you can step > with next-error. Do you actually want xref-find-regexp, and not xref-find-references? > It is lightning-fast: what takes 13 sec with > xref-find-references, takes less than 2 sec with "M-x gid". What's the new time you get from the former? > Perhaps use insert-file-contents-literally, as decoding could slow > down things considerably. No significant difference here, on the given example. By the way, the "insert-file-contents + set-auto-mode" dance comes with a new minor downside: extra chatter from the major modes. E.g. try project-file-regexp with "should have received a copy". We can avoid saving it to the message log, but it appears in the echo area either way. --------------000DF9DF9ACD1293B1D3637F Content-Type: text/x-patch; name="xref-with-temp-buffer.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xref-with-temp-buffer.diff" diff --git a/lisp/cedet/semantic/symref/cscope.el b/lisp/cedet/semantic/symref/cscope.el index 4890b5b..3abd8b3 100644 --- a/lisp/cedet/semantic/symref/cscope.el +++ b/lisp/cedet/semantic/symref/cscope.el @@ -60,6 +60,9 @@ semantic-symref-tool-cscope (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-cscope--line-re + "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-cscope)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -78,8 +81,13 @@ semantic-symref-tool-cscope ;; We have to return something at this point. subtxt))) ) - (t - (when (re-search-forward "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) " nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-cscope--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1)) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; :resulttype is 'line + (when (re-search-forward semantic-symref-cscope--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1))) )))) diff --git a/lisp/cedet/semantic/symref/global.el b/lisp/cedet/semantic/symref/global.el index e4c114e..a33427e 100644 --- a/lisp/cedet/semantic/symref/global.el +++ b/lisp/cedet/semantic/symref/global.el @@ -49,6 +49,9 @@ semantic-symref-tool-global (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-global--line-re + "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-global)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -57,8 +60,13 @@ semantic-symref-tool-global ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-global--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 3) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) " nil t) + (when (re-search-forward semantic-symref-global--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 3)) )))) diff --git a/lisp/cedet/semantic/symref/grep.el b/lisp/cedet/semantic/symref/grep.el index 5d1fea8..868e6c3 100644 --- a/lisp/cedet/semantic/symref/grep.el +++ b/lisp/cedet/semantic/symref/grep.el @@ -188,6 +188,9 @@ semantic-symref-grep-shell ;; Return the answer ans)) +(defconst semantic-symref-grep--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-grep)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -195,8 +198,13 @@ semantic-symref-grep-shell ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-grep--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + (when (re-search-forward semantic-symref-grep--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 1)) )))) diff --git a/lisp/cedet/semantic/symref/idutils.el b/lisp/cedet/semantic/symref/idutils.el index 4127d7a..db3e9a0 100644 --- a/lisp/cedet/semantic/symref/idutils.el +++ b/lisp/cedet/semantic/symref/idutils.el @@ -49,6 +49,9 @@ semantic-symref-tool-idutils (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-idutils--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-idutils)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -59,8 +62,13 @@ semantic-symref-tool-idutils ((eq (oref tool :searchtype) 'tagcompletions) (when (re-search-forward "^\\([^ ]+\\) " nil t) (match-string 1))) - (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-idutils--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1) default-directory) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; resulttype is line + (when (re-search-forward semantic-symref-idutils--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1) default-directory)) )))) diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el index feed0fb..7354f8b 100644 --- a/lisp/progmodes/xref.el +++ b/lisp/progmodes/xref.el @@ -859,19 +859,25 @@ xref-collect-references ;; to force the backend to use `default-directory'. (let* ((ede-minor-mode nil) (default-directory dir) + ;; FIXME: Remove CScope and Global from the recognized tools? + ;; The current implementations interpret the symbol search as + ;; "find all calls to the given function", but not function + ;; definition. And they return nothing when passed a variable + ;; name, even a global one. (semantic-symref-tool 'detect) (case-fold-search nil) - (res (semantic-symref-find-references-by-name symbol 'subdirs)) - (hits (and res (oref res hit-lines))) - (orig-buffers (buffer-list))) + (inst (semantic-symref-instantiate :searchfor symbol + :searchtype 'symbol + :searchscope 'subdirs + :resulttype 'line-and-text)) + (hits (semantic-symref-perform-search inst)) + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (unwind-protect (cl-mapcan (lambda (hit) (xref--collect-matches - hit (format "\\_<%s\\_>" (regexp-quote symbol)))) + hit (format "\\_<%s\\_>" (regexp-quote symbol)) + tmp-buffer)) hits) - ;; TODO: Implement "lightweight" buffer visiting, so that we - ;; don't have to kill them. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) ;;;###autoload (defun xref-collect-matches (regexp files dir ignores) @@ -890,34 +896,24 @@ xref-collect-matches files (expand-file-name dir) ignores)) - (orig-buffers (buffer-list)) (buf (get-buffer-create " *xref-grep*")) (grep-re (caar grep-regexp-alist)) - (counter 0) - reporter - hits) + hits + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (with-current-buffer buf (erase-buffer) (call-process-shell-command command nil t) (goto-char (point-min)) (while (re-search-forward grep-re nil t) - (push (cons (string-to-number (match-string 2)) - (match-string 1)) + (push (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))) hits))) - (setq reporter (make-progress-reporter - (format "Collecting search results...") - 0 (length hits))) (unwind-protect (cl-mapcan (lambda (hit) - (prog1 - (progress-reporter-update reporter counter) - (cl-incf counter)) - (xref--collect-matches hit regexp)) + (xref--collect-matches hit regexp tmp-buffer)) (nreverse hits)) - (progress-reporter-done reporter) - ;; TODO: Same as above. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) (defun xref--rgrep-command (regexp files dir ignores) (require 'find-dired) ; for `find-name-arg' @@ -980,30 +976,49 @@ xref--regexp-to-extended (match-string 1 str))))) str t t)) -(defun xref--collect-matches (hit regexp) - (pcase-let* ((`(,line . ,file) hit) - (buf (or (find-buffer-visiting file) - (semantic-find-file-noselect file)))) - (with-current-buffer buf - (save-excursion +(defvar xref--temp-buffer-file-name nil) + +(defun xref--collect-matches (hit regexp tmp-buffer) + (pcase-let* ((`(,line ,file ,text) hit) + (buf (find-buffer-visiting file))) + (if buf + (with-current-buffer buf + (save-excursion + (goto-char (point-min)) + (forward-line (1- line)) + (xref--collect-matches-1 regexp file line + (line-beginning-position) + (line-end-position)))) + (with-current-buffer tmp-buffer + (erase-buffer) + (unless (equal file xref--temp-buffer-file-name) + (insert-file-contents file nil 0 200) + (setq-local delay-mode-hooks t) + (set-auto-mode t) + (setq-local xref--temp-buffer-file-name file) + (setq-local inhibit-read-only t) + (erase-buffer)) + (insert text) (goto-char (point-min)) - (forward-line (1- line)) - (let ((line-end (line-end-position)) - (line-beg (line-beginning-position)) - matches) - (syntax-propertize line-end) - ;; FIXME: This results in several lines with the same - ;; summary. Solve with composite pattern? - (while (re-search-forward regexp line-end t) - (let* ((beg-column (- (match-beginning 0) line-beg)) - (end-column (- (match-end 0) line-beg)) - (loc (xref-make-file-location file line beg-column)) - (summary (buffer-substring line-beg line-end))) - (add-face-text-property beg-column end-column 'highlight - t summary) - (push (xref-make-match summary loc (- end-column beg-column)) - matches))) - (nreverse matches)))))) + (xref--collect-matches-1 regexp file line + (point) + (point-max)))))) + +(defun xref--collect-matches-1 (regexp file line line-beg line-end) + (let (matches) + (syntax-propertize line-end) + ;; FIXME: This results in several lines with the same + ;; summary. Solve with composite pattern? + (while (re-search-forward regexp line-end t) + (let* ((beg-column (- (match-beginning 0) line-beg)) + (end-column (- (match-end 0) line-beg)) + (loc (xref-make-file-location file line beg-column)) + (summary (buffer-substring line-beg line-end))) + (add-face-text-property beg-column end-column 'highlight + t summary) + (push (xref-make-match summary loc (- end-column beg-column)) + matches))) + (nreverse matches))) (provide 'xref) --------------000DF9DF9ACD1293B1D3637F--