From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#49836: Support ripgrep in semantic-symref-tool-grep Date: Sat, 7 Aug 2021 17:12:37 +0300 Message-ID: References: <87sfzrbke1.fsf_-_@mail.linkov.net> <875ywn556i.fsf@mail.linkov.net> <87sfzo3nvm.fsf@mail.linkov.net> <87tuk38kq9.fsf@mail.linkov.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="6447"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 Cc: 49836@debbugs.gnu.org To: Juri Linkov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Aug 07 16:13:12 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mCN4a-0001Vu-62 for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 07 Aug 2021 16:13:12 +0200 Original-Received: from localhost ([::1]:36730 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mCN4Y-0000jy-9Q for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 07 Aug 2021 10:13:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:47976) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mCN4R-0000jh-CS for bug-gnu-emacs@gnu.org; Sat, 07 Aug 2021 10:13:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:41441) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mCN4Q-0006MF-Je for bug-gnu-emacs@gnu.org; Sat, 07 Aug 2021 10:13:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mCN4Q-0006Wn-Eh for bug-gnu-emacs@gnu.org; Sat, 07 Aug 2021 10:13:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 07 Aug 2021 14:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49836 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 49836-submit@debbugs.gnu.org id=B49836.162834557125076 (code B ref 49836); Sat, 07 Aug 2021 14:13:02 +0000 Original-Received: (at 49836) by debbugs.gnu.org; 7 Aug 2021 14:12:51 +0000 Original-Received: from localhost ([127.0.0.1]:52987 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mCN4E-0006WO-RU for submit@debbugs.gnu.org; Sat, 07 Aug 2021 10:12:51 -0400 Original-Received: from mail-wr1-f54.google.com ([209.85.221.54]:43939) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mCN49-0006W6-GD for 49836@debbugs.gnu.org; Sat, 07 Aug 2021 10:12:49 -0400 Original-Received: by mail-wr1-f54.google.com with SMTP id h14so14947598wrx.10 for <49836@debbugs.gnu.org>; Sat, 07 Aug 2021 07:12:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=D4qaG7jlrKXbdaiy9F1TNNTGozC/623OyS+rx37mDK4=; b=UwUaCqFLQUIpS7/z2l4UxoEW6vh15Ej5hzS9bjiru1z6GPy5VzwObvm58XvUdBDzw2 VMPq0CGb3gVxoWVbp0wuw/GozQNQxgFSFL6jUwX9XDOAwrydpbfyDlV3DIDYKVwOdcYu NABuxvApKFwkMNb+3f6n368KNr4cSxoEfQctJ8dpjEipldTVGsq9yXHrUwVZcR3+XqEI A91CB7FHYGBv7NeuZkv3DGCEJzHBur142bdsHEKx7zmkYTCJjHcSq/qcl9lzP7qxP9UP Jsdu63EOkYSf7PJ09VKgE2HQBdZOp31JTifoSqc5jmssQUaIaC084kapH5JAf8CaTbzp y9cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=D4qaG7jlrKXbdaiy9F1TNNTGozC/623OyS+rx37mDK4=; b=S0QsDGXlkcEaE8t9J8MZFnAf/n4oVTIG6XuY4LiDoUdjEakmXnKRzQdZa8c3ggHBOX lwKQRzaXxmGGswD/12eSG75l7TSTNP4pvScmWsOTP7mi4H+zdoxo7ogRn9G5s6g7WUNc A+gb7J7wCPElpNuX4OG9B7UMpRupe5WMiqtS+IYJSOR9jUf2fHW2L+7FdTUyBJ4fd3vs PclZg+WhKk11kwe0lUuRShGiqtpO5iBfdiwKnpvu4nbyQxOd7pUlP0pENOP0C9WvB+nq uGB8Eb4YVUmd2Rjmgfv2rFHYv8NuA3x4w5DrwgEGTyVT/m3Ju/rfMQhPaj+he5YKcCsc cNAA== X-Gm-Message-State: AOAM530w7tdTKB8Q+QG8GubOwpchawQcTUxCdDuSodQkdooqBf37RIaO wlzCCGA5c+gU8oZRXnDbDLR7AOODjfw= X-Google-Smtp-Source: ABdhPJxdzJmPhw0BrvspCdaezAlwj3zZ3Nztzz7zoJxg0/EEvf1HsZDnK/nSREzCTUYbjJjgOpPOXw== X-Received: by 2002:a5d:4ac5:: with SMTP id y5mr16379006wrs.125.1628345559678; Sat, 07 Aug 2021 07:12:39 -0700 (PDT) Original-Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id n7sm11676109wmd.3.2021.08.07.07.12.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 07 Aug 2021 07:12:38 -0700 (PDT) In-Reply-To: <87tuk38kq9.fsf@mail.linkov.net> Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:211381 Archived-At: On 06.08.2021 03:35, Juri Linkov wrote: > semantic-symref-grep-use-template constructs such command line: > > "rg ... -e \\\\bsoap-type-is-array\\?\\\\b" > > that finds matches. The correct one will probably look like "rg ... -e \\\\bsoap-type-is-array\\\\?\\\\b" (same number of backslashes before '?' as before 'b'), and it won't find any. The one you mentioned will find false positives. E.g., try searching for 'file-name-as-directory?'. Or 'carr?'. >> See above. But also consider what happens if a user sees that grep-program >> is now customizable and ripgrep is an officially supported value. They >> change it to "rg", and then suddenly their 'M-x rgrep' input has to use the >> extended regexp format? > > This difference could be explained in the documentation. If it comes to that, yes, but it's usually better to fix usability problems that just document them. >> Worse than that, any third-party package that uses grep-find-template will >> suddenly have a high chance of failing if they pass any nontrivial regexps >> to it, especially if those have groupings or alternations. > > This already happened after trying to customize grep-find-template > to use rg broke xref-find-references, so the problem already exists > and needs to be solved. The problem exists, and has been for a long time: grep.el doesn't properly support the "alternative" search programs, which are very popular now. Its abstraction is leaky and doesn't work with anything but grep. But I think that means we need a better abstraction. Let's try to make sure we don't create bigger problems when fixing it. And "packages stop working when I customize grep-program" sounds worse than "I can't customize grep-program to 'rg', so my searches are a bit slower than they could have been". >> It's a hard problem: grep.el is not prepared for abstracting like that. If >> we at least standardized it internally on Extended format, that would at >> least remove one source of uncertainty and bugs. The user still can input >> basic regexps interactively, we can translate them easily. > > Is there a package that can translate between them reliably? For the limited purpose of symref/grep, we could use xref--regexp-to-extended. It's already used in xref-matches-in-directory and xref-matches-in-files. Better name/documentation and tests are pending. Note that it actually translates from a (subset of) Emacs regexp to Extended and back (it's reversible). The proper basic regexp syntax treats '+' and '?' as normal characters unless escaped, but they're special in Emacs regexps. The above function is how one can use Emacs syntax (though only limited a subset, for now) in project-find-regexp. I also saw some commits to ELPA yesterday, that show that Consult includes a more advanced version of this feature: https://git.savannah.gnu.org/cgit/emacs/elpa.git/commit/?h=externals/consult&id=7bd3e44929d44cf0e17f38e943e9be2bd6014237 https://git.savannah.gnu.org/cgit/emacs/elpa.git/commit/?h=externals/consult&id=95dadd98a6a0f08955f67f1e9a7cc312435a86b8 Not sure how mature it is (seems still in development), but perhaps we could move it to the core sooner or later. And use it instead, if it does provide any improvement for our use case here. >> Further: seeing xref-search-program-alist, people asked for support for >> other similar programs, such as ag and pt. Any solution we end up with, we >> should try to ensure they are valid values of grep-program as well. > > Why not, semantic-symref already supports alternative tools > such as cscope, global, idutils. So xref could support more too. It's easy enough for Xref, yes. It only has to support one single, well-defined scenario. >>> + (if (equal grep-program "rg") >>> + (format "(^|\\W)%s(\\W|$)" >>> + (oref tool searchfor)) >>> + (format "\\(^\\|\\W\\)%s\\(\\W\\|$\\)" >>> + (oref tool searchfor)))))) >> >> This can work. Except the comparison should be with "grep", I think: all >> other alternatives only work with the Extended format. > > I'm worried about the case when the user customizes > 'grep-program' to e.g. an absolute path "/bin/grep" > or "/usr/local/bin/grep", etc. (string-match "\\bgrep\\b" grep-program) could take care of this. To sum up, I'm all for adding some clutches to symref/grep.el, to support your advanced scenario, right now. As for having grep-program customizable, perhaps we should add some new feature/abstraction/package? To avoid breakage, and for it to be opt-in for any new callers from Lisp. Or indeed have templates use Extended syntax, and grep-expand-template translate REGEXP to it. That can cause breakage for existing users, though, those who already customize grep-find-template, etc, to their particular programs.