[ELPA] New package: mctags

[ELPA] New package: mctags

[Chen Bin]

Hi,
I developed a package using Ctags.

Project URL: https://github.com/redguardtoo/mctags/

I would like this package be accepted by ELPA.

I'm the only developer of this package which is only dependent on
package counsel. Counsel is already accepted by ELPA.

I've Emacs signed Copyright Papers when contributing company-mode,
https://github.com/company-mode/company-mode/pull/13

My situation is not changed.

Here is summary of mctags:

Fast and complete Ctags solution.

Usage:
  "M-x mctags-find-tag-at-point" to navigate.  This command will also
  run `mctags-scan-code' automatically if tags file is not built yet.

  "M-x mctags-scan-code" to create tags file
  "M-x mctags-grep" to grep

Tips:
You can use ivy's negative pattern to filter candidates.
For example, input "keyword1 !keyword2 keyword3" means:
  "(keyword1 and (not (keyword2 or keyword3))"

See https://github.com/redguardtoo/mctags/ for more advanced tips.

-- 
Best Regards,
Chen Bin

--
Help me, help you

[ELPA] New package: mctags

[Chen Bin]

[-- Attachment #1: Type: text/plain, Size: 761 bytes --]

Hi,
Sorry to re-send mail. I forget to attach the source code in previous
mail.

I developed a package using Ctags. https://github.com/redguardtoo/mctags/

I would like this package be accepted by ELPA.

I'm the only developer of this package which is only dependent on
package counsel. Counsel is already accepted by ELPA.

I've Emacs signed Copyright Papers when contributing company-mode,
https://github.com/company-mode/company-mode/pull/13

My situation is not changed.

Here is summary of mctags:

Fast and complete Ctags solution.

Usage:
  "M-x mctags-find-tag-at-point" to navigate.  This command will also
  run `mctags-scan-code' automatically if tags file is not built yet.

  "M-x mctags-scan-code" to create tags file
  "M-x mctags-grep" to grep


[-- Attachment #2: mctags.el --]
[-- Type: application/emacs-lisp, Size: 20650 bytes --]

[-- Attachment #3: Type: text/plain, Size: 50 bytes --]


-- 
Best Regards,
Chen Bin

--
Help me, help you

Re: [ELPA] New package: mctags

[Eli Zaretskii]

> From: Chen Bin <chenbin.sh@gmail.com>
> Date: Thu, 12 Oct 2017 22:05:16 +1100
> 
> I developed a package using Ctags. https://github.com/redguardtoo/mctags/
> 
> I would like this package be accepted by ELPA.
> 
> I'm the only developer of this package which is only dependent on
> package counsel. Counsel is already accepted by ELPA.
> 
> I've Emacs signed Copyright Papers when contributing company-mode,
> https://github.com/company-mode/company-mode/pull/13
> 
> My situation is not changed.
> 
> Here is summary of mctags:
> 
> Fast and complete Ctags solution.
> 
> Usage:
>   "M-x mctags-find-tag-at-point" to navigate.  This command will also
>   run `mctags-scan-code' automatically if tags file is not built yet.
> 
>   "M-x mctags-scan-code" to create tags file
>   "M-x mctags-grep" to grep

Can you please tell how is this package different from the built-in
etags.el package?

Thanks.

Re: [ELPA] New package: mctags

[chen bin]

[-- Attachment #1: Type: text/plain, Size: 1622 bytes --]

- If there are multiple matches, you can filter the candidates in candidate
window.
- the tags file is automatically created in current project. . BTW, project
root is automatically detected if you use git/svn/hg
- if no match found, fallback to `mctags-grep` to grep in current project.
So you should always found matched string
- Improvement on performance (for example, ripgrep is automatically used as
grep program if installed. GNU grep is fallback grep program)
- tags file could be automatically updated when user save current file.
There is algorithm behind



On Thu, Oct 12, 2017 at 11:11 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Chen Bin <chenbin.sh@gmail.com>
> > Date: Thu, 12 Oct 2017 22:05:16 +1100
> >
> > I developed a package using Ctags. https://github.com/redguardtoo
> /mctags/
> >
> > I would like this package be accepted by ELPA.
> >
> > I'm the only developer of this package which is only dependent on
> > package counsel. Counsel is already accepted by ELPA.
> >
> > I've Emacs signed Copyright Papers when contributing company-mode,
> > https://github.com/company-mode/company-mode/pull/13
> >
> > My situation is not changed.
> >
> > Here is summary of mctags:
> >
> > Fast and complete Ctags solution.
> >
> > Usage:
> >   "M-x mctags-find-tag-at-point" to navigate.  This command will also
> >   run `mctags-scan-code' automatically if tags file is not built yet.
> >
> >   "M-x mctags-scan-code" to create tags file
> >   "M-x mctags-grep" to grep
>
> Can you please tell how is this package different from the built-in
> etags.el package?
>
> Thanks.
>



-- 
help me, help you.

[-- Attachment #2: Type: text/html, Size: 2641 bytes --]

Re: [ELPA] New package: mctags

[Eli Zaretskii]

> From: chen bin <chenbin.sh@gmail.com>
> Date: Fri, 13 Oct 2017 00:24:17 +1100
> 
> - If there are multiple matches, you can filter the candidates in candidate window.

I think xref-find-definitions, when it uses the etags back-end,
already supports that, doesn't it?

> - the tags file is automatically created in current project.

Yes, but AFAICT, the package uses a somewhat naïve way of generating
TAGS, in effect passing all the non-trivial file names to etags.  Some
projects might use more sophisticated methods, for example see what
src/Makefile does in Emacs to tag both the Lisp and the C names of the
Emacs primitives.

> - if no match found, fallback to `mctags-grep` to grep in current project. So you should always found matched
> string

But the matches found this way are not necessarily definitions, they
can be references.

> - Improvement on performance (for example, ripgrep is automatically used as grep program if installed. GNU
> grep is fallback grep program)

Well, finding a tag via etags.el doesn't require Grep at all.

> - tags file could be automatically updated when user save current file.

Is this really a good idea?  Updating TAGS means you need also revisit
it, which could be a problem in some complex setups, when there's more
than one tag table active at the same time.  OTOH, etags.el is written
in a way that makes frequent updates of TAGS unnecessary.

Re: [ELPA] New package: mctags

[chen bin]

On Fri, Oct 13, 2017 at 3:13 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: chen bin <chenbin.sh@gmail.com>
>> Date: Fri, 13 Oct 2017 00:24:17 +1100
>>
>> - If there are multiple matches, you can filter the candidates in candidate window.
>
> I think xref-find-definitions, when it uses the etags back-end,
> already supports that, doesn't it?
I've been using xref for some time. As I can see, it just gives your
the list of matches in a buffer. It can't filter further with pattern
or negative pattern or combination of patterns.

For example, I can filter candidates in mctags with
"keyword1\|keyword2 !keyword3" (matching either "keyword1" or
"keyword2" but not keyword3)
>
>> - the tags file is automatically created in current project.
>
> Yes, but AFAICT, the package uses a somewhat naïve way of generating
> TAGS, in effect passing all the non-trivial file names to etags.  Some
Good question. For tags creation, mctags now focus on out of box
userexperience (For example, by default, `*.o`, `*.elc` are ignored`).
Extra option could be added to tweak ctags cli in the future. But it's
totally fine if you use mctags only for code navigation and leave TAGS
creation to other solutions.

Please note mctags RESPECTS the existing tags file created by other
solutions. mctags will NOT override existing TAGS created by other
programs (Makefile, for example) without user's confirmation.

> projects might use more sophisticated methods, for example see what
> src/Makefile does in Emacs to tag both the Lisp and the C names of the
> Emacs primitives.
>
Please note mctags RESPECTS the existing tags file created by other
programs. It's IMPOSSIBLE mctags will override existing TAGS created
by other programs (Makefile, for example) without user's confirmation.

>> - if no match found, fallback to `mctags-grep` to grep in current project. So you should always found matched
>> string
>
> But the matches found this way are not necessarily definitions, they
> can be references.
>
If definition is found, the grep will not be called. One reason I need
this is that the TAGS might not be updated for the latest code change
yet. Please note the UI make user clear that by displaying message "NO
tag found. Now you are seeing grep results" at the top of candidate
window.

>> - Improvement on performance (for example, ripgrep is automatically used as grep program if installed. GNU
>> grep is fallback grep program)
>
> Well, finding a tag via etags.el doesn't require Grep at all.
Grep is called if and only if mctags-find-tag fails.  It's a useful
feature when TAGS is not updated yet.

For example, when I'm focusing on coding, I run `M-x find-tag` but no
matches found. I've to stop to ask myself: "What' wrong? Did I forget
to add definition? Or TAGS is not updated yet? Should I wait for ctags
to finish updating? Or should I manually restart ctags process?". I'm
forced out of "Flow" status by a simple "No matches found" message
because too many questions popup. But in mctags, we display "No tag
found" message PLUS the grep results. So I don't need THINK, I just
TAKE ACTION to filter grep results.

In my practice, this is huge productivity boost.
>
>> - tags file could be automatically updated when user save current file.
>
> Is this really a good idea?  Updating TAGS means you need also revisit
> it, which could be a problem in some complex setups, when there's more
> than one tag table active at the same time.  OTOH, etags.el is written
> in a way that makes frequent updates of TAGS unnecessary.

Some people might preferring TAGS auto-updating instead of manually
running ctags from time to time. Anyway, TAGS auto updating is not
enabled by default. So it will not conflict with your current
ctags/etags setup. You can safely use `mctags-find-tag` without any
worries on conflicts with other packages. At minimum, we give the
users an extra option to update TAGS. Users have freedom to choose
between these options.


mctags is especially newbie friendly. Open `mctags.el`, `eval-buffer`,
then `M-x mctags-find-tag-at-point`, that's enough. You don't need
even create TAGS file manually.

-- 
help me, help you.

Re: [ELPA] New package: mctags

[Eli Zaretskii]

> From: chen bin <chenbin.sh@gmail.com>
> Date: Fri, 13 Oct 2017 12:39:54 +1100
> Cc: emacs-devel@gnu.org
> 
> >> - If there are multiple matches, you can filter the candidates in candidate window.
> >
> > I think xref-find-definitions, when it uses the etags back-end,
> > already supports that, doesn't it?
> I've been using xref for some time. As I can see, it just gives your
> the list of matches in a buffer. It can't filter further with pattern
> or negative pattern or combination of patterns.

The list is usually very short (most of the time, only one candidate),
so the need for sophisticated filtering is quite low.

> But it's totally fine if you use mctags only for code navigation and
> leave TAGS creation to other solutions.
> 
> Please note mctags RESPECTS the existing tags file created by other
> solutions. mctags will NOT override existing TAGS created by other
> programs (Makefile, for example) without user's confirmation.

I understand.  It just seemed to me that, if we ignore for the moment
the features for creating/updating TAGS, what's left is very little,
and its contribution to the existing functionality is minor.  It
doesn't seem to justify a new package.

Maybe a better way forward is to extend etags.el with a few optional
features.

Anyway, these are just my opinions.  I'd be interested to hear from
others.

Thanks.

Re: [ELPA] New package: mctags

[chen bin]

In web front end development,there could be dozens of candidates. For
example, in ReactJS development,  `props.method1` could be defined in
parent or sibling components. Each component corresponds to a
independent JS files. So you might need filter a lot of candidates by
js file name. Advanced filter is must have feature.

In web front end development, new wheels are keep being invented. For
example, consider css,  We got
css/less/sass/scss/postcss/styled-component, six planguages. So ctags
should be very generic to extract tags from these
languages/frameworks.

`mctags` is targeting on web developers. It has different road map
from `etags.el`. `mctags` tries to fuzzy match as many candidates as
possible without considering precision at first stage. Then filter out
the noise in second stage.

I tested with my current project. Using same TAGS file, `mctags` will
find more candidates than `xref`. Web developers need extra "noise"
because our projects are usually mixture of at least four syntax (css,
html, js, plus a language at server side,  C#/Java/PHP ...).. Actual
tag definition could exist anywhere.

My impression is `etags.el` comply with the convention of C/Perl
developers who might not understand what web developers really want.

On Fri, Oct 13, 2017 at 7:47 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: chen bin <chenbin.sh@gmail.com>
>> Date: Fri, 13 Oct 2017 12:39:54 +1100
>> Cc: emacs-devel@gnu.org
>>
>> >> - If there are multiple matches, you can filter the candidates in candidate window.
>> >
>> > I think xref-find-definitions, when it uses the etags back-end,
>> > already supports that, doesn't it?
>> I've been using xref for some time. As I can see, it just gives your
>> the list of matches in a buffer. It can't filter further with pattern
>> or negative pattern or combination of patterns.
>
> The list is usually very short (most of the time, only one candidate),
> so the need for sophisticated filtering is quite low.
>
>> But it's totally fine if you use mctags only for code navigation and
>> leave TAGS creation to other solutions.
>>
>> Please note mctags RESPECTS the existing tags file created by other
>> solutions. mctags will NOT override existing TAGS created by other
>> programs (Makefile, for example) without user's confirmation.
>
> I understand.  It just seemed to me that, if we ignore for the moment
> the features for creating/updating TAGS, what's left is very little,
> and its contribution to the existing functionality is minor.  It
> doesn't seem to justify a new package.
>
> Maybe a better way forward is to extend etags.el with a few optional
> features.
>
> Anyway, these are just my opinions.  I'd be interested to hear from
> others.
>
> Thanks.



-- 
help me, help you.

Re: [ELPA] New package: mctags

[Dmitry Gutov]

On 10/13/17 11:47 AM, Eli Zaretskii wrote:

>>> I think xref-find-definitions, when it uses the etags back-end,
>>> already supports that, doesn't it?
>> I've been using xref for some time. As I can see, it just gives your
>> the list of matches in a buffer. It can't filter further with pattern
>> or negative pattern or combination of patterns.
> 
> The list is usually very short (most of the time, only one candidate),
> so the need for sophisticated filtering is quite low.

We could add filtering by file name and/or line contents to xref buffers 
too, though.

Re: [ELPA] New package: mctags

[Stefan Monnier]

> I've been using xref for some time. As I can see, it just gives your
> the list of matches in a buffer. It can't filter further with pattern
> or negative pattern or combination of patterns.

We introduced xref to try and provide a uniform UI to all the various
ways to get etags-like functionality.  So we want to move *away* from
UIs specific to a particular etags-like tool.

So if you found xref insufficient, instead of a new package I'd much
rather see either xref extended to cover your needs (or its etags
backend extended if the missing functionality is in that backend, or
a new xref backend if extending the current backend proves
impractical).

> For example, I can filter candidates in mctags with
> "keyword1\|keyword2 !keyword3" (matching either "keyword1" or
> "keyword2" but not keyword3)

As Eli mentioned when looking for definitions, in most cases this
sophistication should not be needed, but I'm sure there can be
circumstances where it can make sense, and indeed it can make a lot of
sense when looking for references (rather than definitions), so it would
make sense to add such a filtering feature to xref.

>>> - the tags file is automatically created in current project.
>> Yes, but AFAICT, the package uses a somewhat naïve way of generating
>> TAGS, in effect passing all the non-trivial file names to etags.  Some
> Good question. For tags creation, mctags now focus on out of box
> userexperience (For example, by default, `*.o`, `*.elc` are ignored`).

Auto-creation is good indeed (personally I'd favor including only the
files which are version controlled, so as to reuse the .<foo>ignore
info already setup by the user instead of hard coding things like *.o).

I think it'd be good to add to etags.el the ability to auto-create TAGS file.

> Please note mctags RESPECTS the existing tags file created by other
> solutions. mctags will NOT override existing TAGS created by other
> programs (Makefile, for example) without user's confirmation.

The problem here is what happens if you use mctags *before* running
"make tags", in which case mctags will not know that it should respect
the (not-yet) existing tags file.

So I think the better answer is to let the user teach etags.el how to
create the TAGS file in such special cases (i.e. it can have a default
system, based on *.o thingies or based on VCS data, but that can be
overridden on a per-project basis, e.g. via .dir-locals.el).

> Grep is called if and only if mctags-find-tag fails.  It's a useful
> feature when TAGS is not updated yet.

Such a feature should fit into xref as well.

> Some people might preferring TAGS auto-updating instead of manually
> running ctags from time to time.

Agreed.  Adding that functionality to etags.el would be nice (obviously
optional as well, since again it might depend on how we build the TAGS
file and things like that).


        Stefan

Fwd: [ELPA] New package: mctags

[chen bin]

---------- Forwarded message ----------
From: chen bin <chenbin.sh@gmail.com>
Date: Sat, Oct 14, 2017 at 11:51 AM
Subject: Re: [ELPA] New package: mctags
To: Dmitry Gutov <dgutov@yandex.ru>


Sure. I also learned a lot by studying xref code.

mctags is targeted on a niche market (web front end developers) while
`xref` is a powerful one-for-all package. So they have totally
different road map.

For example, as a web developer, I prefer mctags to sacrifice
precision by fuzzy matching more candidates at the first stage. But
`xref` might prefer precision because it's also used by C/Perl
developers.

I've planed many more features for web front end developers in `mctags`.

As I know, there are a few auto-completion packages hosted on ELPA or
built in Emacs (complete-symbol, hippie-expand, company-mode, ivy's
completion at point). They all serve the users well in different
scenarios. For example, I use both hippie-expand and company-mode.
hippie-expand to auto-complete words in mini-buffer and company-mode
for anything.

Even there are some overlapping minor features between xref and
mctags, that's not a bad thing. A little bit rival is only good to
both packages, and more importantly, it gives the user choices. They
have freedom to choose the package fit in their specific need.

mctags respects the TAGS created by other programs. So it's impossible
mctags to conflicts with other programs. I understand someone has
concern about auto-updating which might override existing TAGS. But
auto-updating is disabled by default. I also have plan to make
auto-updating support any third party programs in the future by
allowing users to customize it.

Regards,
Chen

On Fri, Oct 13, 2017 at 11:33 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 10/13/17 11:47 AM, Eli Zaretskii wrote:
>
>>>> I think xref-find-definitions, when it uses the etags back-end,
>>>> already supports that, doesn't it?
>>>
>>> I've been using xref for some time. As I can see, it just gives your
>>> the list of matches in a buffer. It can't filter further with pattern
>>> or negative pattern or combination of patterns.
>>
>>
>> The list is usually very short (most of the time, only one candidate),
>> so the need for sophisticated filtering is quite low.
>
>
> We could add filtering by file name and/or line contents to xref buffers
> too, though.



--
help me, help you.


-- 
help me, help you.

Re: [ELPA] New package: mctags

[Tom Tromey]

>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

Stefan> So I think the better answer is to let the user teach etags.el how to
Stefan> create the TAGS file in such special cases (i.e. it can have a default
Stefan> system, based on *.o thingies or based on VCS data, but that can be
Stefan> overridden on a per-project basis, e.g. via .dir-locals.el).

I have some patches to etags (not etags.el) to do this.

The idea is to have a ".etags" config file that tells etags what to do.
Then "etags --find" will read this file, walk the directory tree, and
update the TAGS file.

Here's a sample config file based on what gdb's "make TAGS" currently
does:

    # Ignore some test suite files.
    !testsuite/*.[chylmsS]

    # Ignore some other things.
    !*.def
    !*/Makefile*

    # Handle DejaGNU (Tcl) files using regexps.
    *.exp    --language=none --regex='/\(gdb_caching_proc\|proc\|proc_with_prefix\)[ \t]+\([^ \t]+\)/\2/'



Making etags.el auto-update with this is simple: just run "etags --find".
It would also be easy to script this using inotify-tools.

This work still has some minor things to clean up.  I didn't make
--include work (I think --find should stop at any subdirectory .etags it
finds, and then process those directories separately -- that is, include
should be automatically handled); and there's some argument validation
to be implemented.

If this sounds interesting, I can clean it up and send it.

Tom

Re: [ELPA] New package: mctags

[Stefan Monnier]

> The idea is to have a ".etags" config file that tells etags what to do.
> Then "etags --find" will read this file, walk the directory tree, and
> update the TAGS file.

That sounds cool.  Can you also tell it to just use the files under
VCS control?


        Stefan

Re: [ELPA] New package: mctags

[Tom Tromey]

>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The idea is to have a ".etags" config file that tells etags what to do.
>> Then "etags --find" will read this file, walk the directory tree, and
>> update the TAGS file.

Stefan> That sounds cool.  Can you also tell it to just use the files under
Stefan> VCS control?

I didn't implement this; I think because since switching to git I don't
really leave source-like files lying around any more.

I suppose it could be done by adding a way for "--find" mode to take
files as input, and then asking the VCS to pipe the file list into it.

Tom

Re: [ELPA] New package: mctags

[Tom Tromey]

>>>>> "Tom" == Tom Tromey <tom@tromey.com> writes:

Tom> The idea is to have a ".etags" config file that tells etags what to do.
Tom> Then "etags --find" will read this file, walk the directory tree, and
Tom> update the TAGS file.
[...]
Tom> If this sounds interesting, I can clean it up and send it.

I still have this, and it's cleaned up a bit (still no docs but at least
now it's a clean-ish patch series).  So, if you want it, I can send it
some where.  However, it seems to me that GNU Global might be a better
fit for this space.  Why not just use it?

Tom