* Unified project interface
@ 2015-06-04 11:43 Dmitry Gutov
2015-06-04 14:40 ` Stephen Leake
` (2 more replies)
0 siblings, 3 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-04 11:43 UTC (permalink / raw)
To: emacs-devel; +Cc: Jorgen Schaefer, Bozhidar Batsov
Hi all,
A while ago, there was floated a "unified project root" proposal. I'd
like to resume that discussion.
Unfortunately, the code proposed back them doesn't serve the immediate
need I have in mind: basically, I need that function to return a *list*
of directories, because both etags and elisp xref backends operate on
multiple directories, via tags-table-list and load-path respectively.
If we have it, we'll be able to untie `xref-find-regexp' from xref
backends, because really there's nothing language-specific to this command.
Now, before adding this feature to the core, are there other related
general questions we want to be able to answer? Depending on that, a
"project" might be better implemented as a cl-struct, providing
specialized implementations for those several generic methods.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-04 11:43 Unified project interface Dmitry Gutov
@ 2015-06-04 14:40 ` Stephen Leake
2015-06-05 0:08 ` Dmitry Gutov
2015-06-06 10:20 ` Bozhidar Batsov
2015-06-06 12:32 ` Eric Ludlam
2 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-06-04 14:40 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> A while ago, there was floated a "unified project root" proposal. I'd
> like to resume that discussion.
>
> Unfortunately, the code proposed back them doesn't serve the immediate
> need I have in mind: basically, I need that function to return a
> *list* of directories, because both etags and elisp xref backends
> operate on multiple directories, via tags-table-list and load-path
> respectively.
>
> If we have it, we'll be able to untie `xref-find-regexp' from xref
> backends, because really there's nothing language-specific to this
> command.
The list of directories that xref-find-regexp needs to search is not
the project root; it is the list of directories included by the project
source code.
For that you need a function like "project-source-directories", which
could be customized for each project backend.
I handle this for ada-mode by defining a project file syntax. A project
file defines a list of source directories, among other things. The list
of source directories then gets stored in compilation-search-path.
For ada-mode, project-source-directories would just return
compilation-search-path.
The ada-mode project file can be anywhere; in my projects, it is usually
_not_ at the "project root directory", but down one or two layers in
build/ or build/release/.
I could add "prj_root" to the project file syntax; then ada-mode could
provide a "unified project root" backend that knows what the current
project file is, and returns the value from prj_root. But that would not
be useful in ada-mode; that's why it's not there yet.
The notion of "project root directory" is more useful for configuration
management; typically a config management "project" consists of the tree
of files under one root. But ada-mode knowns nothing about config
management; that's vc's job.
Hmm - vc could query for the current project root, and ada-mode would
provide the answer. That might be useful, and a good reason to have a
unified project interface.
> Now, before adding this feature to the core, are there other related
> general questions we want to be able to answer? Depending on that, a
> "project" might be better implemented as a cl-struct, providing
> specialized implementations for those several generic methods.
There are lots of project meta-data that the ada-mode project file syntax
provides; compiler options, object directory, case exceptions, etc. Some
of those might be common with other projects.
Getting a basic infrastructure in place first is a good idea; we can
always promote things that turn out to be common among many backends.
In any case, I believe a cl-struct is the way to go.
I don't think we should try to unify the xref and project backends, but
every major mode will probably want to provide both.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-04 14:40 ` Stephen Leake
@ 2015-06-05 0:08 ` Dmitry Gutov
2015-06-05 10:08 ` Stephen Leake
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-05 0:08 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 06/04/2015 05:40 PM, Stephen Leake wrote:
> The list of directories that xref-find-regexp needs to search is not
> the project root; it is the list of directories included by the project
> source code.
Not exactly. xref-find-regexp is interested in all kinds of files, not
just source files. That includes, say, the README file at the top of the
project.
> For that you need a function like "project-source-directories", which
> could be customized for each project backend.
However, you raise an interesting point. Whereas xref-find-references
would like to search in all load-path directories, maybe it would be
enough if xref-find-regexp only searches inside the current "project
root", for some definition of that notion.
I wonder how we could make it work this way. Make "project root" an
orthogonal feature?
> The ada-mode project file can be anywhere; in my projects, it is usually
> _not_ at the "project root directory", but down one or two layers in
> build/ or build/release/.
We can't rely on every Elisp project declaring a "project root" in the
same way, though.
> Hmm - vc could query for the current project root, and ada-mode would
> provide the answer. That might be useful, and a good reason to have a
> unified project interface.
VC is one option for this. Or the project may be not registered in any
VCS yet, and the root would be determined based on, say, the presence of
configure.ac. Again, this suggests that notions of "source directories"
and "project root" can be somewhat orthogonal.
On the other hand, after the project root is determined, it might want
to add some new element(s) to the source directories list.
> There are lots of project meta-data that the ada-mode project file syntax
> provides; compiler options, object directory, case exceptions, etc. Some
> of those might be common with other projects.
Right. We'll need those pieces of metadata that are useful to more than
one subsystem, and can be employed in a general way. Though a generic
metadata storage might be useful as well: thus, if some minor mode has
read the project file and parsed the compiler options, it can set the
related metadata, so that code completion and linter could use it
without waiting for Emacs core to standardize it.
> I don't think we should try to unify the xref and project backends, but
> every major mode will probably want to provide both.
Mostly those major modes that can launch and interact with a smart
external process, I think. The rest (except for Elisp) will leave it to
minor modes.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-05 0:08 ` Dmitry Gutov
@ 2015-06-05 10:08 ` Stephen Leake
2015-06-05 13:03 ` Stephen Leake
` (2 more replies)
0 siblings, 3 replies; 87+ messages in thread
From: Stephen Leake @ 2015-06-05 10:08 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 06/04/2015 05:40 PM, Stephen Leake wrote:
>
>> The list of directories that xref-find-regexp needs to search is not
>> the project root; it is the list of directories included by the project
>> source code.
>
> Not exactly. xref-find-regexp is interested in all kinds of files, not
> just source files. That includes, say, the README file at the top of
> the project.
No problem; make sure that directory is in project-source-directories.
README is a perfectly valid source file; it is written in the "plain
text" language (for which no compiler is needed ;).
I have no problem with projects including more than one language in the
sources; most of mine have Ada, Texinfo, LaTeX, text, elisp, and make
source code. Which is one reason why project and xref cannot be merged
(xref must be language-specific, even for tags).
It might make sense to parameterize project-source-directories
with a language (or major-mode) name, to get the corresponding subset.
>> For that you need a function like "project-source-directories", which
>> could be customized for each project backend.
>
> However, you raise an interesting point. Whereas xref-find-references
> would like to search in all load-path directories,
'load-path' is an elisp notion. For other languages "source-path" is
more appropriate.
> maybe it would be enough if xref-find-regexp only searches inside the
> current "project root", for some definition of that notion.
No, xref-find-regexp should search project-source-directories.
Most real projects include other projects, so limiting the search to
only the top project is wrong in general (although that might be a
useful option in some use cases).
In addition, there might be a directory under project root that should
_not_ be searched; the object file directory for ada-mode, for example.
project-source-directories should return the union of the source
directories for all of the included projects. That's what load-path is
for elisp, and what compilation-search-path is for ada-mode and other
language modes.
For elisp, project-source-directories should simply return load-path.
> I wonder how we could make it work this way. Make "project root" an
> orthogonal feature?
If you mean "orthogonal to source directories", then yes, that is what I
am suggesting.
>> The ada-mode project file can be anywhere; in my projects, it is usually
>> _not_ at the "project root directory", but down one or two layers in
>> build/ or build/release/.
>
> We can't rely on every Elisp project declaring a "project root" in the
> same way, though.
We can make it a requirement in order to use the general tool. But first
we have to justify it; ada-mode has never needed that notion; neither
has elisp. In my experience, only config management needs it.
>> Hmm - vc could query for the current project root, and ada-mode would
>> provide the answer. That might be useful, and a good reason to have a
>> unified project interface.
>
> VC is one option for this. Or the project may be not registered in any
> VCS yet, and the root would be determined based on, say, the presence
> of configure.ac. Again, this suggests that notions of "source
> directories" and "project root" can be somewhat orthogonal.
Not just "somewhat"; "completely" :).
It also points out that "project root" is poorly defined; I think that
is because it's not at all clear when and why we need it.
> On the other hand, after the project root is determined, it might want
> to add some new element(s) to the source directories list.
"it" is what here ? Can you give an example?
>> There are lots of project meta-data that the ada-mode project file syntax
>> provides; compiler options, object directory, case exceptions, etc. Some
>> of those might be common with other projects.
>
> Right. We'll need those pieces of metadata that are useful to more
> than one subsystem, and can be employed in a general way. Though a
> generic metadata storage might be useful as well: thus, if some minor
> mode has read the project file and parsed the compiler options, it can
> set the related metadata, so that code completion and linter could use
> it without waiting for Emacs core to standardize it.
ada-mode uses a plist to represent the project metadata, and has examples
of minor modes adding to the plist.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-05 10:08 ` Stephen Leake
@ 2015-06-05 13:03 ` Stephen Leake
2015-06-05 13:14 ` Dmitry Gutov
2015-06-07 23:22 ` Dmitry Gutov
2015-06-07 23:15 ` Dmitry Gutov
2015-07-24 23:43 ` Dmitry Gutov
2 siblings, 2 replies; 87+ messages in thread
From: Stephen Leake @ 2015-06-05 13:03 UTC (permalink / raw)
To: emacs-devel
Stephen Leake <stephen_leake@stephe-leake.org> writes:
> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> On 06/04/2015 05:40 PM, Stephen Leake wrote:
>>
>>> The list of directories that xref-find-regexp needs to search is not
>>> the project root; it is the list of directories included by the project
>>> source code.
>>
>> Not exactly. xref-find-regexp is interested in all kinds of files, not
>> just source files. That includes, say, the README file at the top of
>> the project.
>
> No problem; make sure that directory is in project-source-directories.
>
> README is a perfectly valid source file; it is written in the "plain
> text" language (for which no compiler is needed ;).
>
> I have no problem with projects including more than one language in the
> sources; most of mine have Ada, Texinfo, LaTeX, text, elisp, and make
> source code. Which is one reason why project and xref cannot be merged
> (xref must be language-specific, even for tags).
>
> It might make sense to parameterize project-source-directories
> with a language (or major-mode) name, to get the corresponding subset.
There are a couple of use cases here that require different functions:
1) You are editing a LaTeX file containing documentation for a Python
module implemented in Python and C++. You want to find all mentions/references
related to feature "foo", in LaTeX, Python, and C++ source files. So you
invoke project-find-regexp, which searches for *.tex, *.py, *.hhp, *.cpp
in project-source-directories.
2) You are refactoring a C++ function foo::bar, and want to find all
references to bar* for comparison. You invoke xref-find-regexp, which
searches xref-source-directories.
There are a couple of implementation options here:
- implement project-source-directories as the union of
xref-source-directories for all backends listed in a project file
- implement xref-source-directories as a subset of
project-source-directories for each backend.
I'm inclined to the latter; it's closer to what ada-mode does now.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-05 13:03 ` Stephen Leake
@ 2015-06-05 13:14 ` Dmitry Gutov
2015-06-08 1:24 ` Stephen Leake
2015-06-07 23:22 ` Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-05 13:14 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 06/05/2015 04:03 PM, Stephen Leake wrote:
> 2) You are refactoring a C++ function foo::bar, and want to find all
> references to bar* for comparison. You invoke xref-find-regexp, which
> searches xref-source-directories.
You'll call xref-find-references for that. Its default implementation
will use a similar set of directories to xref-find-regexp, but it can be
limited to the proper "source directories".
Sorry, I'll comment on the rest later.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-04 11:43 Unified project interface Dmitry Gutov
2015-06-04 14:40 ` Stephen Leake
@ 2015-06-06 10:20 ` Bozhidar Batsov
2015-06-06 10:29 ` Dmitry Gutov
2015-06-06 12:32 ` Eric Ludlam
2 siblings, 1 reply; 87+ messages in thread
From: Bozhidar Batsov @ 2015-06-06 10:20 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: Jorgen Schaefer, emacs-devel
[-- Attachment #1: Type: text/plain, Size: 1133 bytes --]
Can you explain in simple terms what exactly are the use cases for the
functionality? Being able to find definitions and grep in projects? I
recall having some discussion a while back, but the details are really
vague in my head.
On 4 June 2015 at 13:43, Dmitry Gutov <dgutov@yandex.ru> wrote:
> Hi all,
>
> A while ago, there was floated a "unified project root" proposal. I'd like
> to resume that discussion.
>
> Unfortunately, the code proposed back them doesn't serve the immediate
> need I have in mind: basically, I need that function to return a *list* of
> directories, because both etags and elisp xref backends operate on multiple
> directories, via tags-table-list and load-path respectively.
>
> If we have it, we'll be able to untie `xref-find-regexp' from xref
> backends, because really there's nothing language-specific to this command.
>
> Now, before adding this feature to the core, are there other related
> general questions we want to be able to answer? Depending on that, a
> "project" might be better implemented as a cl-struct, providing specialized
> implementations for those several generic methods.
>
[-- Attachment #2: Type: text/html, Size: 1479 bytes --]
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-06 10:20 ` Bozhidar Batsov
@ 2015-06-06 10:29 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-06 10:29 UTC (permalink / raw)
To: Bozhidar Batsov; +Cc: Jorgen Schaefer, emacs-devel
On 06/06/2015 01:20 PM, Bozhidar Batsov wrote:
> Can you explain in simple terms what exactly are the use cases for the
> functionality?
That's what this thread is about: to discuss different use cases for the
unified project interface.
> Being able to find definitions and grep in projects? I
> recall having some discussion a while back, but the details are really
> vague in my head.
xref-find-regexp will use it (always, I think). xref-find-references
will also use it, at least the default implementation.
I haven't considered, thus far, creating a default xref-find-definitions
implementation along these lines.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-04 11:43 Unified project interface Dmitry Gutov
2015-06-04 14:40 ` Stephen Leake
2015-06-06 10:20 ` Bozhidar Batsov
@ 2015-06-06 12:32 ` Eric Ludlam
2015-06-06 18:44 ` Dmitry Gutov
2 siblings, 1 reply; 87+ messages in thread
From: Eric Ludlam @ 2015-06-06 12:32 UTC (permalink / raw)
To: Dmitry Gutov, emacs-devel; +Cc: Jorgen Schaefer, Bozhidar Batsov
On 06/04/2015 07:43 AM, Dmitry Gutov wrote:
> Hi all,
>
> A while ago, there was floated a "unified project root" proposal. I'd
> like to resume that discussion.
>
[...]
>
> Now, before adding this feature to the core, are there other related
> general questions we want to be able to answer? Depending on that, a
> "project" might be better implemented as a cl-struct, providing
> specialized implementations for those several generic methods.
In that old thread, I had proposed using EDE either as a supplier to a
generic interface, or as a way to build said interface. Several people
pointed out flaws in EDE's project detection scheme. Since that time I
have accounted for those recommendations and added a severe test suite
for detecting projects more cleanly. Those changes were merged into
Emacs a while back.
As an aside, thanks for the advice back then.
In general, if there is a proposed generic project root type I'll be
happy to enabled EDE to supply data to it. Alternately, if there is an
interest in an EDE-lite where key functionality such as detection and
providing a root path is pulled up into something that doesn't turn on
all the other unwanted EDE features, I'd be happy to advise or help.
For an example, in 25.x, in cedet/ede/generic.el, look for
`ede-enable-generic-projects' for some simple definitions. I'm sure the
generic project type in EDE is overkill for what is wanted, so I'm
assuming there would need to be a stripped down version.
As a reminder, ede uses CLOS, so there is a base object representing the
project, and you can run methods against it to get information which
would be simple to expand to any type of desired generic interface.
Eric
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-06 12:32 ` Eric Ludlam
@ 2015-06-06 18:44 ` Dmitry Gutov
2015-06-06 19:28 ` Eli Zaretskii
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-06 18:44 UTC (permalink / raw)
To: Eric Ludlam, emacs-devel; +Cc: Jorgen Schaefer, Bozhidar Batsov
Hi Eric,
On 06/06/2015 03:32 PM, Eric Ludlam wrote:
> In general, if there is a proposed generic project root type I'll be
> happy to enabled EDE to supply data to it.
That's good.
> Alternately, if there is an
> interest in an EDE-lite where key functionality such as detection and
> providing a root path is pulled up into something that doesn't turn on
> all the other unwanted EDE features, I'd be happy to advise or help.
>
> For an example, in 25.x, in cedet/ede/generic.el, look for
> `ede-enable-generic-projects' for some simple definitions. I'm sure the
> generic project type in EDE is overkill for what is wanted, so I'm
> assuming there would need to be a stripped down version.
While it looks pretty decent, I'm among those'd prefer to see CEDET
moved out to GNU ELPA.
And anyway, it would be better to have the smallest possible API that,
say, Projectile could implement without relying on EDE being present and
loaded.
But any lessons EDE can teach us about making that API better, would be
great to have.
> As a reminder, ede uses CLOS, so there is a base object representing the
> project, and you can run methods against it to get information which
> would be simple to expand to any type of desired generic interface.
I imagine the new API will work along the same principle, only using
cl-generic instead of eieio.
Does EDE store information about paths outside of the current project
root? Like linked projects, or C include directories, or load-path
elements (if EDE supports Elisp projects)?
Does it allow to quickly grep across all of them, or use
semantic-symref, again, across all of them?
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-06 18:44 ` Dmitry Gutov
@ 2015-06-06 19:28 ` Eli Zaretskii
2015-06-07 22:29 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Eli Zaretskii @ 2015-06-06 19:28 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: contact, emacs-devel, bozhidar, eric
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 6 Jun 2015 21:44:12 +0300
> Cc: Jorgen Schaefer <contact@jorgenschaefer.de>,
> Bozhidar Batsov <bozhidar@batsov.com>
>
> I'm among those'd prefer to see CEDET moved out to GNU ELPA.
If we want the IDE capabilities of Emacs become better, this should
never happen.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-06 19:28 ` Eli Zaretskii
@ 2015-06-07 22:29 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-07 22:29 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: contact, emacs-devel, bozhidar, eric
On 06/06/2015 10:28 PM, Eli Zaretskii wrote:
> If we want the IDE capabilities of Emacs become better, this should
> never happen.
IME, CEDET is fairly interdependent, and Semantic's approach doesn't
seem to be useful enough for, say, code completion in dynamic languages.
I see the path forward as different CEDET features growing into more
flexible APIs that different packages can implement.
Like semantic-symref now has a generic counterpart of
xref-find-references, and while the latter has a simplistic grep-based
implementation, CEDET can define its own xref backend, to implement the
latter through the former.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-05 10:08 ` Stephen Leake
2015-06-05 13:03 ` Stephen Leake
@ 2015-06-07 23:15 ` Dmitry Gutov
2015-06-08 1:59 ` Stephen Leake
2015-07-24 23:43 ` Dmitry Gutov
2 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-07 23:15 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 06/05/2015 01:08 PM, Stephen Leake wrote:
> No problem; make sure that directory is in project-source-directories.
How would that work, in technical terms? Which entity will "make sure"
of that? Will a project-backend "source directories" implementation
always cl-call-next-method? Will the default impl simply return the
value of some variable, assigned by a major mode?
> README is a perfectly valid source file; it is written in the "plain
> text" language (for which no compiler is needed ;).
I see a few problems with that:
- README has no source code (it doesn't affect the logic of any
program). "sources" already has a common meaning, and at least in one
proprietary editor it's just one a subset of all project files (see the
nomenclature here:
https://www.jetbrains.com/idea/help/content-root.html). Further, if we
can separate them, we could make sure xref-find-references only searches
in the "proper" source directories.
- There's no direct way to find the directory holding the README when
you only have the value of load-path or compilation-search-path (because
it's quite often one level above one of the path elements). So it would
make sense to have a different name (and a different function/slot/etc)
for directories that might be reached that way.
> I have no problem with projects including more than one language in the
> sources; most of mine have Ada, Texinfo, LaTeX, text, elisp, and make
> source code. Which is one reason why project and xref cannot be merged
> (xref must be language-specific, even for tags).
xref should take the language into account, but it should work across
them as well. For instance, a Ruby method call in an *.html.erb template
file should get recognized as a reference to that method, and its "find
definition" should lead to the corresponding *.rb file.
> It might make sense to parameterize project-source-directories
> with a language (or major-mode) name, to get the corresponding subset.
That's probably not necessary: after all, only the default
xref-find-references implementation will use project-source-directories
in a naive way. A smarter xref backend should determine itself which
directories to search.
> 'load-path' is an elisp notion. For other languages "source-path" is
> more appropriate.
I didn't mean to suggest to call anything generic a "load path".
> No, xref-find-regexp should search project-source-directories.
Maybe even project-directories (project-roots?), if we agree to separate
those notions.
> Most real projects include other projects, so limiting the search to
> only the top project is wrong in general (although that might be a
> useful option in some use cases).
Maybe when called with C-u. It's not easy to allow to specify multiple
directories that way anyway.
> In addition, there might be a directory under project root that should
> _not_ be searched; the object file directory for ada-mode, for example.
project-ignore-directories should be a thing. Probably in the format of
grep-find-ignored-directories (and we might have a function
corresponding to grep-find-ignored-files). Trouble is,
projectile-ignored-directories seems to be more flexible already, so
maybe that will have to be accompanied with an upgrade to how
grep-find-ignored-directories are handled.
> We can make it a requirement in order to use the general tool. But first
> we have to justify it; ada-mode has never needed that notion; neither
> has elisp. In my experience, only config management needs it.
Like mentioned above, for example, the distinction would help with the
default xref-find-references implementation.
>> On the other hand, after the project root is determined, it might want
>> to add some new element(s) to the source directories list.
>
> "it" is what here ? Can you give an example?
Some package that defines that root-finding logic. Not every major mode
knows where the relevant source directories are, or it might not know
some of them.
If we've opened one of the files in a third-party Elisp project that
hasn't been loaded yet (so not in load-path), a project implementation
can still add an element or several to project-source-directories, so
that xref-find-references returns somewhat useful results anyway.
Or, say, for Ruby: while the major mode might conceivably be taught
about paths where to find system-wide libraries, the locations of source
files in a given project depend on the project's type, and that must be
something a project implementation knows better. Even if it'll simply
add the root to project-source-directories.
> ada-mode uses a plist to represent the project metadata, and has examples
> of minor modes adding to the plist.
Cool. Then it must be useful indeed.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-05 13:03 ` Stephen Leake
2015-06-05 13:14 ` Dmitry Gutov
@ 2015-06-07 23:22 ` Dmitry Gutov
2015-06-08 1:35 ` [SPAM UNSURE] " Stephen Leake
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-07 23:22 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 06/05/2015 04:03 PM, Stephen Leake wrote:
> 1) You are editing a LaTeX file containing documentation for a Python
> module implemented in Python and C++. You want to find all mentions/references
> related to feature "foo", in LaTeX, Python, and C++ source files. So you
> invoke project-find-regexp, which searches for *.tex, *.py, *.hhp, *.cpp
> in project-source-directories.
project-find-regexp? Do you imagine we'll need a separate command like that?
> 2) You are refactoring a C++ function foo::bar, and want to find all
> references to bar* for comparison. You invoke xref-find-regexp, which
> searches xref-source-directories.
Like mentioned, it'll hopefully be xref-find-references.
> There are a couple of implementation options here:
>
> - implement project-source-directories as the union of
> xref-source-directories for all backends listed in a project file
This sounds like you envision a one-backend-per-language kind of
situation. I think that there should be just one backend there, set in a
minor mode, that knows enough about this mixed project's structure, to
know where to search.
> - implement xref-source-directories as a subset of
> project-source-directories for each backend.
>
> I'm inclined to the latter; it's closer to what ada-mode does now.
Could you explain what that means exactly? How is it implemented?
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-05 13:14 ` Dmitry Gutov
@ 2015-06-08 1:24 ` Stephen Leake
2015-06-09 18:16 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-06-08 1:24 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 06/05/2015 04:03 PM, Stephen Leake wrote:
>
>> 2) You are refactoring a C++ function foo::bar, and want to find all
>> references to bar* for comparison. You invoke xref-find-regexp, which
>> searches xref-source-directories.
>
> You'll call xref-find-references
That handles a specific identifier, not all identifiers macthing "bar.*"
(to use proper regexp syntax, rather than shell glob).
> for that. Its default implementation
> will use a similar set of directories to xref-find-regexp, but it can
> be limited to the proper "source directories".
How will you specify two (or more?) sets of directories for xref?
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: [SPAM UNSURE] Re: Unified project interface
2015-06-07 23:22 ` Dmitry Gutov
@ 2015-06-08 1:35 ` Stephen Leake
2015-06-09 19:04 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-06-08 1:35 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 06/05/2015 04:03 PM, Stephen Leake wrote:
>
>> 1) You are editing a LaTeX file containing documentation for a Python
>> module implemented in Python and C++. You want to find all mentions/references
>> related to feature "foo", in LaTeX, Python, and C++ source files. So you
>> invoke project-find-regexp, which searches for *.tex, *.py, *.hhp, *.cpp
>> in project-source-directories.
>
> project-find-regexp? Do you imagine we'll need a separate command like
> that?
Yes, that's the point. That's how you distinuish between multi-language
and single-language searches.
>> There are a couple of implementation options here:
>>
>> - implement project-source-directories as the union of
>> xref-source-directories for all backends listed in a project file
>
> This sounds like you envision a one-backend-per-language kind of
> situation.
Yes, for xref.
> I think that there should be just one backend there, set in
> a minor mode, that knows enough about this mixed project's structure,
> to know where to search.
I don't see how that would be better; that minor mode would end up doing
what I'm suggesting the "project" code do.
>> - implement xref-source-directories as a subset of
>> project-source-directories for each backend.
>>
>> I'm inclined to the latter; it's closer to what ada-mode does now.
>
> Could you explain what that means exactly? How is it implemented?
I don't have a working example of this precise structure now. What I'm
imagining is a "project file" that lists source directories without
regard to language. To be concrete, imagine a project with the following
structure:
/root
src
*.c files
doc
*.tex files
build
foo.prj -- the project file
*.make files
foo.prj contains (among other things):
src_dir=../src
src_dir=../doc
src_dir=.
So project-source-directories is the list ("root/src" "root/doc"
"root/build").
If we are in a .c buffer, and we request xref-source-directories, the
code looks thru project-source-directories for directories that contain
c files, and returns only ("root/src"). This could be an expensive
check, so we'd want to cache the results.
Alternately, foo.prj could have:
language=c
language=tex
language=make
src_dir(c)=../src
src_dir(tex)=../doc
src_dir(make)=.
(You could get fancier and use "latex" as a language name, and have
another statement that maps that to ".tex". Which is needed for C and
Ada anyway, since they use two file extensions; .h/.c, .ads/.adb).
That imposes the requirement on the user that they know what files are
where.
Note that the directories do not have to be language exclusive; they are
here just for clarity.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-07 23:15 ` Dmitry Gutov
@ 2015-06-08 1:59 ` Stephen Leake
2015-06-09 22:31 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-06-08 1:59 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 06/05/2015 01:08 PM, Stephen Leake wrote:
>
>> No problem; make sure that directory is in project-source-directories.
>
> How would that work, in technical terms? Which entity will "make sure"
> of that?
The user; they are the only ones that know what "the project" is.
> Will a project-backend "source directories" implementation
> always cl-call-next-method?
I don't understand this.
> Will the default impl simply return the value of some variable,
> assigned by a major mode?
default implementation of what function?
>> README is a perfectly valid source file; it is written in the "plain
>> text" language (for which no compiler is needed ;).
>
> I see a few problems with that:
>
> - README has no source code (it doesn't affect the logic of any
> program).
It's the other way around; there is no object code corresponding to the
README source code. But that's not very significant.
> "sources" already has a common meaning,
What is that? I take it to mean "any human-readable file". Which thus
includes README, Makefile, .git/config, etc.
> and at least in one proprietary editor it's just one a subset of all
> project files (see the nomenclature here:
> https://www.jetbrains.com/idea/help/content-root.html).
That gives me 404
> Further, if we can separate them, we could make sure
> xref-find-references only searches in the "proper" source directories.
What do you mean by "proper"?
We need it to search what the user wants to search; the user needs to be
able to specify that clearly and easily.
> - There's no direct way to find the directory holding the README when
> you only have the value of load-path or compilation-search-path
If the user specifies compilation-search-path (directly or indirectly),
and the user wants to be able to search README, then the user must
include the directory containing README in compilation-search-path.
Is that a problem?
> So it would make sense to have a different name (and a different
> function/slot/etc) for directories that might be reached that way.
I don't follow; can you give more details, and a clear example?
>> I have no problem with projects including more than one language in the
>> sources; most of mine have Ada, Texinfo, LaTeX, text, elisp, and make
>> source code. Which is one reason why project and xref cannot be merged
>> (xref must be language-specific, even for tags).
>
> xref should take the language into account, but it should work across
> them as well. For instance, a Ruby method call in an *.html.erb
> template file should get recognized as a reference to that method, and
> its "find definition" should lead to the corresponding *.rb file.
Ah. Now you are talking about one file that mixes source in two
languages. Yes, that is a problem. There have been discussions here of
multi-mode files. I don't know how that would work in practice.
I have projects that mix Ada and C++; the project level cross reference
facility handles both, using the current xref API. That works because
gcc generates the same cross reference info for those two langauages; I
doubt there is something similar for Ruby and html.
>> It might make sense to parameterize project-source-directories
>> with a language (or major-mode) name, to get the corresponding subset.
>
> That's probably not necessary: after all, only the default
> xref-find-references implementation will use
> project-source-directories in a naive way. A smarter xref backend
> should determine itself which directories to search.
I don't see how that is at all possible, especially in the face of
hierarchical projects.
Only the user knows what a "project" is; they write the hierachical
project files that tell the system what the project structure is.
>> In addition, there might be a directory under project root that should
>> _not_ be searched; the object file directory for ada-mode, for example.
>
> project-ignore-directories should be a thing. Probably in the format
> of grep-find-ignored-directories (and we might have a function
> corresponding to grep-find-ignored-files). Trouble is,
> projectile-ignored-directories seems to be more flexible already, so
> maybe that will have to be accompanied with an upgrade to how
> grep-find-ignored-directories are handled.
>
>> We can make it a requirement in order to use the general tool. But first
>> we have to justify it; ada-mode has never needed that notion; neither
>> has elisp. In my experience, only config management needs it.
>
> Like mentioned above, for example, the distinction would help with the
> default xref-find-references implementation.
You seem to be missing my point; I see no need for project-root, outside
of configuration management. So I don't see why xref should care about
"root" at all.
>>> On the other hand, after the project root is determined, it might want
>>> to add some new element(s) to the source directories list.
>>
>> "it" is what here ? Can you give an example?
>
> Some package that defines that root-finding logic. Not every major
> mode knows where the relevant source directories are, or it might not
> know some of them.
As I've said, none of the languages I deal with care about a "root"
directory. Only git, mtn, cvs care.
Hmm. Java imposes something like a root by forcing the directory
structure to match the hiearchical package naming. But referencing other
package hierarchies complicates that, just as in Ada and C++.
> If we've opened one of the files in a third-party Elisp project that
> hasn't been loaded yet (so not in load-path), a project implementation
> can still add an element or several to project-source-directories, so
> that xref-find-references returns somewhat useful results anyway.
What does that have to do with "root"?
The third-party elisp file will presumably have some paths in "(require
...)" forms that will define a set of directories to be added to
project-source-directories.
> Or, say, for Ruby: while the major mode might conceivably be taught
> about paths where to find system-wide libraries, the locations of
> source files in a given project depend on the project's type, and that
> must be something a project implementation knows better. Even if it'll
> simply add the root to project-source-directories.
What "root"? Is that the directory containing the Ruby file that we
opened? Why is that a "root" as opposed to just some source directory?
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-08 1:24 ` Stephen Leake
@ 2015-06-09 18:16 ` Dmitry Gutov
2015-06-09 18:21 ` Eli Zaretskii
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-09 18:16 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 06/08/2015 04:24 AM, Stephen Leake wrote:
> That handles a specific identifier, not all identifiers macthing "bar.*"
> (to use proper regexp syntax, rather than shell glob).
I suppose that kind of command could be implemented on top of the
current xref API: first, find all identifiers that match this regexp
(the "apropos" action, or maybe just use the
identifier-completion-table), then find all references to each of them.
Or probably have a new generic method, do the above in the default
implementation and allow backends to override it.
> How will you specify two (or more?) sets of directories for xref?
Considering we don't currently specify even *one* set of directories
xref can use (that is one of the goals of this discussion)... How about
the same way as I'll specify that one, only twice?
Two variables, two generic methods, or so on.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-09 18:16 ` Dmitry Gutov
@ 2015-06-09 18:21 ` Eli Zaretskii
2015-06-09 18:49 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Eli Zaretskii @ 2015-06-09 18:21 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: stephen_leake, emacs-devel
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 9 Jun 2015 21:16:35 +0300
>
> On 06/08/2015 04:24 AM, Stephen Leake wrote:
>
> > That handles a specific identifier, not all identifiers macthing "bar.*"
> > (to use proper regexp syntax, rather than shell glob).
>
> I suppose that kind of command could be implemented on top of the
> current xref API: first, find all identifiers that match this regexp
> (the "apropos" action, or maybe just use the
> identifier-completion-table), then find all references to each of them.
>
> Or probably have a new generic method, do the above in the default
> implementation and allow backends to override it.
Both etags and id-utils support regexp search, so a separate method
sounds like a better idea, IMO.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-09 18:21 ` Eli Zaretskii
@ 2015-06-09 18:49 ` Dmitry Gutov
2015-06-09 19:03 ` Eli Zaretskii
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-09 18:49 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: stephen_leake, emacs-devel
On 06/09/2015 09:21 PM, Eli Zaretskii wrote:
> Both etags and id-utils support regexp search, so a separate method
> sounds like a better idea, IMO.
etags doesn't support search for references, which is what this
subthread is about. id-utils does, though.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-09 18:49 ` Dmitry Gutov
@ 2015-06-09 19:03 ` Eli Zaretskii
0 siblings, 0 replies; 87+ messages in thread
From: Eli Zaretskii @ 2015-06-09 19:03 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: stephen_leake, emacs-devel
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 9 Jun 2015 21:49:52 +0300
> Cc: stephen_leake@stephe-leake.org, emacs-devel@gnu.org
>
> On 06/09/2015 09:21 PM, Eli Zaretskii wrote:
>
> > Both etags and id-utils support regexp search, so a separate method
> > sounds like a better idea, IMO.
>
> etags doesn't support search for references, which is what this
> subthread is about.
Depends on the program that created the database, I guess.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: [SPAM UNSURE] Re: Unified project interface
2015-06-08 1:35 ` [SPAM UNSURE] " Stephen Leake
@ 2015-06-09 19:04 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-09 19:04 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 06/08/2015 04:35 AM, Stephen Leake wrote:
> Yes, that's the point. That's how you distinuish between multi-language
> and single-language searches.
That is inadequate. An ideal "project" implementation should be smart
enough to figure out whether to look for the symbol's definition in the
files with the same extension as the current one, or in files with some
other extensions as well.
Elisp is lucky to have all its environment at hand. In many cases, for
decent "find definitions" result, we'll need to know the project
structure first anyway.
>> I think that there should be just one backend there, set in
>> a minor mode, that knows enough about this mixed project's structure,
>> to know where to search.
>
> I don't see how that would be better; that minor mode would end up doing
> what I'm suggesting the "project" code do.
Maybe. Why wouldn't that be better?
> I don't have a working example of this precise structure now. What I'm
> imagining is a "project file" that lists source directories without
> regard to language. To be concrete, imagine a project with the following
> structure:
That looks decent, but if you mean that xref-source-directories would
take the whole set of source directories defined for that project and
only take those from it that correspond to the current language, that
seems unnecessarily limiting to me (unless there's a way it can be very
smart about that, and the notion of "current language" will be
generalized to possibly mean an arbitrary set of major modes and file
extensions).
Further, I don't want to have xref-source-directories as a separate
variable or a function, unless we have to.
The idea is to add a new feature to Emacs that deals with projects, and
make xref use *that*.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-08 1:59 ` Stephen Leake
@ 2015-06-09 22:31 ` Dmitry Gutov
2015-06-10 7:13 ` Steinar Bang
2015-07-08 0:25 ` Dmitry Gutov
0 siblings, 2 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-06-09 22:31 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 06/08/2015 04:59 AM, Stephen Leake wrote:
> The user; they are the only ones that know what "the project" is.
Let's cut out the user out of the discussion and just say that the
project system knows stuff. Because it can know everything we want to
ask the user; we just need to figure out the right set of questions.
>> Will a project-backend "source directories" implementation
>> always cl-call-next-method?
>
> I don't understand this.
If project-source-directories is a generic method, maybe we can say that
its implementation should almost always (unless it really knows what
it's doing) call the next applicable implementation.
That implementation could be set up to dispatch based on the value of
major-mode.
>> Will the default impl simply return the value of some variable,
>> assigned by a major mode?
>
> default implementation of what function?
project-source-directories.
>> "sources" already has a common meaning,
>
> What is that? I take it to mean "any human-readable file". Which thus
> includes README, Makefile, .git/config, etc.
Maybe there's value in allowing this semantic split to (sometimes?) go
along different lines, but the two main directions I want to see
commands work are these:
- Search all files that belong to any of the projects we're currently
interested in.
- Search only some of those files; ones that are likely programmatically
(semantically?) related to the one we're editing right now.
The latter could have a decent-enough implementation using Grep, but it
would be better if we could only search directories that are related.
Maybe it's indeed an excessive requirement, though. After all, backends
can provide their own implementations of "find references" already.
>> and at least in one proprietary editor it's just one a subset of all
>> project files (see the nomenclature here:
>> https://www.jetbrains.com/idea/help/content-root.html).
>
> That gives me 404
That link still works for me. Maybe your email client mangles it somehow?
> We need it to search what the user wants to search; the user needs to be
> able to specify that clearly and easily.
I intend to allow the user to specify the exact directory for
xref-find-regexp to search, when called with a prefix. However, allowing
to input an arbitrary set of directories is obviously non-trivial (if we
want to have an interface that's not annoying).
> If the user specifies compilation-search-path (directly or indirectly),
> and the user wants to be able to search README, then the user must
> include the directory containing README in compilation-search-path.
>
> Is that a problem?
I mentioned compilation-search-path only as an example, though I didn't
expect that you might suggest putting the project root into
compilation-search-path.
Anyway, emacs-lisp-mode doesn't set compilation-search-path. And
hopefully you're not suggesting to put project roots into load-path,
just to be able to search READMEs.
>> So it would make sense to have a different name (and a different
>> function/slot/etc) for directories that might be reached that way.
>
> I don't follow; can you give more details, and a clear example?
If my java project is in ~/vc/fooberz, then project-directories would be
'("~/vc/fooberz"), and project-source-directories would be
'("~/vc/fooberz/src"). That seems obvious enough, so maybe I don't
understand the question.
> Ah. Now you are talking about one file that mixes source in two
> languages. Yes, that is a problem. There have been discussions here of
> multi-mode files.
It's targentially related, but the difficulties in displaying and
editing multi-mode files are entirely different from the subject of this
discussion.
> I have projects that mix Ada and C++; the project level cross reference
> facility handles both, using the current xref API. That works because
> gcc generates the same cross reference info for those two langauages; I
> doubt there is something similar for Ruby and html.
Maybe, or maybe not; but that's not very relevant. If the tool *can* be
written, the API is manageable. In the meantime, we can use Grep.
>> That's probably not necessary: after all, only the default
>> xref-find-references implementation will use
>> project-source-directories in a naive way. A smarter xref backend
>> should determine itself which directories to search.
>
> I don't see how that is at all possible, especially in the face of
> hierarchical projects.
I suppose all parts of the "hierarchical project" will employ the same
project definition logic, possibly enabled by the same minor mode. And
that minor mode might as well implement the specialized xref backend
that will know where to search.
> Only the user knows what a "project" is; they write the hierachical
> project files that tell the system what the project structure is.
See the beginning of this email.
> You seem to be missing my point; I see no need for project-root, outside
> of configuration management. So I don't see why xref should care about
> "root" at all.
Or you're missing mine.
The distinction I'm (or was) suggesting is not between the root and
everything else, but between "source directories" and "other directories
the user might want to search in", and we'll often want to see the
project's root directory inside the latter list (when not in both).
>>>> On the other hand, after the project root is determined, it might want
>>>> to add some new element(s) to the source directories list.
>>>
>>> "it" is what here ? Can you give an example?
>>
>> Some package that defines that root-finding logic. Not every major
>> mode knows where the relevant source directories are, or it might not
>> know some of them.
>
> As I've said, none of the languages I deal with care about a "root"
> directory. Only git, mtn, cvs care.
Allow me to repeat: "it" is the package that defines the root-finding
logic. Defines the logic that determines the bounds of the current (and
possibly linked) project(s).
The act of find the project root directory may sometimes be skipped in
some odd cases, but otherwise, it'll usually be the first step to
determining the current project.
> Hmm. Java imposes something like a root by forcing the directory
> structure to match the hiearchical package naming. But referencing other
> package hierarchies complicates that, just as in Ada and C++.
So there'll be several roots. Or project-directories. No big deal.
>> If we've opened one of the files in a third-party Elisp project that
>> hasn't been loaded yet (so not in load-path), a project implementation
>> can still add an element or several to project-source-directories, so
>> that xref-find-references returns somewhat useful results anyway.
>
> What does that have to do with "root"?
The root directory usually, a) contains data about the project
structure, b) includes files we want to search.
> The third-party elisp file will presumably have some paths in "(require
> ...)" forms that will define a set of directories to be added to
> project-source-directories.
First of all, those forms specify path relative to *any* load-path
element. We can't add every permutation. Second, I was describing the
case where the requires load-path element weren't even set yet.
Third, either way it won't include the files that depend on the current
one. Those are important, too.
>> Or, say, for Ruby: while the major mode might conceivably be taught
>> about paths where to find system-wide libraries, the locations of
>> source files in a given project depend on the project's type, and that
>> must be something a project implementation knows better. Even if it'll
>> simply add the root to project-source-directories.
>
> What "root"? Is that the directory containing the Ruby file that we
> opened?
It's some parent directory of the currently opened file. Probably not
the direct parent. It's important because it contains files related to
the current one.
> Why is that a "root" as opposed to just some source directory?
That's a weird question. It's a root of the project. It *is* also one of
the source directories, in this example.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-09 22:31 ` Dmitry Gutov
@ 2015-06-10 7:13 ` Steinar Bang
2015-07-08 0:25 ` Dmitry Gutov
1 sibling, 0 replies; 87+ messages in thread
From: Steinar Bang @ 2015-06-10 7:13 UTC (permalink / raw)
To: emacs-devel
>>>>> Dmitry Gutov <dgutov@yandex.ru>:
> On 06/08/2015 04:59 AM, Stephen Leake wrote:
>>> and at least in one proprietary editor it's just one a subset of all
>>> project files (see the nomenclature here:
>>> https://www.jetbrains.com/idea/help/content-root.html).
>> That gives me 404
Could be the ")" or ")." at the end of the URL...?
It's always good to use a space after the end of the URL (Gnus managed
to extract the URL without the trailing ").", though...).
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-09 22:31 ` Dmitry Gutov
2015-06-10 7:13 ` Steinar Bang
@ 2015-07-08 0:25 ` Dmitry Gutov
2015-07-11 13:43 ` Bozhidar Batsov
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-08 0:25 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 06/10/2015 01:31 AM, Dmitry Gutov wrote:
> If project-source-directories is a generic method, maybe we can say that
> its implementation should almost always (unless it really knows what
> it's doing) call the next applicable implementation.
>
> That implementation could be set up to dispatch based on the value of
> major-mode.
I've pushed that to the branch 'scratch/project', please take a look.
The split between "project directories" and "project source directories"
is not 100% necessary, but I think it can be useful, provided the
semantic difference between the two is clear.
What I want to do with project-source-directories here, is to allow a
major mode to provide a default list (or, actually, a function that
would compute it, because `load-path' can change at runtime).
This way, unless a project implementation explicitly overrides it, Elisp
authors will have the whole load-path used for searching.
Maybe using a specialized implementation using &context is not the best
thing to do there. Alternatively, we can introduce a new variable (like
`project-source-directories-function'), and refer to it in the default
`project-source-directories' implementation, as well as its docstring.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-08 0:25 ` Dmitry Gutov
@ 2015-07-11 13:43 ` Bozhidar Batsov
2015-07-11 14:17 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Bozhidar Batsov @ 2015-07-11 13:43 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: Stephen Leake, emacs-devel
[-- Attachment #1: Type: text/plain, Size: 1824 bytes --]
I finally took a quick look at the current code (and the discussion so far)
and I have a few remarks:
* even within source directories it's useful to be able to exclude certain
subdirs from project-level operations. How is this going to be handled?
* it's not 100% clear to me what are major mode writes supposed to provide
as "project implementations"
* In 24.4 a `vc-root-dir` was added. Seems it overlaps a bit with
`project.el`
* how can Projectile (or a similar package) leverage `project.el`?
On 8 July 2015 at 03:25, Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 06/10/2015 01:31 AM, Dmitry Gutov wrote:
>
> If project-source-directories is a generic method, maybe we can say that
>> its implementation should almost always (unless it really knows what
>> it's doing) call the next applicable implementation.
>>
>> That implementation could be set up to dispatch based on the value of
>> major-mode.
>>
>
> I've pushed that to the branch 'scratch/project', please take a look.
>
> The split between "project directories" and "project source directories"
> is not 100% necessary, but I think it can be useful, provided the semantic
> difference between the two is clear.
>
> What I want to do with project-source-directories here, is to allow a
> major mode to provide a default list (or, actually, a function that would
> compute it, because `load-path' can change at runtime).
>
> This way, unless a project implementation explicitly overrides it, Elisp
> authors will have the whole load-path used for searching.
>
> Maybe using a specialized implementation using &context is not the best
> thing to do there. Alternatively, we can introduce a new variable (like
> `project-source-directories-function'), and refer to it in the default
> `project-source-directories' implementation, as well as its docstring.
>
>
[-- Attachment #2: Type: text/html, Size: 2418 bytes --]
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-11 13:43 ` Bozhidar Batsov
@ 2015-07-11 14:17 ` Dmitry Gutov
2015-07-12 14:42 ` Dmitry Gutov
2015-07-13 8:49 ` Bozhidar Batsov
0 siblings, 2 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-11 14:17 UTC (permalink / raw)
To: Bozhidar Batsov; +Cc: Stephen Leake, emacs-devel
On 07/11/2015 04:43 PM, Bozhidar Batsov wrote:
> I finally took a quick look at the current code (and the discussion so
> far) and I have a few remarks:
>
> * even within source directories it's useful to be able to exclude
> certain subdirs from project-level operations. How is this going to be
> handled?
A new function (or two?) will be needed for that. Is there a particular
reason Projectile has both ignored-directories and ignored-files?
I'm also undecided on what it (or they) should return. Should that be
just a list of shell globs? That would integrate fine with grep-find.
It also has both grep-find-ignored-directories and
grep-find-ignored-directories. I wonder if that was done for performance.
> * it's not 100% clear to me what are major mode writes supposed to
> provide as "project implementations"
major modes aren't really supposed to. Since the current project is
dependent on the directory and not a specific files, it's best left to
minor modes.
An implementation is a new project-find-functions element and a set of
cl-defmethod forms to go with it. Maybe also a cl-struct definition for
the project structure, but as you can see from the current project.el
contents, you can do without it.
> * In 24.4 a `vc-root-dir` was added. Seems it overlaps a bit with
> `project.el`
Only if the project is based on VC. As you can see project-try-vc uses
that backend action, if not the function itself.
> * how can Projectile (or a similar package) leverage `project.el`?
Do something along these lines:
(defun project-try-projectile ()
(when (projectile-project-root) 'projectile)
(add-hook 'project-find-functions #'project-try-projectile)
(cl-defmethod project-root (project (eql projectile))
(projectile-project-root))
...and so on.
The first difficulty will be that cl-generic (and hence, cl-defmethod)
are unavailable in the previous Emacs versions. Maybe put all this in a
separate file and (require ...) it conditionally on the Emacs version.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-11 14:17 ` Dmitry Gutov
@ 2015-07-12 14:42 ` Dmitry Gutov
2015-07-13 8:49 ` Bozhidar Batsov
1 sibling, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-12 14:42 UTC (permalink / raw)
To: Bozhidar Batsov; +Cc: Stephen Leake, emacs-devel
On 07/11/2015 05:17 PM, Dmitry Gutov wrote:
> I'm also undecided on what it (or they) should return. Should that be
> just a list of shell globs? That would integrate fine with grep-find.
I've pushed a single-function glob-based implementation.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-11 14:17 ` Dmitry Gutov
2015-07-12 14:42 ` Dmitry Gutov
@ 2015-07-13 8:49 ` Bozhidar Batsov
2015-07-13 10:23 ` Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Bozhidar Batsov @ 2015-07-13 8:49 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: Stephen Leake, emacs-devel
[-- Attachment #1: Type: text/plain, Size: 2512 bytes --]
On 11 July 2015 at 17:17, Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 07/11/2015 04:43 PM, Bozhidar Batsov wrote:
>
>> I finally took a quick look at the current code (and the discussion so
>> far) and I have a few remarks:
>>
>> * even within source directories it's useful to be able to exclude
>> certain subdirs from project-level operations. How is this going to be
>> handled?
>>
>
> A new function (or two?) will be needed for that. Is there a particular
> reason Projectile has both ignored-directories and ignored-files?
>
> I'm also undecided on what it (or they) should return. Should that be just
> a list of shell globs? That would integrate fine with grep-find.
>
> It also has both grep-find-ignored-directories and
> grep-find-ignored-directories. I wonder if that was done for performance.
Might have been an oversight on my part. Right now nothing comes to mind
and I'll have to consult the source to remember. Originally there was only
the option to ignore folders, this much I remember.
>
>
> * it's not 100% clear to me what are major mode writes supposed to
>> provide as "project implementations"
>>
>
> major modes aren't really supposed to. Since the current project is
> dependent on the directory and not a specific files, it's best left to
> minor modes.
>
e.g. Rails mode or something like this?
>
> An implementation is a new project-find-functions element and a set of
> cl-defmethod forms to go with it. Maybe also a cl-struct definition for the
> project structure, but as you can see from the current project.el contents,
> you can do without it.
>
> * In 24.4 a `vc-root-dir` was added. Seems it overlaps a bit with
>> `project.el`
>>
>
> Only if the project is based on VC. As you can see project-try-vc uses
> that backend action, if not the function itself.
>
> * how can Projectile (or a similar package) leverage `project.el`?
>>
>
> Do something along these lines:
>
> (defun project-try-projectile ()
> (when (projectile-project-root) 'projectile)
>
> (add-hook 'project-find-functions #'project-try-projectile)
>
> (cl-defmethod project-root (project (eql projectile))
> (projectile-project-root))
>
> ...and so on.
>
> The first difficulty will be that cl-generic (and hence, cl-defmethod) are
> unavailable in the previous Emacs versions. Maybe put all this in a
> separate file and (require ...) it conditionally on the Emacs version.'
>
Yeah, backward compatibility is problematic. I guess those can't be
backported to the `cl-lib` package in ELPA?
[-- Attachment #2: Type: text/html, Size: 3842 bytes --]
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-13 8:49 ` Bozhidar Batsov
@ 2015-07-13 10:23 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-13 10:23 UTC (permalink / raw)
To: Bozhidar Batsov; +Cc: Stephen Leake, emacs-devel
On 07/13/2015 11:49 AM, Bozhidar Batsov wrote:
> Might have been an oversight on my part. Right now nothing comes to mind
> and I'll have to consult the source to remember. Originally there was only
> the option to ignore folders, this much I remember.
I've looked at the code, and you have two functions that returns globs
to ignore: projectile-paths-to-ignore and projectile-patterns-to-ignore.
They're split along different lines, though (rooted vs non-rooted
patterns). I'm not sure how useful that would be.
And projectile-ignored-files/directories return actual file names inside
the project, that's not what I want here either.
For now, it seems you'll implement `project-ignores' by using the value
returned by `projectile-parse-dirconfig-file' (with a slight change:
replacing / at bos with ./)
> e.g. Rails mode or something like this?
Yep. ENSIME, Malabar, etc. Or Projectile.
> Yeah, backward compatibility is problematic. I guess those can't be
> backported to the `cl-lib` package in ELPA?
Well, maybe if there's enough of a demand someone will do it.
It's not there yet, and I'm not sure how much work it'll require.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-06-05 10:08 ` Stephen Leake
2015-06-05 13:03 ` Stephen Leake
2015-06-07 23:15 ` Dmitry Gutov
@ 2015-07-24 23:43 ` Dmitry Gutov
2015-07-25 0:55 ` Stephen Leake
2 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-24 23:43 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
Let's maybe continue this discussion?
On 06/05/2015 01:08 PM, Stephen Leake wrote:
> No, xref-find-regexp should search project-source-directories.
>
> Most real projects include other projects, so limiting the search to
> only the top project is wrong in general (although that might be a
> useful option in some use cases).
I suppose, and maybe I was wrong to suggest a distinction between
"source" directories and the rest of them. However, here's one which
could be better: directories to search vs. directories to modify.
Like, I might want to search the full system includes path, in a C
project, but when performing a replacement operation (such as
xref-query-replace), I should probably limit myself to the directories
with code I actually own. At the moment, it's the distinction I imagine
for project-search-path vs project-directories.
> In addition, there might be a directory under project root that should
> _not_ be searched; the object file directory for ada-mode, for example.
We have that in project-ignores now.
> We can make [root] a requirement in order to use the general tool. But first
> we have to justify it; ada-mode has never needed that notion; neither
> has elisp. In my experience, only config management needs it.
> ...
> It also points out that "project root" is poorly defined; I think that
> is because it's not at all clear when and why we need it.
Here's the first use case I know of:
There's a third-party project called rspec-mode. It allows one to switch
between a source file and its test, and to run tests in a compilation
buffer. For both purposes, it needs to determine the directory which to
build the relative paths against, and in which to run the 'rspec'
program. Currently, it does a locate-dominating-file like routine
(looking for Gemfile or Rakefile), but it could switch to the project
API in the future.
You're probably aware of similar tools.
So, now we have both project-root and project-directories. However, the
relationship between them looks murky, from the consumer's standpoint.
Like, must the former always be included in the latter?
I'm considering removing project-root, and copying a part of Atom's API
[0]: adding a function project-relativize-path, which will, given an
arbitrary directory, if it's inside project-directories, split its file
name in two and return a cons, with one of project-directories in the
car, and the relative file name against it in the cdr.
Then instead of project-root, a consumer will use (car
(project-relativize-path default-directory)).
Thoughts?
[0] https://atom.io/docs/api/v1.0.2/Project#instance-relativizePath
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-24 23:43 ` Dmitry Gutov
@ 2015-07-25 0:55 ` Stephen Leake
2015-07-25 7:29 ` Eli Zaretskii
2015-07-26 2:11 ` Dmitry Gutov
0 siblings, 2 replies; 87+ messages in thread
From: Stephen Leake @ 2015-07-25 0:55 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> Let's maybe continue this discussion?
>
> On 06/05/2015 01:08 PM, Stephen Leake wrote:
>
>> No, xref-find-regexp should search project-source-directories.
>>
>> Most real projects include other projects, so limiting the search to
>> only the top project is wrong in general (although that might be a
>> useful option in some use cases).
>
> I suppose, and maybe I was wrong to suggest a distinction between
> "source" directories and the rest of them. However, here's one which
> could be better: directories to search vs. directories to modify.
>
> Like, I might want to search the full system includes path, in a C
> project, but when performing a replacement operation (such as
> xref-query-replace), I should probably limit myself to the directories
> with code I actually own.
Yes, that's a good use case.
> At the moment, it's the distinction I
> imagine for project-search-path vs project-directories.
I would suggest the terminology "main project" vs "included projects"
for this; the "main project" is the code you own; the "included
projects" are all the others (included the system includes).
So this would be an argument to xref-find-regexp;
(defun xref-find-regexp (regexp &optional not-included-projects)
"Search all files in all project directories for REGEXP.
If NOT-INCLUDED-PROJECTS is non-nil (default nil), search only the main
project. Otherwise, search the main project and included projects."
To implement this, project-search-path might want to be an alist of:
(PROJECT-NAME <search-path>)
where the first element is the main project and the rest are the
included ones.
>> In addition, there might be a directory under project root that should
>> _not_ be searched; the object file directory for ada-mode, for example.
>
> We have that in project-ignores now.
Right. That's not consistent with current things like load-path. I
should think at some point in the implementation of any "search project"
function you will want an explicit list of directories (and maybe files)
to search, which will be the union of project-search-path minus
project-ignores. So it seems simpler to specify that list directly.
The user interface (the syntax of the project file) could be structured
as includes and ignores (excludes).
>> We can make [root] a requirement in order to use the general tool. But first
>> we have to justify it; ada-mode has never needed that notion; neither
>> has elisp. In my experience, only config management needs it.
>> ...
>> It also points out that "project root" is poorly defined; I think that
>> is because it's not at all clear when and why we need it.
>
> Here's the first use case I know of:
>
> There's a third-party project called rspec-mode. It allows one to
> switch between a source file and its test, and to run tests in a
> compilation buffer. For both purposes, it needs to determine the
> directory which to build the relative paths against, and in which to
> run the 'rspec' program. Currently, it does a locate-dominating-file
> like routine (looking for Gemfile or Rakefile), but it could switch to
> the project API in the future.
Ok, makes sense.
> You're probably aware of similar tools.
monotone, git, subversion ... :).
Recently I've been playing with Google's Android Studio (shudder; only
to make Emacs do what it does, I promise :). It defines a project root
by a dominating file.
On the other hand, AdaCore GPS (an Ada-specific IDE) defines a project
by a project file that specifies various lists of directories (source,
object, executable); there is no root directory. The project file itself
can be anywhere, it doesn't have to be in one of the listed directories.
That's what Emacs Ada mode does as well.
> So, now we have both project-root and project-directories. However,
> the relationship between them looks murky, from the consumer's
> standpoint. Like, must the former always be included in the latter?
I think they are disjoint concepts. Nothing wrong with keeping both;
some tools will need them, some won't.
> I'm considering removing project-root, and copying a part of Atom's
> API [0]: adding a function project-relativize-path, which will, given
> an arbitrary directory, if it's inside project-directories, split its
> file name in two and return a cons, with one of project-directories in
> the car, and the relative file name against it in the cdr.
If project-directories contains all directories (not just root ones),
this is just (file-name-directory path) . (file-name-non-directory
path).
> Then instead of project-root, a consumer will use (car
> (project-relativize-path default-directory)).
I don't follow; that means the value of 'root directory' changes
depending on where the current file is.
Unless project-directories contains only one directory, which would then
be the root.
Given the variable project-root, it makes sense to define
project-relativize-path to return the portion of the path that is
relative to project-root, and error if there is no such portion (ie an
absolute path outside the root tree). Monotone has such a function in
the C++ code.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-25 0:55 ` Stephen Leake
@ 2015-07-25 7:29 ` Eli Zaretskii
2015-07-26 2:12 ` Dmitry Gutov
2015-07-26 2:11 ` Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Eli Zaretskii @ 2015-07-25 7:29 UTC (permalink / raw)
To: Stephen Leake; +Cc: emacs-devel
> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Fri, 24 Jul 2015 19:55:09 -0500
>
> I would suggest the terminology "main project" vs "included projects"
> for this; the "main project" is the code you own; the "included
> projects" are all the others (included the system includes).
IMO, system header files are not the same as "included projects",
because the system headers are immutable.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-25 0:55 ` Stephen Leake
2015-07-25 7:29 ` Eli Zaretskii
@ 2015-07-26 2:11 ` Dmitry Gutov
2015-07-26 11:22 ` Stephen Leake
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-26 2:11 UTC (permalink / raw)
To: emacs-devel
Hi Stephen,
Sorry for the delay; I had to think on it.
On 07/25/2015 03:55 AM, Stephen Leake wrote:
> I would suggest the terminology "main project" vs "included projects"
> for this; the "main project" is the code you own; the "included
> projects" are all the others (included the system includes).
There's sense in this, but is "an included project you don't own, which
is not in system includes" a significant use case?
In my own experience, either the projects on which the current one
depends are installed in system directories, and thus are
indistinguishable from system includes (such as Ruby gems, installed via
"gem install"), or the linked project's tree lives near the current one,
and we can consider it a part of the main project, for the purposes of
editing.
Because even if the second project is a third-party dependency,
sometimes you want to make changes to it anyway, and the user might get
miffed if xref-query-replace prohibits that.
> So this would be an argument to xref-find-regexp;
>
> (defun xref-find-regexp (regexp &optional not-included-projects)
> "Search all files in all project directories for REGEXP.
> If NOT-INCLUDED-PROJECTS is non-nil (default nil), search only the main
> project. Otherwise, search the main project and included projects."
It should probably be a new command, because otherwise specifying that
second argument interactively would be a pain (C-u already makes
xref-find-regexp ask a bunch of questions).
> To implement this, project-search-path might want to be an alist of:
>
> (PROJECT-NAME <search-path>)
>
> where the first element is the main project and the rest are the
> included ones.
Maybe do it differently?
If we do have linked projects (and the separation between
main/included), some generic function, probably named differently, would
return a list of those project instances. Then the caller will ask each
instance for its root (or the directories).
There's no need for the main project's search-path entry there,
xref-find-regexp can just use its root (or the directories) as well.
>> We have that in project-ignores now.
>
> Right. That's not consistent with current things like load-path. I
Indeed. But maybe supporting ignores for project-search-path is not a
big priority. There's not much to ignore in /usr/include, for instance.
The directories in load-path also usually don't contain weird files or
build artefacts (and when they do, it's not like elisp-search-path
really can account for those). So maybe a good first approximation would
be to simply not apply the ignores list to the project-search-path
elements outside of the project tree.
If that's not good enough, project-ignores can accept a directory as its
argument (it must be among those returned by project-directories, or
project-search-path), and return the appropriate ignores list.
This will definitely help if each element in project-directories is a
separate (e.g. Git) repository, with its own .gitignore.
> should think at some point in the implementation of any "search project"
> function you will want an explicit list of directories (and maybe files)
> to search, which will be the union of project-search-path minus
> project-ignores. So it seems simpler to specify that list directly.
I'm not sure about that. It seems then the implementation will have to
walk the whole project tree itself, and collect all directories that
don't match the ignores (because .gitignore syntax allows not-anchored
entries for directories). And if we're to support ignoring files (and
obvious use case), it'll become a list of files, which can grow pretty
big. I think it's better to delegate this job to 'find', as long as that
doesn't force us to compromise on functionality.
> The user interface (the syntax of the project file) could be structured
> as includes and ignores (excludes).
.gitignore also supports whitelisting entries (which override the
excludes). I haven't gotten around to writing support for those.
>> You're probably aware of similar tools.
>
> monotone, git, subversion ... :).
Not sure about the other two, but Git doesn't need to be told the
project root. It can be called from any directory, and works with
relative paths quite well.
> Recently I've been playing with Google's Android Studio (shudder; only
> to make Emacs do what it does, I promise :). It defines a project root
> by a dominating file.
Of course. Why else one would be playing with it? ;)
In this case, project-directories can return a list with one element.
> On the other hand, AdaCore GPS (an Ada-specific IDE) defines a project
> by a project file that specifies various lists of directories (source,
> object, executable); there is no root directory. The project file itself
> can be anywhere, it doesn't have to be in one of the listed directories.
> That's what Emacs Ada mode does as well.
In this case, it seems, the relevant project implementation will be
based on user explicitly telling it where the project file lives (by
calling a particular command, with a prompt), and then the project will
be enabled globally, for the whole Emacs session.
project-directories then will return some of the directories specified
in the project file (except the object ones, maybe, if there's nothing
we can edit there), but probably (?) not the one where the project file
lies, if it's somewhere separately.
Would there be any significant value in project-root returning the
directory of the project file?
> I think they are disjoint concepts. Nothing wrong with keeping both;
> some tools will need them, some won't.
That would mean documenting the distinction, so that implementers know
what to do. I'm increasingly less clear on it.
> If project-directories contains all directories (not just root ones),
> this is just (file-name-directory path) . (file-name-non-directory
> path).
It would be. But I expect project-directories to only include the roots.
IOW, no project-directories element should be a subdirectory of another.
>> Then instead of project-root, a consumer will use (car
>> (project-relativize-path default-directory)).
>
> I don't follow; that means the value of 'root directory' changes
> depending on where the current file is.
That's correct. If the file is inside the "main" project directory, it
will use the main project root. If the file is in another directory,
it's in a "linked" body of code (which would often be called a project
as well), and its root is used.
Atom has taken this approach: each element in
atom.project.getDirectories() corresponds to a Git repository. See the
link I sent previously, and also https://github.com/atom/atom/pull/3691.
This may look wrong for certain configurations (your AdaCore example
comes to mind), but would the concept of root be even useful in that case?
> Given the variable project-root, it makes sense to define
> project-relativize-path to return the portion of the path that is
> relative to project-root, and error if there is no such portion (ie an
> absolute path outside the root tree). Monotone has such a function in
> the C++ code.
Sure. But if project-root is not among project-directories, I'm not sure
when, and for what cases, that directory can be used.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-25 7:29 ` Eli Zaretskii
@ 2015-07-26 2:12 ` Dmitry Gutov
2015-07-26 2:45 ` Eli Zaretskii
2015-07-26 11:25 ` Stephen Leake
0 siblings, 2 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-26 2:12 UTC (permalink / raw)
To: Eli Zaretskii, Stephen Leake; +Cc: emacs-devel
On 07/25/2015 10:29 AM, Eli Zaretskii wrote:
> IMO, system header files are not the same as "included projects",
> because the system headers are immutable.
AFAICT, that's exactly what Stephen meant by "included projects":
projects, whose code we're not allowed to change.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 2:12 ` Dmitry Gutov
@ 2015-07-26 2:45 ` Eli Zaretskii
2015-07-26 11:25 ` Stephen Leake
1 sibling, 0 replies; 87+ messages in thread
From: Eli Zaretskii @ 2015-07-26 2:45 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: stephen_leake, emacs-devel
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 26 Jul 2015 05:12:44 +0300
> Cc: emacs-devel@gnu.org
>
> On 07/25/2015 10:29 AM, Eli Zaretskii wrote:
>
> > IMO, system header files are not the same as "included projects",
> > because the system headers are immutable.
>
> AFAICT, that's exactly what Stephen meant by "included projects":
> projects, whose code we're not allowed to change.
Then the name is misleading.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 2:11 ` Dmitry Gutov
@ 2015-07-26 11:22 ` Stephen Leake
2015-07-26 17:23 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-07-26 11:22 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> Hi Stephen,
>
> Sorry for the delay; I had to think on it.
>
> On 07/25/2015 03:55 AM, Stephen Leake wrote:
>
>> I would suggest the terminology "main project" vs "included projects"
>> for this; the "main project" is the code you own; the "included
>> projects" are all the others (included the system includes).
>
> There's sense in this, but is "an included project you don't own,
> which is not in system includes" a significant use case?
Yes; any third party library that you should not edit, because the
sources are controlled upstream.
> In my own experience, either the projects on which the current one
> depends are installed in system directories, and thus are
> indistinguishable from system includes (such as Ruby gems, installed
> via "gem install"),
Ok. I don't see the notion of "system directories" as important here;
the important point is whether the libraries are read-only or
read/write. We should not rely on where they happen to reside to decide
that; sometimes I edit "system directories" when working on patches for
upstream.
Or just ignore the read-only issue in the project code; file permissions
are good enough.
>> should think at some point in the implementation of any "search project"
>> function you will want an explicit list of directories (and maybe files)
>> to search, which will be the union of project-search-path minus
>> project-ignores. So it seems simpler to specify that list directly.
>
> I'm not sure about that. It seems then the implementation will have to
> walk the whole project tree itself, and collect all directories that
> don't match the ignores (because .gitignore syntax allows not-anchored
> entries for directories).
Yes. You do that once when the project is opened, and cache the result
in the variable project-search-path. Then all code that searches
projects is simpler; it deals with a simple list of directories, plus a
list of file extensions (just as locate-file does).
>> The user interface (the syntax of the project file) could be structured
>> as includes and ignores (excludes).
>
> .gitignore also supports whitelisting entries (which override the
> excludes). I haven't gotten around to writing support for those.
Right. .gitignore is part of "the project file" in this sense.
>> Recently I've been playing with Google's Android Studio (shudder; only
>> to make Emacs do what it does, I promise :). It defines a project root
>> by a dominating file.
>
> Of course. Why else one would be playing with it? ;)
>
> In this case, project-directories can return a list with one element.
No, Emacs needs to read the Studio project file and extract the search
path; it's just another project file syntax.
>> On the other hand, AdaCore GPS (an Ada-specific IDE) defines a project
>> by a project file that specifies various lists of directories (source,
>> object, executable); there is no root directory. The project file itself
>> can be anywhere, it doesn't have to be in one of the listed directories.
>> That's what Emacs Ada mode does as well.
>
> In this case, it seems, the relevant project implementation will be
> based on user explicitly telling it where the project file lives (by
> calling a particular command, with a prompt), and then the project
> will be enabled globally, for the whole Emacs session.
Yes. No different from any other Emacs project; the user has to tell
Emacs where it is. If that's a directory, Emacs searches for project
file extensions it knows about, and reads that. If it's a file, no
searching needed.
> Would there be any significant value in project-root returning the
> directory of the project file?
Emacs should trust what the project file says; it has an explicit list of
"source code directories". Emacs should not care whether that contains
the project file itself or not; that's up to the user.
Since the external tools that use the project file can report errors in
its syntax, I usually include the project file in a directory that is in
the source path. But that's my choice, not Emacs's.
>> I think they are disjoint concepts. Nothing wrong with keeping both;
>> some tools will need them, some won't.
>
> That would mean documenting the distinction, so that implementers know
> what to do. I'm increasingly less clear on it.
project-root; a single absolute directory, usually containing
significant files for external tools (ie .git/), often the root of the
source tree.
note that the _only_ requirement here is that it is a single
absolute directory; the rest is just suggestions.
project-search-path; a list of directories, containing files related to
the project. This will normally include project-root. It may also
contain included projects.
this might want to be split into project-source-search-path,
project-object-search-path, but since Emacs is mostly interested in
editing sources, only the source path is interesting.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 2:12 ` Dmitry Gutov
2015-07-26 2:45 ` Eli Zaretskii
@ 2015-07-26 11:25 ` Stephen Leake
1 sibling, 0 replies; 87+ messages in thread
From: Stephen Leake @ 2015-07-26 11:25 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/25/2015 10:29 AM, Eli Zaretskii wrote:
>
>> IMO, system header files are not the same as "included projects",
>> because the system headers are immutable.
>
> AFAICT, that's exactly what Stephen meant by "included projects":
> projects, whose code we're not allowed to change.
No, _some_ included projects are read-only; others are read-write. I
don't think the distinction is useful at the project level; file
permissions handle it.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 11:22 ` Stephen Leake
@ 2015-07-26 17:23 ` Dmitry Gutov
2015-07-26 18:57 ` Stephen Leake
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-26 17:23 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/26/2015 02:22 PM, Stephen Leake wrote:
> Yes; any third party library that you should not edit, because the
> sources are controlled upstream.
Is it common to have its sources checked out in a directory nearby?
> Ok. I don't see the notion of "system directories" as important here;
> the important point is whether the libraries are read-only or
> read/write. We should not rely on where they happen to reside to decide
> that; sometimes I edit "system directories" when working on patches for
> upstream.
There is a difference, from my perspective: usually, there are no object
files in system libraries, no build files, or .gitignore files.
> Or just ignore the read-only issue in the project code; file permissions
> are good enough.
Maybe ignoring that is fine (are we backtracking on the separation of
the directories for the purposes of search and edit operations?), but
file permissions won't help in a lot of cases.
In Ruby and Python communities, for instance, it's common to install the
language runtime, with all libraries, inside the developer's home
directory (using RVM/rbenv for Ruby, or virtualenv for Python). And I
might edit some of files there for debugging purposes, but if I forget
about those edits later, only bad can come of it. Likewise, I usually
wouldn't want a global replace operation to touch them.
>> I'm not sure about that. It seems then the implementation will have to
>> walk the whole project tree itself, and collect all directories that
>> don't match the ignores (because .gitignore syntax allows not-anchored
>> entries for directories).
>
> Yes. You do that once when the project is opened, and cache the result
And then the user creates a new dir (or switches to a different branch,
revision, etc), and the cache goes out of sync. How do we handle that?
On the other hand, delegating the scan of the filesystem to 'find' seems
performant enough. Someone correct me if I'm wrong, but most of the time
is spent in Grep anyway.
> in the variable project-search-path. Then all code that searches
> projects is simpler; it deals with a simple list of directories, plus a
> list of file extensions (just as locate-file does).
locate-file seems to use a whitelist of suffixes, which is distinctly
less powerful than with what's offered by .gitignore. I think keeping
compatibility with it (and similar .bzrignore and .hgignore) is valuable.
> No, Emacs needs to read the Studio project file and extract the search
> path; it's just another project file syntax.
But according to you, the "search path" will be just the directories
inside the project's tree, right? That can be determined solely on the
location of that file.
What will "included projects" look like, in an average Android project?
> Yes. No different from any other Emacs project; the user has to tell
> Emacs where it is. If that's a directory, Emacs searches for project
> file extensions it knows about, and reads that. If it's a file, no
> searching needed.
The VC project implementation is currently based on the current buffer's
directory. No need to explicitly specify it, Emacs find the repo root
without user intervention.
If you've ever tried Projectile, it works in a similar fashion. Android
projects could be detected like that as well.
> Emacs should trust what the project file says; it has an explicit list of
> "source code directories". Emacs should not care whether that contains
> the project file itself or not; that's up to the user.
Will this miss all the other directories in the project, such as
documentation, maybe? One would expect xref-find-regexp to search them, too.
> Since the external tools that use the project file can report errors in
> its syntax, I usually include the project file in a directory that is in
> the source path. But that's my choice, not Emacs's.
One would expect xref-find-regexp to search in the project file, too.
> project-root; a single absolute directory, usually containing
> significant files for external tools (ie .git/), often the root of the
> source tree.
Would we have project-root mainly for the purpose of xref-find-regexp
being able to search in the project file?
If project-root is not in project-search-path, can there be other,
unrelated files in that directory? How would we make sure
xref-find-regexp ignores them?
Seems like any other tool that would use it would be specific to that
kind of project (such as project file checker). Maybe the path to the
project file should be set via metadata, using a key like ada-project-file.
> note that the _only_ requirement here is that it is a single
> absolute directory; the rest is just suggestions.
That's a problem, IMO. If project-root is loosely defined, it's not
useful to third-party code that doesn't know much more about your kind
of project.
And as per below, we're be including source directories from both the
current project and the included projects into project-search-path,
right? Seemingly on equal rights. But then, each of those projects
likely has a .gitignore, a build file, and so on, but project-root will
only return the location of project files for the current project? That
seems inconsistent.
> project-search-path; a list of directories, containing files related to
> the project. This will normally include project-root. It may also
> contain included projects.
So, this tries to combine project-search-path and project-directories,
as they're defined now? From the previous email, I imagined a
project-specific project-search-path plus an accessor like
project-dependencies, which would contain a list of project objects
corresponding to the included projects.
> this might want to be split into project-source-search-path,
> project-object-search-path, but since Emacs is mostly interested in
> editing sources, only the source path is interesting.
Right. I'd call it project-directories, though.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 17:23 ` Dmitry Gutov
@ 2015-07-26 18:57 ` Stephen Leake
2015-07-26 23:56 ` John Yates
2015-07-27 13:00 ` Dmitry Gutov
0 siblings, 2 replies; 87+ messages in thread
From: Stephen Leake @ 2015-07-26 18:57 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> In Ruby and Python communities, for instance, it's common to install
> the language runtime, with all libraries, inside the developer's home
> directory (using RVM/rbenv for Ruby, or virtualenv for Python). And I
> might edit some of files there for debugging purposes, but if I forget
> about those edits later, only bad can come of it. Likewise, I usually
> wouldn't want a global replace operation to touch them.
Then it's not "global", is it? It's "operate on this set of projects,
exclude that set". So you need a way to specify sets of projects.
>>> I'm not sure about that. It seems then the implementation will have to
>>> walk the whole project tree itself, and collect all directories that
>>> don't match the ignores (because .gitignore syntax allows not-anchored
>>> entries for directories).
>>
>> Yes. You do that once when the project is opened, and cache the result
>
> And then the user creates a new dir (or switches to a different
> branch, revision, etc), and the cache goes out of sync. How do we
> handle that?
project-cache-refresh. Only the user knows when it should be done; they
may want to keep the slightly out of date cache because something isn't
finished with the edits yet.
>> No, Emacs needs to read the Studio project file and extract the search
>> path; it's just another project file syntax.
>
> But according to you, the "search path" will be just the directories
> inside the project's tree, right?
No, I have been explicitly saying that is not the case; the search path
is whatever the project file says it is.
> What will "included projects" look like, in an average Android
> project?
A list of project names. Depending on the project manager, that implies
a list of project files in one syntax or another, each of which
specifies a set of project directories (explicitly or implicitly).
>> Yes. No different from any other Emacs project; the user has to tell
>> Emacs where it is. If that's a directory, Emacs searches for project
>> file extensions it knows about, and reads that. If it's a file, no
>> searching needed.
>
> The VC project implementation is currently based on the current
> buffer's directory. No need to explicitly specify it, Emacs find the
> repo root without user intervention.
Yes.
> If you've ever tried Projectile, it works in a similar fashion.
> Android projects could be detected like that as well.
Maybe. Depends on the project manager software. Some have a dominating
file, some do it some other way.
Why do we care? Just support whatever method the project manger backend
requires. project-find-project-file is a dispatching function on the backend.
>> Emacs should trust what the project file says; it has an explicit list of
>> "source code directories". Emacs should not care whether that contains
>> the project file itself or not; that's up to the user.
>
> Will this miss all the other directories in the project, such as
> documentation, maybe? One would expect xref-find-regexp to search
> them, too.
Trust what the project file says. The user told Emacs to use the project
file; it should not try to be smarter.
If the user wants documentation in the project, then it is in the
project file. If the project manager tool is brain-dead and can't handle
documentation, the the user can switch tools.
>> Since the external tools that use the project file can report errors in
>> its syntax, I usually include the project file in a directory that is in
>> the source path. But that's my choice, not Emacs's.
>
> One would expect xref-find-regexp to search in the project file, too.
Only if I tell it to by including it in the project file. If I tell it
to _not_ search the project file, it should do that.
>> project-root; a single absolute directory, usually containing
>> significant files for external tools (ie .git/), often the root of the
>> source tree.
>
> Would we have project-root mainly for the purpose of xref-find-regexp
> being able to search in the project file?
No. We should have project-root for tools that need a root directory.
xref-find-regexp needs a search path, not a root directory.
> If project-root is not in project-search-path, can there be other,
> unrelated files in that directory? How would we make sure
> xref-find-regexp ignores them?
xref-find-regexp searches project-search-path. project-search-path is
set by reading the project files. The project files are provided by the
user. Voila, xref-find-regexp does _exactly_ what the user wants, no
more, no less.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 18:57 ` Stephen Leake
@ 2015-07-26 23:56 ` John Yates
2015-07-27 1:49 ` Dmitry Gutov
2015-07-27 11:12 ` Stephen Leake
2015-07-27 13:00 ` Dmitry Gutov
1 sibling, 2 replies; 87+ messages in thread
From: John Yates @ 2015-07-26 23:56 UTC (permalink / raw)
To: Stephen Leake; +Cc: Emacs developers
[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]
On Sun, Jul 26, 2015 at 2:57 PM, Stephen Leake <
stephen_leake@stephe-leake.org> wrote:
> > And then the user creates a new dir (or switches to a different
> > branch, revision, etc), and the cache goes out of sync. How do we
> > handle that?
>
> project-cache-refresh. Only the user knows when it should be done; they
> may want to keep the slightly out of date cache because something isn't
> finished with the edits yet.
This seems to me to be "rationalizing a zit". I conjecture that you only
advocate such a UI because you imagine a not-particularly performant
implementation. If there were no cost to keeping the cache current
(equivalent to stating that a naive user never would _never_ need to learn
of that cache) would you still advocate such a UI?
In small projects the cost of a simplistic brute-force change determination
is likely to be entirely acceptable. On systems that support emacs'
file-notify watching every directory within a project should be able to
keep cache refresh overhead to negligible levels (potentially improved by
only watch the modifiable portions of the project).
In the exceedingly rare case where a user needs to keep work hidden from
project searches he can either modify the project definition to blacklist
temporarily the areas he wants excluded. Alternatively we could provide a
toggle for very advanced users to suppress cache refersh.
Bottom line: start by getting the model of state and the UI right. After
that if the performance in corner cases needs help then give advanced users
tools to tune the system. But please do not foist a user-maintained cache
on casual users.
/john
[-- Attachment #2: Type: text/html, Size: 2190 bytes --]
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 23:56 ` John Yates
@ 2015-07-27 1:49 ` Dmitry Gutov
2015-07-27 11:12 ` Stephen Leake
1 sibling, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-27 1:49 UTC (permalink / raw)
To: John Yates, Stephen Leake; +Cc: Emacs developers
On 07/27/2015 02:56 AM, John Yates wrote:
> If there were no cost to keeping the cache
> current (equivalent to stating that a naive user never would _never_
> need to learn of that cache) would you still advocate such a UI?
I think the word "cache" itself implies a storage that can get stale.
> Bottom line: start by getting the model of state and the UI right.
> After that if the performance in corner cases needs help then give
> advanced users tools to tune the system. But please do not foist a
> user-maintained cache on casual users.
Right. Even if there's a cache mechanism, I'd very much like to see its
invalidation by automatic means (via file-notify, or similar), at least
by default.
Anyway, I'd prefer to implement as many operations as feasible without
needing that cache. If the search through the project tree spends an
order of magnitude more time searching than listing the files, there's
not point in caching the latter.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 23:56 ` John Yates
2015-07-27 1:49 ` Dmitry Gutov
@ 2015-07-27 11:12 ` Stephen Leake
2015-07-27 11:27 ` Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-07-27 11:12 UTC (permalink / raw)
To: emacs-devel
John Yates <john@yates-sheets.org> writes:
> On Sun, Jul 26, 2015 at 2:57 PM, Stephen Leake <
> stephen_leake@stephe-leake.org> wrote:
>
>> > And then the user creates a new dir (or switches to a different
>> > branch, revision, etc), and the cache goes out of sync. How do we
>> > handle that?
>>
>> project-cache-refresh. Only the user knows when it should be done; they
>> may want to keep the slightly out of date cache because something isn't
>> finished with the edits yet.
>
>
> This seems to me to be "rationalizing a zit". I conjecture that you only
> advocate such a UI because you imagine a not-particularly performant
> implementation. If there were no cost to keeping the cache current
> (equivalent to stating that a naive user never would _never_ need to learn
> of that cache) would you still advocate such a UI?
The use case is this:
You have checked out a fully working workspace, in order to do some
refactoring.
Emacs reads the project file and caches everything. In particular,
xref-find-definition works nicely.
Now you start to make changes. Some of the files have incorrect syntax,
or don't compile because of undefined names.
If you attempt to refresh the cache at this point, you will lose _all_
xref, because of the syntax and name errors.
On the other hand, the existing cache allows most xref commands to still
work. So you keep it for a while.
This assumes the mechanism that refreshes the cache is sensitive to such
issues. This is true for the Ada xref backend; it uses the xref info
output by the compiler, so files must compile. If the file fails to
compile for any reason, there is no xref info available.
It is also true that the user time penalty for refreshing the cache is
significant for Ada mode, and grows with the size of the project.
> In small projects the cost of a simplistic brute-force change determination
> is likely to be entirely acceptable.
Yes, that's true, especially if the cache builder handles syntax errors
gracefully, and is very fast. So that should be one option.
> In the exceedingly rare case where a user needs to keep work hidden from
> project searches he can either modify the project definition to blacklist
> temporarily the areas he wants excluded.
That's a different use case than I'm talking about, and one I've never
run across; can you elaborate?
I guess that's one way to implement "find-replace in this subset of the
project files".
> Alternatively we could provide a toggle for very advanced users to
> suppress cache refresh.
Yes, that's what I'd like.
> Bottom line: start by getting the model of state and the UI right. After
> that if the performance in corner cases needs help then give advanced users
> tools to tune the system. But please do not foist a user-maintained cache
> on casual users.
Yes, good approach.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-27 11:12 ` Stephen Leake
@ 2015-07-27 11:27 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-27 11:27 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/27/2015 02:12 PM, Stephen Leake wrote:
> Emacs reads the project file and caches everything. In particular,
> xref-find-definition works nicely.
>
> Now you start to make changes. Some of the files have incorrect syntax,
> or don't compile because of undefined names.
>
> If you attempt to refresh the cache at this point, you will lose _all_
> xref, because of the syntax and name errors.
I thought you were talking about caching the list of files, not their
contents? The former won't help you with keeping xref-find-definitions
working.
The latter is a whole different (complex) discussion, largely unrelated
to the project API. We've touched on it in the thread about hidden buffers.
> On the other hand, the existing cache allows most xref commands to still
> work. So you keep it for a while.
You may be thinking of some cached data somewhere in the implementation
of the xref backend. That's a valid approach, but neither the project
API, nor xref API should know anything about it. It's an implementation
detail.
> This assumes the mechanism that refreshes the cache is sensitive to such
> issues. This is true for the Ada xref backend; it uses the xref info
> output by the compiler, so files must compile. If the file fails to
> compile for any reason, there is no xref info available.
You can keep the xref buffer with the results of the last search, for a
while, but as you continue editing files, those entries will also begin
to point at wrong locations.
The exact point at which they stop working will depend on the specific
implementation of xref-location-marker, though. xref-etags-location is
designed to be quite resilient, for instance.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-26 18:57 ` Stephen Leake
2015-07-26 23:56 ` John Yates
@ 2015-07-27 13:00 ` Dmitry Gutov
2015-07-27 13:02 ` Dmitry Gutov
2015-07-28 1:21 ` Stephen Leake
1 sibling, 2 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-27 13:00 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/26/2015 09:57 PM, Stephen Leake wrote:
> Then it's not "global", is it? It's "operate on this set of projects,
> exclude that set". So you need a way to specify sets of projects.
Maybe the "global" is not the right word, but I'm quite confident that
if I ask an average user what a "search across project" action should
do, it would replace in the current project, (maybe) in the included
ones, and definitely not in the installed libraries. Even if they're
writable.
Even if we add a "choose which projects to touch" interface, it's good
to have a reasonable default.
>> But according to you, the "search path" will be just the directories
>> inside the project's tree, right?
>
> No, I have been explicitly saying that is not the case; the search path
> is whatever the project file says it is.
Sorry, I've replied to that part before reading the rest, and then
forgot about it.
> A list of project names. Depending on the project manager, that implies
> a list of project files in one syntax or another, each of which
> specifies a set of project directories (explicitly or implicitly).
Could you give an example?
Since they're referred only by names, this sounds like a list of
dependencies on the installed packages. Those we usually don't want to
edit, and I'd classify them as system includes.
> Trust what the project file says. The user told Emacs to use the project
> file; it should not try to be smarter.
So, xref-find-regexp won't call project-root. Got it.
> If the user wants documentation in the project, then it is in the
> project file. If the project manager tool is brain-dead and can't handle
> documentation, the the user can switch tools.
In that case, will the user add the documentation directory to the
search-path section of the project file?
> No. We should have project-root for tools that need a root directory.
> xref-find-regexp needs a search path, not a root directory.
It could include the root directory in the search path, if it's not
already there. But it seems you don't want to do that. Okay.
> xref-find-regexp searches project-search-path. project-search-path is
> set by reading the project files. The project files are provided by the
> user. Voila, xref-find-regexp does _exactly_ what the user wants, no
> more, no less.
Very well.
To sum up, I see a strong case for project-search-path as you described
it (maybe under a different name; like project-directories, as it is
now), and so far, no real case for project-root.
Further, the system search path should be separate, because it's usually
specified like that in project files already, in my experience.
And if someone were to create, say, an ECB panel showing the current
project layout, they might appreciate having the information about these
directories being different.
If you (or someone else) don't like using "project search path" for the
system libraries path, feel free to suggest another name.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-27 13:00 ` Dmitry Gutov
@ 2015-07-27 13:02 ` Dmitry Gutov
2015-07-28 1:21 ` Stephen Leake
1 sibling, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-27 13:02 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/27/2015 04:00 PM, Dmitry Gutov wrote:
> Maybe "global" is not the right word, but I'm quite confident that
> if I ask an average user what a "search across project" action should
^ replace
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-27 13:00 ` Dmitry Gutov
2015-07-27 13:02 ` Dmitry Gutov
@ 2015-07-28 1:21 ` Stephen Leake
2015-07-28 11:05 ` Stephen Leake
2015-07-28 14:18 ` Dmitry Gutov
1 sibling, 2 replies; 87+ messages in thread
From: Stephen Leake @ 2015-07-28 1:21 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/26/2015 09:57 PM, Stephen Leake wrote:
>
>> A list of project names. Depending on the project manager, that implies
>> a list of project files in one syntax or another, each of which
>> specifies a set of project directories (explicitly or implicitly).
>
> Could you give an example?
Ada, gradle.
> Since they're referred only by names, this sounds like a list of
> dependencies on the installed packages. Those we usually don't want to
> edit, and I'd classify them as system includes.
I don't know who this "we" is, but I usually structure a large project
as a main with several lower level libraries, all of which I maintain,
and I edit them all together to implement new functionality in main. So
that has to be a choice.
>> Trust what the project file says. The user told Emacs to use the project
>> file; it should not try to be smarter.
>
> So, xref-find-regexp won't call project-root. Got it.
Actually, 'xref-find-regexp' should be named 'project-find-regexp' (or
prj-find-regexp). Keep 'xref' for strictly cross-reference stuff;
'project' is for more general project stuff.
Either that, or drop 'project' and just call it all 'xref'.
>> If the user wants documentation in the project, then it is in the
>> project file. If the project manager tool is brain-dead and can't handle
>> documentation, the the user can switch tools.
>
> In that case, will the user add the documentation directory to the
> search-path section of the project file?
Yes, except most backend project files don't have things called
"search-path", since they are not for editors; it would be
"source-path".
> To sum up, I see a strong case for project-search-path as you
> described it (maybe under a different name; like project-directories,
> as it is now), and so far, no real case for project-root.
Ok, we are converging.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 1:21 ` Stephen Leake
@ 2015-07-28 11:05 ` Stephen Leake
2015-07-28 14:33 ` Dmitry Gutov
2015-07-28 14:18 ` Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-07-28 11:05 UTC (permalink / raw)
To: emacs-devel
Stephen Leake <stephen_leake@stephe-leake.org> writes:
> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> On 07/26/2015 09:57 PM, Stephen Leake wrote:
>>
>>> A list of project names. Depending on the project manager, that implies
>>> a list of project files in one syntax or another, each of which
>>> specifies a set of project directories (explicitly or implicitly).
>>
>> Could you give an example?
>
> Ada, gradle.
>
>> Since they're referred only by names, this sounds like a list of
>> dependencies on the installed packages. Those we usually don't want to
>> edit, and I'd classify them as system includes.
>
> I don't know who this "we" is, but I usually structure a large project
> as a main with several lower level libraries, all of which I maintain,
> and I edit them all together to implement new functionality in main. So
> that has to be a choice.
>
>>> Trust what the project file says. The user told Emacs to use the project
>>> file; it should not try to be smarter.
>>
>> So, xref-find-regexp won't call project-root. Got it.
>
> Actually, 'xref-find-regexp' should be named 'project-find-regexp' (or
> prj-find-regexp). Keep 'xref' for strictly cross-reference stuff;
> 'project' is for more general project stuff.
>
> Either that, or drop 'project' and just call it all 'xref'.
>
>>> If the user wants documentation in the project, then it is in the
>>> project file. If the project manager tool is brain-dead and can't handle
>>> documentation, the the user can switch tools.
>>
>> In that case, will the user add the documentation directory to the
>> search-path section of the project file?
>
> Yes, except most backend project files don't have things called
> "search-path", since they are not for editors; it would be
> "source-path".
>
>> To sum up, I see a strong case for project-search-path as you
>> described it (maybe under a different name; like project-directories,
>> as it is now), and so far, no real case for project-root.
>
> Ok, we are converging.
I just realized project.el is in emacs master; I thought it was only in
the experimental branch.
So some specific change proposals:
- Rename 'project-directories' to 'project-root-directories' or
'project-roots'. The current project root should always be first in
the list.
- 'project-search-path' should not include 'project-root-directories'.
- 'elisp-search-path' should not include 'package-user-dir';
package-user-dir is not an elisp source directory, and load-path
already includes the subdirs of it, if packages are initialized.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 1:21 ` Stephen Leake
2015-07-28 11:05 ` Stephen Leake
@ 2015-07-28 14:18 ` Dmitry Gutov
2015-07-28 16:15 ` Stephen Leake
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-28 14:18 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/28/2015 04:21 AM, Stephen Leake wrote:
>> Could you give an example?
>
> Ada, gradle.
I meant an example (or several) of those declarations from an actual
project file.
Here an excerpt from an imaginary Gemfile:
gem 'foo', '~> 1.2.3'
gem 'bar', path: '../bar'
Having the latter kind of declaration might not be considered the best
idea, but it's often nice to be able to do that. Anyway, I'm sure there
are corresponding configurations in other ecosystems that are more
idiomatic.
> I don't know who this "we" is, but I usually structure a large project
> as a main with several lower level libraries, all of which I maintain,
> and I edit them all together to implement new functionality in main. So
> that has to be a choice.
IME, aside from a few libraries one (person/team/company) maintains
themselves, there's also a long list of external dependencies.
In the example above, 'bar' will be one of project-directory-roots, and
'foo' will only be in project-search-path.
> Actually, 'xref-find-regexp' should be named 'project-find-regexp' (or
> prj-find-regexp). Keep 'xref' for strictly cross-reference stuff;
Sounds good to me. I'll make that change as soon as xref has a suitable
public interface for that.
> 'project' is for more general project stuff.
It's also used in the xref-find-references implementations in elisp-mode
and etags, which I'm inclined to converge into one default
implementation that uses the info from the project package. I think
that's fine.
> Either that, or drop 'project' and just call it all 'xref'.
project is foremost intended to be used for third-party code. It's a
feature that's been requested for a while.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 11:05 ` Stephen Leake
@ 2015-07-28 14:33 ` Dmitry Gutov
2015-07-28 15:45 ` Stephen Leake
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-28 14:33 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/28/2015 02:05 PM, Stephen Leake wrote:
> I just realized project.el is in emacs master; I thought it was only in
> the experimental branch.
It would be cleaner to have it separate, but being able to dogfood it
now is quite handy.
> So some specific change proposals:
>
> - Rename 'project-directories' to 'project-root-directories' or
> 'project-roots'.
Either is fine with me, but note that y having "root" in the name we
give up on adding a special "project-root" function in the future, one
that you described earlier.
> The current project root should always be first in the list.
> - 'project-search-path' should not include 'project-root-directories'.
I say we document it like that (or say that it's "allowed not to
include"), but the consumers will still take care of the duplication
(call project--prune-directories). That's easier on the implementations.
> - 'elisp-search-path' should not include 'package-user-dir';
> package-user-dir is not an elisp source directory, and load-path
> already includes the subdirs of it, if packages are initialized.
What's the harm? It's an optimization: this way, we only call 'find'
once per elpa dir, instead of doing it for each installed package. Not
sure what's the overhead on that, but it seems wasteful.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 14:33 ` Dmitry Gutov
@ 2015-07-28 15:45 ` Stephen Leake
2015-07-28 16:25 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-07-28 15:45 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/28/2015 02:05 PM, Stephen Leake wrote:
>
>> - Rename 'project-directories' to 'project-root-directories' or
>> 'project-roots'.
>
> Either is fine with me, but note that y having "root" in the name we
> give up on adding a special "project-root" function in the future, one
> that you described earlier.
No, I'm happy with this:
'project-root'; find the root directory of the current project.
'project-roots'; return a list of related project roots.
That's what renaming 'project-directories' to 'project-roots' gives us.
Although some will object to the small name difference between
'project-root' and 'project-roots'.
>> - 'project-search-path' should not include 'project-root-directories'.
>
> I say we document it like that (or say that it's "allowed not to
> include"), but the consumers will still take care of the duplication
> (call project--prune-directories). That's easier on the
> implementations.
Are you agreeing that the default implementation should not include
project-root-directories?
The point is that if you only customize by setting
project-search-path-function, you can't exclude project-directories.
And I thought we agreed that project-search-path is disjoint from
project-directories.
>> - 'elisp-search-path' should not include 'package-user-dir';
>> package-user-dir is not an elisp source directory, and load-path
>> already includes the subdirs of it, if packages are initialized.
>
> What's the harm? It's an optimization: this way, we only call 'find'
> once per elpa dir, instead of doing it for each installed package. Not
> sure what's the overhead on that, but it seems wasteful.
As with all such optimizations, it can only be justified by actual
measurement.
But I missed the fact that the default project-search-dirs uses
'project--prune-directories' to throw away the subdirs; so while
'elisp-search-dirs' contains redundant directories,
'project-search-dirs' does not.
I find it confusing mixing two customization mechanisms; I started
another thread on that.
It would be less confusing if the default definition of
project-search-path did _not_ call project--prune-directories, but
elisp-search-path _did_. Then project--prune-directories should be
renamed without the '--'.
The doc string for project-search-path says "source directories", not
"source root directories". That needs to be fixed.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 14:18 ` Dmitry Gutov
@ 2015-07-28 16:15 ` Stephen Leake
2015-07-28 18:44 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-07-28 16:15 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/28/2015 04:21 AM, Stephen Leake wrote:
>
>>> Could you give an example?
>>
>> Ada, gradle.
> I meant an example (or several) of those declarations from an actual
> project file.
From an Ada project file:
aggregate project Books_Agg is
for Project_Path use
("../../../org.stephe_leake.sal/build/release",
"../../../org.stephe_leake.gtk/build/release",
"../../../org.stephe_leake.makerules");
for Project_Files use ("books.gpr");
end Books_Agg;
with "gtkada";
with "gnatcoll_sqlite";
with "sal";
with "sal_gtk";
with "standard_common";
project Books is
for Source_Dirs use ("../../source", "../../test");
...
end Books;
Project_Path says where to find projects (there is also an implicit
system location); 'with "...";' says what projects to include. In this
case, "gtkada" and "gnatcoll_sqlite" are in the system location, and are
read only; sal, sal_gtk and standard_common read/write. But the Ada
project tool (as provided by the compiler vendor) makes no distinction
between "system" and "user" libraries; more precisely, it relies on the
file system read/write permissions to do that.
From an Android gradle file:
dependencies {
compile project(':licensesdialoglibrary')
compile project(':quickScroll')
compile project(':velocityviewpagerlibrary')
compile project(':dragsortlistviewlibrary')
compile project(':viewpagerindicatorlibrary')
compile project(':circularImageView')
compile project(':picasso')
compile files('libs/android-async-http-1.4.2-66-g4b6eb97.jar')
compile files('libs/commons-io-2.4.jar')
compile files('libs/commons-lang3-3.1.jar')
compile files('libs/commons-logging.jar')
compile files('libs/dashclock-api-r2.0.jar')
compile files('libs/google-http-client-1.16.0-rc.jar')
compile files('libs/google-http-client-android-1.16.0-rc.jar')
compile files('libs/jaudiotagger-2.0.4-20111207.115108-15.jar')
compile files('libs/libGoogleAnalyticsServices.jar')
compile files('libs/universal-image-loader-1.9.3-with-sources.jar')
compile files('libs/com.haarman.listviewanimations-2.6.0.jar')
compile files('libs/nineoldandroids-2.4.0.jar')
compile files('libs/renderscript-v8.jar')
compile 'com.android.support:support-v4:+'
compile 'com.google.android.gms:play-services:+'
}
Some of these are read/write, some are read-only; gradle does not have a
mechanism to indicate that.
> Here an excerpt from an imaginary Gemfile:
>
> gem 'foo', '~> 1.2.3'
> gem 'bar', path: '../bar'
>
> Having the latter kind of declaration might not be considered the best
> idea, but it's often nice to be able to do that. Anyway, I'm sure
> there are corresponding configurations in other ecosystems that are
> more idiomatic.
The point is that the abstract project API must handle all of these. In
the case of Ada, there is a command-line tool that parses the project
file and provides the info that Emacs needs.
>> I don't know who this "we" is, but I usually structure a large project
>> as a main with several lower level libraries, all of which I maintain,
>> and I edit them all together to implement new functionality in main. So
>> that has to be a choice.
>
> IME, aside from a few libraries one (person/team/company) maintains
> themselves, there's also a long list of external dependencies.
Yes, we are agreeing; some dependencies are read/write, some are read-only.
How many of each, and whether they are "system" or not, are minor
issues.
> In the example above, 'bar' will be one of project-directory-roots,
> and 'foo' will only be in project-search-path.
Why? What effect does that have on what you can do with 'foo' vs 'bar'?
How can you tell that just from the gem file?
I'm guessing you are saying "foo is maintained by the same team as the
main project (ie read/write), bar is maintained elsewhere (ie
read-only)".
So then files in project-directories are read-only, files in
project-search-path are read/write.
But the current default implementation of project-search-path _inludes_
project-directories, so this is inconsistent.
And we've said that project-directories includes the main project root,
so that's also inconsistent.
So I don't understand what you are doing here.
>> Actually, 'xref-find-regexp' should be named 'project-find-regexp' (or
>> prj-find-regexp). Keep 'xref' for strictly cross-reference stuff;
>
> Sounds good to me. I'll make that change as soon as xref has a
> suitable public interface for that.
I don't understand; this requires deleting things from xref, not adding
to it.
The current implementation of xref-find-regexp uses project-current,
xref-collect-matches, and xref--show-xrefs.
If you make the implementation of project-find-regexp use xref
facilities, then I understand your comment. I was hoping for more
separation.
I suggest renaming 'xref--show-xrefs' to 'project-show-locations', and
have it take a list of locations, not implicitly call
xref-find-function. Also move all of the location stuff to project; it's
more general than xref.
xref-collect-matches uses grep; it can simply be moved to
project-collect-matches, along with the xref-*grep functions it uses.
The doc string needs to mention subdirs.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 15:45 ` Stephen Leake
@ 2015-07-28 16:25 ` Dmitry Gutov
2015-07-29 1:36 ` Stephen Leake
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-28 16:25 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/28/2015 06:45 PM, Stephen Leake wrote:
> 'project-roots'; return a list of related project roots.
>
> That's what renaming 'project-directories' to 'project-roots' gives us.
> Although some will object to the small name difference between
> 'project-root' and 'project-roots'.
If project-roots does not include project-root (so we limit the notion
of "current project" to just one directory tree), it would be better to
rename that to project-related-projects, and return project instances in
there.
This way, the consumer can inquire about each related project's
dependencies, as well as call project-ignores on each of them.
But that becomes more complex, because then you'd expect the consumer to
call project-related-projects on each related project ask well, and
traverse the whole tree, handle loops, etc.
I think it'll be better to incorporate all related projects into the
notion of the current project, return all project roots on equal rights,
give project-ignores an argument (one of the roots), and introduce
project-relativize which, like described previously, will return (ROOT .
RELATIVE-PATH).
In this scheme, project-root is not needed, because we want to treat all
roots somewhat equally, and also because project-root shouldn't use
default-directory, or buffer-file-name.
> Are you agreeing that the default implementation should not include
> project-root-directories?
Yes.
> The point is that if you only customize by setting
> project-search-path-function, you can't exclude project-directories.
>
> And I thought we agreed that project-search-path is disjoint from
> project-directories.
Like I said, "allowed not to include". Not matter if emacs-lisp-mode
sets project-search-path-function, or provides a specialized
implementation of project-search-path, it doesn't know the current
project roots, so it can't exclude them.
> As with all such optimizations, it can only be justified by actual
> measurement.
It's an easy one, though.
> It would be less confusing if the default definition of
> project-search-path did _not_ call project--prune-directories, but
> elisp-search-path _did_.
Some caller up the stack will have to call project--prune-directories
anyway, because we don't want to guarantee that search-path and
project-roots don't mix (see above). However, we could provide a public
function in project.el that will combine both. I having hard time coming
up with the name, though: project-full-search-path doesn't sound too good.
> Then project--prune-directories should be> renamed without the '--'.
I've been waiting for a better name: there are several
-prune-directories function in Emacs already, in different packages, all
with different semantics.
> The doc string for project-search-path says "source directories", not
> "source root directories". That needs to be fixed.
All right. To me, the two sound equivalent, and I meant it like that.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 16:15 ` Stephen Leake
@ 2015-07-28 18:44 ` Dmitry Gutov
2015-07-29 2:27 ` Stephen Leake
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-28 18:44 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/28/2015 07:15 PM, Stephen Leake wrote:
> From an Ada project file:
Thank you.
> Project_Path says where to find projects (there is also an implicit
> system location); 'with "...";' says what projects to include. In this
> case, "gtkada" and "gnatcoll_sqlite" are in the system location, and are
> read only; sal, sal_gtk and standard_common read/write. But the Ada
> project tool (as provided by the compiler vendor) makes no distinction
> between "system" and "user" libraries; more precisely, it relies on the
> file system read/write permissions to do that.
That's unfortunate. But you do specify "custom" locations for sal,
sal_gtk and standard_common in the beginning, right? Then a smart enough
tool could learn that information.
In the meantime, I guess, the Ada projects will have to classify all
dependencies together as search-path.
>
> From an Android gradle file:
>
> dependencies {
>
> compile project(':licensesdialoglibrary')
> compile files('libs/android-async-http-1.4.2-66-g4b6eb97.jar')
> compile 'com.google.android.gms:play-services:+'
> }
>
> Some of these are read/write, some are read-only; gradle does not have a
> mechanism to indicate that.
jars are obviously build artefacts. Doesn't the difference between
compile '...' and compile project('...') signify something, though?
>> Here an excerpt from an imaginary Gemfile:
>>
>> gem 'foo', '~> 1.2.3'
>> gem 'bar', path: '../bar'
>>
> The point is that the abstract project API must handle all of these. In
> the case of Ada, there is a command-line tool that parses the project
> file and provides the info that Emacs needs.
I'm assuming that the build tools in other ecosystems can likewise
separate dependencies into search-path and full-on projects. Or, at
least, that most of them can, and this separation is meaningful.
And that there's no separation along different lines that's more useful
than that.
> Yes, we are agreeing; some dependencies are read/write, some are read-only.
>
> How many of each, and whether they are "system" or not, are minor
> issues.
Actually, neither "system", nor "read/write" terminology strikes me as
optimal. "search path" vs. "projects" sounds better: one could edit a
file inside the system search path, with purpose, and we're should allow
that operation, but not optimize the interface for it.
>> In the example above, 'bar' will be one of project-directory-roots,
>> and 'foo' will only be in project-search-path.
>
> Why? What effect does that have on what you can do with 'foo' vs 'bar'?
>
> How can you tell that just from the gem file?
path: '/..' says that 'bar' is a project residing in a nearby directory,
and it's probably version-controlled. The fact that I have a Git
checkout of 'bar' is a strong indicator that I'm maintaining that
project (though not a guarantee, of course).
It makes sense to perform read/write operations on version-controlled
directories.
'foo', on the other hand, resides in
~/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/foo-1.2.3/, and any
edits I make to it are hard to track or push to the development repo.
> I'm guessing you are saying "foo is maintained by the same team as the
> main project (ie read/write), bar is maintained elsewhere (ie
> read-only)".
The opposite of that.
> So then files in project-directories are read-only, files in
> project-search-path are read/write.
Ditto.
> But the current default implementation of project-search-path _inludes_
> project-directories, so this is inconsistent.
Is allowed to include.
> And we've said that project-directories includes the main project root,
> so that's also inconsistent.
Why?
> So I don't understand what you are doing here.
read/write operations will use project-roots; read-only operations will
use project-roots combined with project-search-path. I though we've
already settled on that.
> I don't understand; this requires deleting things from xref, not adding
> to it.
xref-find-regexp uses things from xref, such as the output buffer, its
major mode, etc. They're not exposed publicly in a proper fashion, yet.
> The current implementation of xref-find-regexp uses project-current,
> xref-collect-matches, and xref--show-xrefs.
Right. So it uses both xref.el and project.el.
> If you make the implementation of project-find-regexp use xref
> facilities, then I understand your comment.
Exactly.
> I was hoping for more separation.
What kind?
> I suggest renaming 'xref--show-xrefs' to 'project-show-locations', and
Why project-? The procedure of displaying the locations doesn't have
much to do with the project API. The xref API, however, is very much
related.
> have it take a list of locations, not implicitly call
> xref-find-function.
The difficulty with that had been the necessity to track the buffers
that xref-find-function creates, in order to be able to delete each of
them that haven't been used otherwise, in xref-quit.
The parallel thread about hidden/temporary buffers should result in
solving that in a different way, and then I'll do just that.
> xref-collect-matches uses grep; it can simply be moved to
> project-collect-matches, along with the xref-*grep functions it uses.
> The doc string needs to mention subdirs.
I think there's value in providing function, in xref, that returns a
list of matches for a given regexp in a given directory, as xref-items.
It's not, by itself, project-related, and other xref backends might use
it for cheap xref-find-references implementations.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 16:25 ` Dmitry Gutov
@ 2015-07-29 1:36 ` Stephen Leake
2015-07-29 2:10 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-07-29 1:36 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/28/2015 06:45 PM, Stephen Leake wrote:
>
> I think it'll be better to incorporate all related projects into the
> notion of the current project, return all project roots on equal
> rights,
That is certainly what Ada mode does, and gradle.
> give project-ignores an argument (one of the roots), and
> introduce project-relativize which, like described previously, will
> return (ROOT . RELATIVE-PATH).
>
> In this scheme, project-root is not needed, because we want to treat
> all roots somewhat equally, and also because project-root shouldn't
> use default-directory, or buffer-file-name.
So you would keep project-roots, but not project-root. I can live with
that (Ada mode needs neither :).
> Not matter if emacs-lisp-mode
> sets project-search-path-function, or provides a specialized
> implementation of project-search-path, it doesn't know the current
> project roots, so it can't exclude them.
How will the elisp backend define project-roots?
Since it doesn't know anything about project-roots, it should either
throw an error, or return nil.
On the other hand, (project-search-path (current-project)) in a .el
buffer on my machine returns:
("c:/Projects/emacs/master-build-mingw64/lisp/"
"c:/Projects/emacs/master/"
"c:/Projects/emacs_stephe.main/emacs_stephe/"
"c:/Projects/emacs_stephe.main/emacs_stephe_site_lisp/"
"c:/Projects/org.emacs.ada-mode/"
"c:/Projects/org.emacs.dvc/lisp/"
"c:/home/stephe/.emacs.d/elpa/")
Those are all 'project-roots' as far as I can tell; they each represent
a logically and physically separate collection of elisp source files
(except that master-build-mingw64/lisp/ actually has no elisp source files).
So for elisp, it seems we are definining project-search-path to be
project-roots.
Which goes a long way to explaining why this has been so confusing.
On the other hand, "(project-directories (project-current))" returns the
single root that contains the current .el file; the same as
"(project-root (project-current))". Why does project-directories not
return a list?
Currently, the only code that uses project-directories is
xref-find-regexp, and it concats it with project-search-path, so it
confuses the two.
Ada mode has no need of project-roots. It seems elisp also has no need
of project-roots. JDE (Java Development Environment) has no need of
project-roots.
>> It would be less confusing if the default definition of
>> project-search-path did _not_ call project--prune-directories, but
>> elisp-search-path _did_.
>
> Some caller up the stack will have to call project--prune-directories
> anyway, because we don't want to guarantee that search-path and
> project-roots don't mix (see above).
The top-level generic API should be agnostic about that; only the
backend knows precisely what project-search-path and project-roots are.
In elisp and Ada, project-roots is simply undefined.
Ada mode wants search-path to be flat, _not_ containing roots. So it
does _not_ what something to call project--prune-directories.
> However, we could provide a
> public function in project.el that will combine both.
Why?
I still have not seen an actual use case for project-roots; it seems the
simplest thing is to simply drop it.
>> The doc string for project-search-path says "source directories", not
>> "source root directories". That needs to be fixed.
>
> All right. To me, the two sound equivalent, and I meant it like that.
The difference is whether recursion into subdirectories is required (in
this case, allowed); that is always worth documenting; it is normally an
option in search functions (think grep vs grep-find;
semantic-symref-find-references-by-name has a :scope argument).
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-29 1:36 ` Stephen Leake
@ 2015-07-29 2:10 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-29 2:10 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/29/2015 04:36 AM, Stephen Leake wrote:
> So you would keep project-roots, but not project-root. I can live with
> that (Ada mode needs neither :).
It's fine if some projects define project-roots to nil. Unfortunately,
as I imagine it, xref-query-replace won't work for them.
Of course, we could add a preference (turned off by default) that
xref-query-replace walks the matches in the search-path, too, not just
in project-roots.
> How will the elisp backend define project-roots?
It won't. Creating a project backend for Elisp seems pretty much
impossible (or rather useless), from where I'm standing.
That's why emacs-lisp-mode sets project-search-path-function instead:
the VC project backend uses it automatically.
> Since it doesn't know anything about project-roots, it should either
> throw an error, or return nil.
But that's exactly the problem: if I'm working on a third-party Elisp
project, it has its own repository, there's no guarantee that its root
directory is added to load-path. If all Elisp code is in the lisp
subdirectory (and it's in load-path), if I call xref-find-regexp, it
would miss all files on the level above.
> On the other hand, (project-search-path (current-project)) in a .el
> buffer on my machine returns:
>
> ("c:/Projects/emacs/master-build-mingw64/lisp/"
> "c:/Projects/emacs/master/"
> "c:/Projects/emacs_stephe.main/emacs_stephe/"
> "c:/Projects/emacs_stephe.main/emacs_stephe_site_lisp/"
> "c:/Projects/org.emacs.ada-mode/"
> "c:/Projects/org.emacs.dvc/lisp/"
> "c:/home/stephe/.emacs.d/elpa/")
>
> Those are all 'project-roots' as far as I can tell; they each represent
> a logically and physically separate collection of elisp source files
> (except that master-build-mingw64/lisp/ actually has no elisp source files).
No, these are just search-path. We don't know where the roots of most of
these "projects" are.
> On the other hand, "(project-directories (project-current))" returns the
> single root that contains the current .el file; the same as
> "(project-root (project-current))". Why does project-directories not
> return a list?
It returns a one-element list.
> Currently, the only code that uses project-directories is
> xref-find-regexp, and it concats it with project-search-path, so it
> confuses the two.
Right. Sorry, that must indeed be confusing. xref-query-replace does not
discriminate against project-search-path yet.
> Ada mode has no need of project-roots. It seems elisp also has no need
> of project-roots. JDE (Java Development Environment) has no need of
> project-roots.
A common Java project usually has one root: the directory containing its
project file (build.xml, pom.xml, or so on).
Even so the project implementation could provide a way to open several
projects together, if explicitly told so by the user. Then it would
return them all in project-roots.
>> Some caller up the stack will have to call project--prune-directories
>> anyway, because we don't want to guarantee that search-path and
>> project-roots don't mix (see above).
>
> The top-level generic API should be agnostic about that; only the
> backend knows precisely what project-search-path and project-roots are.
> In elisp and Ada, project-roots is simply undefined.
So?
The caller would do what xref-find-regexp does now:
(project--prune-directories
(nconc
(project-roots proj)
(project-search-path proj)))
> Ada mode wants search-path to be flat, _not_ containing roots. So it
> does _not_ what something to call project--prune-directories.
It wouldn't hurt anyway.
>> However, we could provide a
>> public function in project.el that will combine both.
>
> Why?
So that the caller doesn't need to call project--prune-directories itself.
> I still have not seen an actual use case for project-roots; it seems the
> simplest thing is to simply drop it.
I have provided a few, along the way of this discussion.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-28 18:44 ` Dmitry Gutov
@ 2015-07-29 2:27 ` Stephen Leake
2015-07-29 22:51 ` Dmitry Gutov
2015-07-29 23:11 ` xref display and multiple locations, " Dmitry Gutov
0 siblings, 2 replies; 87+ messages in thread
From: Stephen Leake @ 2015-07-29 2:27 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/28/2015 07:15 PM, Stephen Leake wrote:
>
>> From an Ada project file:
>
> Thank you.
>
>> Project_Path says where to find projects (there is also an implicit
>> system location); 'with "...";' says what projects to include. In this
>> case, "gtkada" and "gnatcoll_sqlite" are in the system location, and are
>> read only; sal, sal_gtk and standard_common read/write. But the Ada
>> project tool (as provided by the compiler vendor) makes no distinction
>> between "system" and "user" libraries; more precisely, it relies on the
>> file system read/write permissions to do that.
>
> That's unfortunate.
Why? it is what it is; it works.
> But you do specify "custom" locations for sal,
> sal_gtk and standard_common in the beginning, right? Then a smart
> enough tool could learn that information.
I don't know what you mean by "learn"; the user writes the project file,
the tools read it.
Emacs uses one tool to read the project file, and extract a flat
project-search-path (the concatenation of all the Source_Path
attributes).
> In the meantime, I guess, the Ada projects will have to classify all
> dependencies together as search-path.
Yes, that's what I've been saying; it's a valid use case.
That is also what elisp is doing now.
>> From an Android gradle file:
>>
>> dependencies {
>>
>> compile project(':licensesdialoglibrary')
>> compile files('libs/android-async-http-1.4.2-66-g4b6eb97.jar')
>> compile 'com.google.android.gms:play-services:+'
>> }
>>
>> Some of these are read/write, some are read-only; gradle does not have a
>> mechanism to indicate that.
>
> jars are obviously build artefacts. Doesn't the difference between
> compile '...' and compile project('...') signify something, though?
Yes, but I'm not sure what. I haven't used this enough yet.
>>> Here an excerpt from an imaginary Gemfile:
>>>
>>> gem 'foo', '~> 1.2.3'
>>> gem 'bar', path: '../bar'
>>>
>> The point is that the abstract project API must handle all of these. In
>> the case of Ada, there is a command-line tool that parses the project
>> file and provides the info that Emacs needs.
>
> I'm assuming that the build tools in other ecosystems can likewise
> separate dependencies into search-path and full-on projects. Or, at
> least, that most of them can, and this separation is meaningful.
I know of no tool that does that; I do _not_ make that assumption. Where
did you get the idea it was meaningful?
>>> In the example above, 'bar' will be one of project-directory-roots,
>>> and 'foo' will only be in project-search-path.
>>
>> Why? What effect does that have on what you can do with 'foo' vs 'bar'?
>>
>> How can you tell that just from the gem file?
>
> path: '/..' says that 'bar' is a project residing in a nearby
> directory,
Ok
> and it's probably version-controlled. The fact that I have
> a Git checkout of 'bar' is a strong indicator that I'm maintaining
> that project (though not a guarantee, of course).
That's extra information, _not_ in the gem file.
> 'foo', on the other hand, resides in
> ~/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/foo-1.2.3/,
I guess that's what ">.1.2.3" means? It's some ruby-defined library
directory in your home directory. Similar to a "system library"?
> and any edits I make to it are hard to track or push to the
> development repo.
Ok, since it's like a system library.
>> I'm guessing you are saying "foo is maintained by the same team as the
>> main project (ie read/write), bar is maintained elsewhere (ie
>> read-only)".
>
> The opposite of that.
Ok.
>> So then files in project-directories are read-only, files in
>> project-search-path are read/write.
>
> Ditto.
Which is _not_ what the current code does; xref-query-replace (an edit
function) operates on the result of xref-find-regexp (among other
things), which uses (nconc project-search-path project-directories).
>> But the current default implementation of project-search-path _inludes_
>> project-directories, so this is inconsistent.
>
> Is allowed to include.
I'm not talking about the doc string; I'm talking about the actual code,
which is what elisp uses. But you've agreed to change that.
> read/write operations will use project-roots; read-only operations
> will use project-roots combined with project-search-path. I though
> we've already settled on that.
News to me.
If that is the intended use of these two functions, then they should be
named project-path-read-write and project-path-read-only; that will be
_much_ clearer. It can still be up to the backend to decide whether they
are flat or require additional recursion.
I've never found the need for that distinction in Ada mode.
>> I suggest renaming 'xref--show-xrefs' to 'project-show-locations', and
>
> Why project-? The procedure of displaying the locations doesn't have
> much to do with the project API. The xref API, however, is very much
> related.
Locations are tied other things besides cross-reference; cross-reference
is just one way of producing a list of locations.
find-regexp is another way.
compilation errors is another way.
All of those could use a common location API. 'project' is one place to
put it; it could be in its own 'location' namespace.
Currently, 'grep-find' and 'compilation' share the same location API.
Which is why I suggested that xref also share it. But I do like the
current xref location display, so maybe there is a case for them to
remain distinct.
They serve different purposes; the xref display makes it easy to choose
one location from a small list; the compilation display makes it easier
to step through all of them in sequence, and it works even for very long
lists.
There should at least there should be this choice of display; the
location data structure could probably be shared (the compilation
location structure contains more info).
project-find-regexp should use the same location display API as grep-find,
since they serve very similar purposes; the user will want to step thru
all of them. Or maybe not, in some particular use case.
(grep-find could eventually be made obsolete in favor of
project-find-regexp).
>> have it take a list of locations, not implicitly call
>> xref-find-function.
>
> The difficulty with that had been the necessity to track the buffers
> that xref-find-function creates, in order to be able to delete each of
> them that haven't been used otherwise, in xref-quit.
>
> The parallel thread about hidden/temporary buffers should result in
> solving that in a different way, and then I'll do just that.
Good.
>> xref-collect-matches uses grep; it can simply be moved to
>> project-collect-matches, along with the xref-*grep functions it uses.
>> The doc string needs to mention subdirs.
> I think there's value in providing function, in xref, that returns a
> list of matches for a given regexp in a given directory, as
> xref-items.
That's just reimplementing Emacs grep to use the alternate location API; not
worth it.
> It's not, by itself, project-related,
Nor is it xref-related.
> and other xref backends might use it for cheap xref-find-references
> implementations.
After adding path and/or directory recursion support, for searching
project-search-path.
I hope not; better to say "xref not implemented; use Emacs grep". That will
motivate people to implement a real xref backend, as well as immediately
explaining why the xref is not accurate.
It would be nice if <menu-bar> <tools> <grep> defaulted to the
identifier at point.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-29 2:27 ` Stephen Leake
@ 2015-07-29 22:51 ` Dmitry Gutov
2015-07-30 8:17 ` Stephen Leake
2015-07-29 23:11 ` xref display and multiple locations, " Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-29 22:51 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/29/2015 05:27 AM, Stephen Leake wrote:
> Why? it is what it is; it works.
It gives us less information than it could, about the project configuration.
>> But you do specify "custom" locations for sal,
>> sal_gtk and standard_common in the beginning, right? Then a smart
>> enough tool could learn that information.
>
> I don't know what you mean by "learn"; the user writes the project file,
> the tools read it.
Figure out somehow, based on the project file and the contents of the
file system, hopefully without asking the user more questions.
For instance, even if the project file gives no direct indication of
which search-path elements are "proper" projects, some Elisp code could
look inside each of those directories (and one level above, maybe) and
see if there are project files in any of them. And those where there
are, promote to project-roots as well.
That might be harder for Ada, but should be totally doable for
Java-based projects (if build.gradle doesn't provide that info).
> Yes, that's what I've been saying; it's a valid use case.
>
> That is also what elisp is doing now.
Elisp is also relying on runtime information. It's not reliable enough,
IMO, to use as the basis of xref-query-replace, for instance. Like I
mentioned in the other email, the project I've opened right now isn't
necessarily in load-path.
AdaCore and Gradle are in better positions than that.
>> jars are obviously build artefacts. Doesn't the difference between
>> compile '...' and compile project('...') signify something, though?
>
> Yes, but I'm not sure what. I haven't used this enough yet.
That might be worth investigating.
>> I'm assuming that the build tools in other ecosystems can likewise
>> separate dependencies into search-path and full-on projects. Or, at
>> least, that most of them can, and this separation is meaningful.
>
> I know of no tool that does that; I do _not_ make that assumption. Where
> did you get the idea it was meaningful?
The IDEs usually separate the "current project" (one root with its
subtree) from the system dependencies. Especially in the Java world,
where there's just too much code to look at.
Multi-project builds also exist, but they seem to be usually constrained
to the "several projects inside one parent directory" configuration, see
"7.3. Multi-project Java build" in
https://docs.gradle.org/current/userguide/tutorial_java_projects.html.
Also, here's a screenshot of a "project drawer" in a popular IDE:
http://blog.pivotal.io/wp-content/uploads/2012/02/intellij_modules.png
It showcases a separation of the current project's "modules" from
"external libraries".
>> and it's probably version-controlled. The fact that I have
>> a Git checkout of 'bar' is a strong indicator that I'm maintaining
>> that project (though not a guarantee, of course).
>
> That's extra information, _not_ in the gem file.
It's a good assumption. Even if it's not version controlled, it's most
likely a working directory anyway, and the developer's just being
ignorant about version control.
>> 'foo', on the other hand, resides in
>> ~/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/foo-1.2.3/,
>
> I guess that's what ">.1.2.3" means? It's some ruby-defined library
> directory in your home directory. Similar to a "system library"?
Exactly like a system library.
> Which is _not_ what the current code does; xref-query-replace (an edit
> function) operates on the result of xref-find-regexp (among other
> things), which uses (nconc project-search-path project-directories).
Indeed. But that shouldn't be surprising: we are discussing a change
proposal, and xref-find-regexp, xref-query-replace, and other related
code you can see in master, had all been written before the current
iteration of this thread.
>>> But the current default implementation of project-search-path _inludes_
>>> project-directories, so this is inconsistent.
>>
>> Is allowed to include.
>
> I'm not talking about the doc string; I'm talking about the actual code,
> which is what elisp uses. But you've agreed to change that.
Ah, yes, of course. I intend to change that.
But consider this: elisp-search-path is not ideal (the current project
could be not in load-path). So if project-search-path only consults
project-search-path-function, by default, what will the default
implementation of xref-find-references use?
Just (project-search-path (project-current)), or
(project-prune-directories
(append
(project-roots proj)
(project-search-path proj)))
?
The latter moves the same logic into the caller. The former will give
the user false negatives, in the absence of a smart, language-aware
project implementation.
> If that is the intended use of these two functions, then they should be
> named project-path-read-write and project-path-read-only; that will be
> _much_ clearer. It can still be up to the backend to decide whether they
> are flat or require additional recursion.
Err, that sounds silly. Why would you want not to recurse?
The current xref-collect-matches implementation unconditionally
recurses, and it doesn't seem particularly useful to create a variant
that doesn't.
> Locations are tied other things besides cross-reference; cross-reference
> is just one way of producing a list of locations.
>
> find-regexp is another way.
>
> compilation errors is another way.
>
> All of those could use a common location API. 'project' is one place to
> put it; it could be in its own 'location' namespace.
I suppose you're right. But until we have a unified location API (which
I wouldn't count on appearing anytime soon), xref is the better place.
xref-location lives there, and it's the best purpose-agnostic location
class that we have now.
> Currently, 'grep-find' and 'compilation' share the same location API.
> Which is why I suggested that xref also share it. But I do like the
> current xref location display, so maybe there is a case for them to
> remain distinct.
But there's not much API there, is there? To obtain the list of matches
from the Grep or Compilation output, one has to re-search-forward in
that buffer.
>> I think there's value in providing function, in xref, that returns a
>> list of matches for a given regexp in a given directory, as
>> xref-items.
>
> That's just reimplementing Emacs grep to use the alternate location API; not
> worth it.
I think it's useful enough.
>> It's not, by itself, project-related,
>
> Nor is it xref-related.
It's location-related, so to speak. Which live in xref, for now.
> After adding path and/or directory recursion support, for searching
> project-search-path.
Not sure what you mean here.
> I hope not; better to say "xref not implemented; use Emacs grep". That will
> motivate people to implement a real xref backend, as well as immediately
> explaining why the xref is not accurate.
Why would using M-x grep be better? I think it's better to provide an
imperfect implementation that showcases the available features
(bindings, commands, presentation).
That will better motivate third-party developers to improve on other
aspects, such as precision. That's how most open-source projects get
their contributors, anyway: when a feature works almost, but not
exactly, right.
And one shouldn't discount Grep-based searched out of hand. For
instance, a simple xref-find-definitions implementations that searches a
set of directories for "def funcname", would be a lot better that the
absolute nothing we provide out of the box to users who don't use etags.
> It would be nice if <menu-bar> <tools> <grep> defaulted to the
> identifier at point.
M-x report-emacs-bug? That seems unrelated.
^ permalink raw reply [flat|nested] 87+ messages in thread
* xref display and multiple locations, was: Re: Unified project interface
2015-07-29 2:27 ` Stephen Leake
2015-07-29 22:51 ` Dmitry Gutov
@ 2015-07-29 23:11 ` Dmitry Gutov
1 sibling, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-29 23:11 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/29/2015 05:27 AM, Stephen Leake wrote:
> Currently, 'grep-find' and 'compilation' share the same location API.
> Which is why I suggested that xref also share it. But I do like the
> current xref location display, so maybe there is a case for them to
> remain distinct.
>
> They serve different purposes; the xref display makes it easy to choose
> one location from a small list; the compilation display makes it easier
> to step through all of them in sequence, and it works even for very long
> lists.
I've come to believe that it's a mistake. The only xref command where
we're likely to only be only in one location is xref-find-definitions,
and in ideal implementations, it's likely not to show the xref buffer at
all, jumping straight to the sole result.
So in the average case, pressing RET, or clicking on an xref element,
should display the location, but keep the xref buffer displayed as well.
We just need a way to use the current behavior in xref-find-definitions
(or implement a yet-another selection interface for it).
> There should at least there should be this choice of display; the
> location data structure could probably be shared (the compilation
> location structure contains more info).
The current customization point is xref-show-xrefs-function. There are
no different implementations yet.
One nice feature of M-x grep, by the way, is that it shows the matches
immediately as it finds them (whereas the current xref commands are
forced to wait until the output ends, and collect the whole list).
One way to reach parity there would be to teach xref to accept
generators (as implemented in generator.el), as well as regular lists.
> (grep-find could eventually be made obsolete in favor of
> project-find-regexp).
Here's hoping.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-29 22:51 ` Dmitry Gutov
@ 2015-07-30 8:17 ` Stephen Leake
2015-07-31 0:15 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-07-30 8:17 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/29/2015 05:27 AM, Stephen Leake wrote:
>
>>> But you do specify "custom" locations for sal,
>>> sal_gtk and standard_common in the beginning, right? Then a smart
>>> enough tool could learn that information.
>>
>> I don't know what you mean by "learn"; the user writes the project file,
>> the tools read it.
>
> Figure out somehow, based on the project file and the contents of the
> file system, hopefully without asking the user more questions.
No need to "figure it out"; the search path for project files is right
there in the file:
for Project_Path use
("../../../org.stephe_leake.sal/build/release",
"../../../org.stephe_leake.gtk/build/release",
"../../../org.stephe_leake.makerules");
That's where the project files for sal, sal_gtk, standard_common are
found.
This is interpreted relative to the directory the project file is found
in.
> For instance, even if the project file gives no direct indication of
> which search-path elements are "proper" projects, some Elisp code
> could look inside each of those directories (and one level above,
> maybe) and see if there are project files in any of them. And those
> where there are, promote to project-roots as well.
> That might be harder for Ada, but should be totally doable for
> Java-based projects (if build.gradle doesn't provide that info).
For Ada project files, everything is crystal clear; anything mentioned
in "with" is a proper project; nothing else can be. No searching
involved.
Same for gradle; every build.gradle file defines a proper project;
nothing else does.
> But consider this: elisp-search-path is not ideal (the current project
> could be not in load-path).
I don't understand; the user sets load-path. If the user wants to use
project-* and xref-* functions with the current project, why is it not
in load-path?
That's just bad user configuration; no amount of elisp intelligence can
fix that.
xref-find-definitions for elisp relies on the code being loaded, so
using it without the code being loaded makes no sense.
>> It can still be up to the backend to decide whether they
>> are flat or require additional recursion.
>
> Err, that sounds silly. Why would you want not to recurse?
1) Because many Emacs functions don't recurse. For example, I'd like to add:
(defun project-find-file ()
"Prompt for a file name, find it on project-search-path."
(interactive)
(let ((filename
(completing-read
"file: "
(apply-partially
'locate-file-completion-table
(project-search-path (project-current))
nil))))
(find-file (locate-file filename (project-search-path (project-current)) nil))))
This works fine with "load-path" instead of "project-search-path". But
it won't work unless project-search-path is flat;
locate-file-completion-table and locate-file do not recurse.
2) As I've mentioned before, not recursing allows computing the set of
included/excluded directories once, instead of computing it again on
each recursion.
3) Because some tools provide and expect a flat search path (AdaCore
tools, gcc include path, emacs load-path, compilation-search-path,
ff-search-directories).
It is not at all clear how to convert a flat path into a recursive path
with ignores; the solution you have for elisp load-path is a kludge,
and could easily be wrong for some users.
> The current xref-collect-matches implementation unconditionally
> recurses, and it doesn't seem particularly useful to create a variant
> that doesn't.
The backends I'm aware of require flat paths (no recursion). So unless
some plausible backend actually requires recursion, it makes more sense
to assume recursion is not required.
>> Locations are tied other things besides cross-reference; cross-reference
>> is just one way of producing a list of locations.
>>
>> find-regexp is another way.
>>
>> compilation errors is another way.
>>
>> All of those could use a common location API. 'project' is one place to
>> put it; it could be in its own 'location' namespace.
>
> I suppose you're right. But until we have a unified location API
> (which I wouldn't count on appearing anytime soon), xref is the better
> place.
>
> xref-location lives there, and it's the best purpose-agnostic location
> class that we have now.
So move it to "location" instead of "xref". The longest journey starts
with one step ...
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-30 8:17 ` Stephen Leake
@ 2015-07-31 0:15 ` Dmitry Gutov
2015-07-31 16:13 ` Stephen Leake
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-07-31 0:15 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/30/2015 11:17 AM, Stephen Leake wrote:
> For Ada project files, everything is crystal clear; anything mentioned
> in "with" is a proper project; nothing else can be. No searching
> involved.
Fantastic, then Ada gives you all you need to implement both
project-search-path and project-roots.
> I don't understand; the user sets load-path. If the user wants to use
> project-* and xref-* functions with the current project, why is it not
> in load-path?
It's not always the current practice, AFAIK. And adding it to load-path
interactively is not something we should force the users to do.
Consider that when xref-find-references doesn't find anything, it will
say "nothing found", without further explanations, leaving the user to
puzzle out why and what. We should err on the side of showing too much,
rather than too little.
> xref-find-definitions for elisp relies on the code being loaded, so
> using it without the code being loaded makes no sense.
One can also load the code without adding the file to load-path. For
instance, via eval-buffer.
> 1) Because many Emacs functions don't recurse. For example, I'd like to add:
>
> (defun project-find-file ()
> "Prompt for a file name, find it on project-search-path."
> (interactive)
> (let ((filename
> (completing-read
> "file: "
> (apply-partially
> 'locate-file-completion-table
> (project-search-path (project-current))
> nil))))
>
> (find-file (locate-file filename (project-search-path (project-current)) nil))))
>
> This works fine with "load-path" instead of "project-search-path". But
> it won't work unless project-search-path is flat;
> locate-file-completion-table and locate-file do not recurse.
I guess you'd need to implement a "flattening" function for that,
indeed. I don't really see myself using this command, though.
> 2) As I've mentioned before, not recursing allows computing the set of
> included/excluded directories once, instead of computing it again on
> each recursion.
And I countered that, for many commands, computing that set is unlikely
to be the bottleneck. And cache invalidation is not user-friendly.
> 3) Because some tools provide and expect a flat search path (AdaCore
> tools, gcc include path, emacs load-path, compilation-search-path,
> ff-search-directories).
That's not really true.
There are many directories inside /usr/include, but C_INCLUDE_PATH won't
mention most of them. Instead, #include <postgresql/pg_config> will use
/usr/include/postgresql/pg_config.h, as long as /usr/include is in
C_INCLUDE_PATH.
There's only one directory in Emacs load-path with "cedet" in its name,
but (require 'semantic/db) will load lisp/cedet/semantic/db.el.
You're right about compilation-search-path and ff-search-directories,
though.
It seems to be true for the Ada source search path as well, but since
/usr/lib/gcc/x86_64-linux-gnu/4.9/adainclude/ has no subdirectories, the
point is moot. Traversing it recursively wouldn't make anything worse.
> It is not at all clear how to convert a flat path into a recursive path
> with ignores; the solution you have for elisp load-path is a kludge,
> and could easily be wrong for some users.
I'd like to see one of those users first.
>> xref-location lives there, and it's the best purpose-agnostic location
>> class that we have now.
>
> So move it to "location" instead of "xref". The longest journey starts
> with one step ...
Until it's used in at least two different places, creating a separate
package for it is, I think, premature.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-31 0:15 ` Dmitry Gutov
@ 2015-07-31 16:13 ` Stephen Leake
2015-08-01 0:57 ` Dmitry Gutov
2015-08-01 1:14 ` Per-language project-search-path, was: " Dmitry Gutov
0 siblings, 2 replies; 87+ messages in thread
From: Stephen Leake @ 2015-07-31 16:13 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/30/2015 11:17 AM, Stephen Leake wrote:
>
>> I don't understand; the user sets load-path. If the user wants to use
>> project-* and xref-* functions with the current project, why is it not
>> in load-path?
>
> It's not always the current practice, AFAIK.
I still don't understand why. But I've proposed
`project-add-search-path' (below and in another email), so that can be
used to handle this case.
> And adding it to load-path interactively is not something we should
> force the users to do.
Yes, if the user doesn't want a file in load-path, they don't put it
there. But then they will have to use project-add-search-path instead.
We have to expect some help from the user to indicate what's in the
project!
> Consider that when xref-find-references doesn't find anything, it will
> say "nothing found", without further explanations, leaving the user to
> puzzle out why and what. We should err on the side of showing too
> much, rather than too little.
Sorry, I can't guess what you have in mind that the user is doing here.
What is the user searching for, what did they do to specify
project-search-path, and what do they expect xref-find-references to
show?
>> 1) Because many Emacs functions don't recurse. For example, I'd like to add:
>>
>> (defun project-find-file ()
>> "Prompt for a file name, find it on project-search-path."
>> (interactive)
>> (let ((filename
>> (completing-read
>> "file: "
>> (apply-partially
>> 'locate-file-completion-table
>> (project-search-path (project-current))
>> nil))))
>>
>> (find-file (locate-file filename (project-search-path (project-current)) nil))))
>>
>> This works fine with "load-path" instead of "project-search-path". But
>> it won't work unless project-search-path is flat;
>> locate-file-completion-table and locate-file do not recurse.
>
> I guess you'd need to implement a "flattening" function for that,
> indeed.
If project-search-path is recursive, yes. Which is certainly possible to
implement here; I propose a "recursive" project flag below. If that's
set, this function can use directory-files and directories in
project-ignores to do the recursion.
The predicate for locate-file-completion-table (set to nil above) needs
to handle the files in project-ignores as well.
> I don't really see myself using this command, though.
That doesn't mean it's not a valid use case; you need to consider all
users.
I often want to look at project.el (or xref.el, ada-mode.el, etc); I
bind this function (currently using load-path) to M-F12, so M-F12
proj <ret> finds it for me easily.
The Ada user group requested this function for Ada projects (it has a
similar implementation).
>> 2) As I've mentioned before, not recursing allows computing the set of
>> included/excluded directories once, instead of computing it again on
>> each recursion.
>
> And I countered that, for many commands, computing that set is
> unlikely to be the bottleneck. And cache invalidation is not
> user-friendly.
Which is faster/friendlier depends on the details, so we'll have to wait
for actual implementations. All I'm asking is that we do not make design
decisions at the generic project API level that rule out this approach.
>> 3) Because some tools provide and expect a flat search path (AdaCore
>> tools, gcc include path, emacs load-path, compilation-search-path,
>> ff-search-directories).
>
> That's not really true.
>
> There are many directories inside /usr/include, but C_INCLUDE_PATH
> won't mention most of them. Instead, #include <postgresql/pg_config>
> will use /usr/include/postgresql/pg_config.h, as long as /usr/include
> is in C_INCLUDE_PATH.
Yes. So if the user wants to search all relevant includes for some
declaration, they have to treat C_INCLUDE_PATH as recursive. But the
compiler (which _defines_ C_INCLUDE_PATH) does _not_ treat it as
recursive. So it's confused.
> There's only one directory in Emacs load-path with "cedet" in its
> name, but (require 'semantic/db) will load lisp/cedet/semantic/db.el.
Similar situation: `load' does not treat `load-path' as recursive, but
text searches need to. I had not considered that.
I think cedet is the only exception to the 'flat load-path' rule (at
least in core emacs); I hope it goes away, and is not copied by other
projects.
I would provide a special syntax to handle this in
project-add-search-path:
(defun project-add-search-path (project path &optional make-recursive)
"Add PATH (a list of directories) to the search path for PROJECT.
If a directory in PATH ends in \"/**\", also add all directories under
that directory.
If MAKE-RECURSIVE is non-nil, the full project search path is pruned so
that only highest-level directories are present, and all uses of
`project-search-path' recurse into subdirectories. In this case, it is
likely you will also need to specify ignored directories with
`project-add-ignores'.
If MAKE-RECURSIVE is nil, uses of `project-search-path' no not recurse
into subdirectories."
... )
The Ada project files support the ** sytnax.
User projects can have directory structures for which recursive
search from the vc root is wrong:
myproj/
.git/
.gitignore
build/
Makefile
ada_project.gpr
*.exe
obj/
# contains object files for ada files
lisp/
# elisp files for emacs-25+
lisp-emacs-24/
# elisp files defining emacs-25 functions
lisp-emacs-23/
ada/
# ada files
The elisp project (depending on current Emacs version) will only want to
search one of:
(lisp)
(lisp lisp-emacs-24)
(lisp lisp-emacs-23)
The Ada project will only want to search the "ada" and "build"
directories (for the Makefile and Ada project file).
There will be cases where the user will want to search all files; then a
separate project that uses project-add-search-path to add all dirs would
be appropriate.
No search should include the "obj" directory.
This could be handled if the user can explicitly specify project ignores
(not just rely on .gitignore etc; git does _not_ ignore the lisp-emacs-*
and ada directories). More complicated structures are harder to handle
with ignores; you'll end up with a long flat list of ignores, so it
might be simpler to have a flat list of includes instead.
> You're right about compilation-search-path and ff-search-directories,
> though.
>
> It seems to be true for the Ada source search path as well, but since
> /usr/lib/gcc/x86_64-linux-gnu/4.9/adainclude/ has no subdirectories,
> the point is moot. Traversing it recursively wouldn't make anything
> worse.
User Ada projects can easily have directory structures for which
recursive search from the vc root would be wrong; see above.
It would be surprising to an Ada user (used to flat search paths), if the
Emacs project silently started using recursive search paths.
>> It is not at all clear how to convert a flat path into a recursive path
>> with ignores; the solution you have for elisp load-path is a kludge,
>> and could easily be wrong for some users.
>
> I'd like to see one of those users first.
Me, and many other teams at NASA - structures similar to the above,
where the non-included directories can also be release version,
debug/release options, target hardware/os option, compiler
vendor/version, etc.
The jdee project has lisp and java directories.
The kludge is in the root implementation of project-search-path, so it
applies to all project backends that only customize
project-search-path-function.
I have no objection to allowing some project backends to use recursive
path/ignore dirs; I just want to also allow some backends to use
non-recursive path/ignore files. Or the user can choose, via
project-add-search-path above.
>>> xref-location lives there, and it's the best purpose-agnostic location
>>> class that we have now.
>>
>> So move it to "location" instead of "xref". The longest journey starts
>> with one step ...
>
> Until it's used in at least two different places, creating a separate
> package for it is, I think, premature.
Right. We now have those two places; xref-find-definitions,
project-find-regexp.
Unless project-find-regexp uses the compile location data structure,
which would probably be better.
Although it would be even better if project-find-regexp uses the compile
location data structure wrapped in a location-compile class, derived
from the root location class, renamed from xref-location.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-07-31 16:13 ` Stephen Leake
@ 2015-08-01 0:57 ` Dmitry Gutov
2015-08-01 9:50 ` Stephen Leake
2015-08-01 1:14 ` Per-language project-search-path, was: " Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-01 0:57 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/31/2015 07:13 PM, Stephen Leake wrote:
> I often want to look at project.el (or xref.el, ada-mode.el, etc); I
> bind this function (currently using load-path) to M-F12, so M-F12
> proj <ret> finds it for me easily.
To reach the same result, I currently either use ido-switch-buffer (with
ido-use-virtual-buffers set to t) or projectile-find-file (which asks
the user to choose between files in the project, based on file names
relative to the root). The key thing here is substring matching: I also
simply type "project." and see lisp/progmodes/project.el as the only
completion.
Using relative paths also helps with navigating to files with the same
name inside different directories.
> Which is faster/friendlier depends on the details, so we'll have to wait
> for actual implementations. All I'm asking is that we do not make design
> decisions at the generic project API level that rule out this approach.
We do have to make some decisions, though. For instance, supporting both
recursive and non-recursive elements in search-path will require a
separate code path for each function that uses it.
As such, it will require a separate code path in xref-collect-matches
and an alternative for xref--rgrep-command.
>> There are many directories inside /usr/include, but C_INCLUDE_PATH
>> won't mention most of them. Instead, #include <postgresql/pg_config>
>> will use /usr/include/postgresql/pg_config.h, as long as /usr/include
>> is in C_INCLUDE_PATH.
>
> Yes. So if the user wants to search all relevant includes for some
> declaration, they have to treat C_INCLUDE_PATH as recursive. But the
> compiler (which _defines_ C_INCLUDE_PATH) does _not_ treat it as
> recursive. So it's confused.
We need to search files in subdirectories as well, and that is the key
distinction.
The fact that compiler treats the roots in a special way, however, can
be important. Because of that, the list of roots contains information
that can be lost after a naive translation to a list of "flat"
directories. For instance, if you have the roots, you can search them
linearly and tell whether there's a file corresponding to
'postgresql/pg_config' inside one of them.
With a flat list, we could only give an imprecise answer to that
question, and the search might have to perform extra work.
> I think cedet is the only exception to the 'flat load-path' rule (at
> least in core emacs); I hope it goes away, and is not copied by other
> projects.
Even though it's not very popular, it's unlikely to go away. But in any
case, the corresponding path variables in all other ecosystems that I
know of (Ruby, Python, Java, Node.js) follow in the footsteps of C and
Emacs.
And the principle of translating qualified package (or module, or class)
names to file names is quite pervasive.
Ada is an exception, rather than the rule.
> I would provide a special syntax to handle this in
> project-add-search-path:
>
> (defun project-add-search-path (project path &optional make-recursive)
> "Add PATH (a list of directories) to the search path for PROJECT.
>
> If a directory in PATH ends in \"/**\", also add all directories under
> that directory.
>
> If MAKE-RECURSIVE is non-nil, the full project search path is pruned so
> that only highest-level directories are present, and all uses of
> `project-search-path' recurse into subdirectories. In this case, it is
> likely you will also need to specify ignored directories with
> `project-add-ignores'.
This description explains the purpose, but doesn't describe the
resulting data structure that would hold that information. The ones that
come to mind are rather ugly.
> If MAKE-RECURSIVE is nil, uses of `project-search-path' no not recurse
> into subdirectories."
> ... )
>
> The Ada project files support the ** sytnax.
The /** syntax is rather nifty. But why do you propose to have it and
MAKE-RECURSIVE at the same time? We could just interpret /** as the
recursive marker. Then the result could be a list of strings.
> This could be handled if the user can explicitly specify project ignores
> (not just rely on .gitignore etc; git does _not_ ignore the lisp-emacs-*
> and ada directories). More complicated structures are harder to handle
> with ignores; you'll end up with a long flat list of ignores, so it
> might be simpler to have a flat list of includes instead.
I've just started to write about this... Yes indeed, I think every flat
structure can be translated into path/to/dir, plus path/to/dir/*/ in the
ignores. The translation function shouldn't take too much work to write.
There's no need to produce the list of ignores manually. The Ada project
backend could do that for the user. And if the list is long... first,
you can use wildcards. Second, at least xref-collect-matches won't have
to handle the flat case specially. Nor will the other consumers of the
project API.
> It would be surprising to an Ada user (used to flat search paths), if the
> Emacs project silently started using recursive search paths.
The Ada project implementation can provide a friendlier interface/set of
customizations/etc for Ada users. Different communities are used to
different things. Any default would be surprising to someone.
>>> It is not at all clear how to convert a flat path into a recursive path
>>> with ignores; the solution you have for elisp load-path is a kludge,
>>> and could easily be wrong for some users.
>>
>> I'd like to see one of those users first.
>
> Me, and many other teams at NASA - structures similar to the above,
> where the non-included directories can also be release version,
> debug/release options, target hardware/os option, compiler
> vendor/version, etc.
There can be different opinions about this. As a project maintainer, I'd
rather be able to find all references to the function foo, no matter the
target OS of the code, or the targeted compiler version.
> The jdee project has lisp and java directories.
The Java class path is also "recursive".
> The kludge is in the root implementation of project-search-path, so it
> applies to all project backends that only customize
> project-search-path-function.
Please, stop calling major modes that do that "project backends".
Customizing project-search-path-function does not constitute one.
A real backend can always override project-search-path (and smart ones
should).
>>> So move it to "location" instead of "xref". The longest journey starts
>>> with one step ...
>>
>> Until it's used in at least two different places, creating a separate
>> package for it is, I think, premature.
>
> Right. We now have those two places; xref-find-definitions,
> project-find-regexp.
Both still use xref for display. The location values aren't consumed by
anything else. So that's just one place.
> Although it would be even better if project-find-regexp uses the compile
> location data structure wrapped in a location-compile class, derived
> from the root location class, renamed from xref-location.
You're welcome to explore this direction with code.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Per-language project-search-path, was: Re: Unified project interface
2015-07-31 16:13 ` Stephen Leake
2015-08-01 0:57 ` Dmitry Gutov
@ 2015-08-01 1:14 ` Dmitry Gutov
2015-08-01 10:43 ` Stephen Leake
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-01 1:14 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 07/31/2015 07:13 PM, Stephen Leake wrote:
> The elisp project (depending on current Emacs version) will only want to
> search one of:
>
> (lisp)
> (lisp lisp-emacs-24)
> (lisp lisp-emacs-23)
>
> The Ada project will only want to search the "ada" and "build"
> directories (for the Makefile and Ada project file).
By the way, do you think project-search-path should have a major-mode
(or language) argument? Or maybe depend on it implicitly, via the
current buffer?
ede-source-paths dispatches on both PROJECT and MODE; to reuse it (in
the ede project backend), we'd have to produce the mode argument.
The caveats with the given choices are:
- Users can use different major modes for the same files. Normally,
project support shouldn't affect that.
- We have no registry of languages, AFAIK. And different languages can
be spelled differently (C++ vs Cpp, Fortran vs F77, Objective-C vs ObjC).
- Just prescribing to use the current buffer in whatever way the project
backend wants could be the most flexible, but it doesn't solve either
difficulty.
We could ask a similar question for project-ignores as well, though I'd
rather not go that far. It needs a DIRECTORY argument already.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-08-01 0:57 ` Dmitry Gutov
@ 2015-08-01 9:50 ` Stephen Leake
2015-08-01 10:51 ` Stephen Leake
2015-08-01 12:40 ` Dmitry Gutov
0 siblings, 2 replies; 87+ messages in thread
From: Stephen Leake @ 2015-08-01 9:50 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/31/2015 07:13 PM, Stephen Leake wrote:
>
>> Which is faster/friendlier depends on the details, so we'll have to wait
>> for actual implementations. All I'm asking is that we do not make design
>> decisions at the generic project API level that rule out this approach.
>
> We do have to make some decisions, though. For instance, supporting
> both recursive and non-recursive elements in search-path will require
> a separate code path for each function that uses it.
>
> As such, it will require a separate code path in xref-collect-matches
> and an alternative for xref--rgrep-command.
Or they could abort; that would be ok while we are experimenting. It may
be that one or the other will turn out to be The Right Way.
A backend that uses flat paths doesn't have to use those functions. Or,
the people writing that backend can modify them if that seems better
than putting similar code elsewhere. That would be part of the cost of
that design decision.
>> I would provide a special syntax to handle this in
>> project-add-search-path:
>>
>> (defun project-add-search-path (project path &optional make-recursive)
>> "Add PATH (a list of directories) to the search path for PROJECT.
>>
>> If a directory in PATH ends in \"/**\", also add all directories under
>> that directory.
>>
>> If MAKE-RECURSIVE is non-nil, the full project search path is pruned so
>> that only highest-level directories are present, and all uses of
>> `project-search-path' recurse into subdirectories. In this case, it is
>> likely you will also need to specify ignored directories with
>> `project-add-ignores'.
>>
>> If MAKE-RECURSIVE is nil, uses of `project-search-path' no not recurse
>> into subdirectories."
>> ... )
>>
>> The Ada project files support the ** sytnax.
>>
> This description explains the purpose, but doesn't describe the
> resulting data structure that would hold that information. The ones
> that come to mind are rather ugly.
I have no idea what data structures come to mind for you.
This could be implemented with a root projects type:
(cl-defstruct (projects)
user-search-path ;; list of directories added to project-search-path
user-ignores ;; list of ignores added to project-ignores
recursive-search-path
;; if non-nil, project-search-path is treated as recursive in all usees
...
)
Is that "ugly"?
> The /** syntax is rather nifty. But why do you propose to have it and
> MAKE-RECURSIVE at the same time? We could just interpret /** as the
> recursive marker. Then the result could be a list of strings.
Because you never need both. If you are used to using recursive paths,
adding ** in the path to tell the code that will seem odd. So it seems
best to have an explicit flag.
It might be better to specify the recursive flag in the project
constructor; then this function would check it, not set it.
>> This could be handled if the user can explicitly specify project ignores
>> (not just rely on .gitignore etc; git does _not_ ignore the lisp-emacs-*
>> and ada directories). More complicated structures are harder to handle
>> with ignores; you'll end up with a long flat list of ignores, so it
>> might be simpler to have a flat list of includes instead.
>
> I've just started to write about this... Yes indeed, I think every
> flat structure can be translated into path/to/dir, plus path/to/dir/*/
> in the ignores. The translation function shouldn't take too much work
> to write.
The reverse is also true; it is possible to convert recursive
path/ignores into flat path/ignores.
> There's no need to produce the list of ignores manually.
Only if you ignore the use cases I posted. There are situations where
the algorithm will not do what the user wants.
The fact that you don't like them does not mean they are not valid.
Would it be such a problem to provide 'project-add-ignores'?
>> Me, and many other teams at NASA - structures similar to the above,
>> where the non-included directories can also be release version,
>> debug/release options, target hardware/os option, compiler
>> vendor/version, etc.
>
> There can be different opinions about this.
Yes, precisely. And Emacs core should not be enforcing any opinions!
>>>> So move it to "location" instead of "xref". The longest journey starts
>>>> with one step ...
>>>
>>> Until it's used in at least two different places, creating a separate
>>> package for it is, I think, premature.
>>
>> Right. We now have those two places; xref-find-definitions,
>> project-find-regexp.
>
> Both still use xref for display. The location values aren't consumed
> by anything else. So that's just one place.
They both display locations. The code that displays locations should
also be in the locations namespace, not the xref one.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-01 1:14 ` Per-language project-search-path, was: " Dmitry Gutov
@ 2015-08-01 10:43 ` Stephen Leake
2015-08-01 14:12 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-08-01 10:43 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 07/31/2015 07:13 PM, Stephen Leake wrote:
>
>> The elisp project (depending on current Emacs version) will only want to
>> search one of:
>>
>> (lisp)
>> (lisp lisp-emacs-24)
>> (lisp lisp-emacs-23)
>>
>> The Ada project will only want to search the "ada" and "build"
>> directories (for the Makefile and Ada project file).
>
> By the way, do you think project-search-path should have a major-mode
> (or language) argument? Or maybe depend on it implicitly, via the
> current buffer?
My experience is always with a project file; the user explicitly selects
which project file is active. That can be multi-language or
single-language.
So I'm not familiar with the project system trying to guess a project on
the fly, as the current code does.
As you point out, there are use cases for using the major mode to
influence the project, and others for not doing that.
I don't know which would be better as the default; the only general
solution is to require a project file.
> ede-source-paths dispatches on both PROJECT and MODE; to reuse it (in
> the ede project backend), we'd have to produce the mode argument.
Ede also uses the (require 'path/file) style; not surprising, since it
was written by the CEDET team.
I don't see any implementations of ede-source-paths, and only one use of
it, so it's hard to say what the purpose of the mode parameter is.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-08-01 9:50 ` Stephen Leake
@ 2015-08-01 10:51 ` Stephen Leake
2015-08-01 12:42 ` Dmitry Gutov
2015-08-01 12:40 ` Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-08-01 10:51 UTC (permalink / raw)
To: emacs-devel
Stephen Leake <stephen_leake@stephe-leake.org> writes:
> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> The /** syntax is rather nifty. But why do you propose to have it and
>> MAKE-RECURSIVE at the same time? We could just interpret /** as the
>> recursive marker. Then the result could be a list of strings.
I read this wrong.
You _only_ need /** if the paths are _not_ recursive, so this cannot
serve as the recursive flag.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-08-01 9:50 ` Stephen Leake
2015-08-01 10:51 ` Stephen Leake
@ 2015-08-01 12:40 ` Dmitry Gutov
2015-08-01 14:15 ` Stephen Leake
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-01 12:40 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 08/01/2015 12:50 PM, Stephen Leake wrote:
>> As such, it will require a separate code path in xref-collect-matches
>> and an alternative for xref--rgrep-command.
>
> Or they could abort; that would be ok while we are experimenting. It may
> be that one or the other will turn out to be The Right Way.
That's not helpful. Of course we want the API be straightforward and
usable for as many callers as possible. It's the main goal.
I'm not sure how The Right Way would supposedly present itself.
> A backend that uses flat paths doesn't have to use those functions. Or,
> the people writing that backend can modify them if that seems better
> than putting similar code elsewhere. That would be part of the cost of
> that design decision.
It's a high cost.
If the author of such backend intends to reimplement or rewrite every
feature like that, they can just as well not bother with implementing
the Project API. As you can see, there's not much code in project.el, so
they won't miss much if they don't value interoperability.
> This could be implemented with a root projects type:
>
> (cl-defstruct (projects)
> user-search-path ;; list of directories added to project-search-path
> user-ignores ;; list of ignores added to project-ignores
> recursive-search-path
> ;; if non-nil, project-search-path is treated as recursive in all usees
> ...
> )
That doesn't tell me what values project-search-path would return, for
every kind of configuration. That's the important part.
> Is that "ugly"?
Depends on how you feel about double negatives. There should be just one
way to express whether a given directory should be traversed recursively.
(cl-defstruct (projects)
flat-search-path
recursive-search-path
would be one option for the implementation, but what would it mean for
the API? If we have both project-flat-search-path and
project-recursive-search-path as accessors, there's high probability
that some functions will only look up and handle one of them. That's bad.
>> The /** syntax is rather nifty. But why do you propose to have it and
>> MAKE-RECURSIVE at the same time? We could just interpret /** as the
>> recursive marker. Then the result could be a list of strings.
>
> Because you never need both. If you are used to using recursive paths,
> adding ** in the path to tell the code that will seem odd. So it seems
> best to have an explicit flag.
The user could specify the paths in any way they like, are used to, etc.
What's important is what the project backend returns in project-search-path.
>> I've just started to write about this... Yes indeed, I think every
>> flat structure can be translated into path/to/dir, plus path/to/dir/*/
>> in the ignores. The translation function shouldn't take too much work
>> to write.
>
> The reverse is also true; it is possible to convert recursive
> path/ignores into flat path/ignores.
Not without drawbacks. First, it's more computationally intensive (Emacs
would have to traverse the actual directory tree). Second, like I
mentioned before, that loses information: in particular, we won't know
which directories were the root ones.
>> There's no need to produce the list of ignores manually.
>
> Only if you ignore the use cases I posted. There are situations where
> the algorithm will not do what the user wants.
What algorithm? I don't see why you can't write an algorithm that will
both handle flat directories via ignores, as well as honor the user's
wishes.
> The fact that you don't like them does not mean they are not valid.
>
> Would it be such a problem to provide 'project-add-ignores'?
I don't understand why you would state this question here? It doesn't
seem relevant.
>>> Me, and many other teams at NASA - structures similar to the above,
>>> where the non-included directories can also be release version,
>>> debug/release options, target hardware/os option, compiler
>>> vendor/version, etc.
>>
>> There can be different opinions about this.
>
> Yes, precisely. And Emacs core should not be enforcing any opinions!
Err, some opinions are worth being enforced. But anyway, the Ada project
backend can behave any way you wish, in this respect.
>> Both still use xref for display. The location values aren't consumed
>> by anything else. So that's just one place.
>
> They both display locations. The code that displays locations should
> also be in the locations namespace, not the xref one.
What's left for xref, then? Most of the code there deals with displaying
xrefs in the buffer.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-08-01 10:51 ` Stephen Leake
@ 2015-08-01 12:42 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-01 12:42 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 08/01/2015 01:51 PM, Stephen Leake wrote:
> You _only_ need /** if the paths are _not_ recursive, so this cannot
> serve as the recursive flag.
It would serve as "recursive flag" for each particular directory.
Of course, in this approach the directories are flat by default.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-01 10:43 ` Stephen Leake
@ 2015-08-01 14:12 ` Dmitry Gutov
2015-08-01 18:57 ` Stephen Leake
2015-08-02 2:29 ` Eric Ludlam
0 siblings, 2 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-01 14:12 UTC (permalink / raw)
To: Stephen Leake, emacs-devel; +Cc: Eric Ludlam
On 08/01/2015 01:43 PM, Stephen Leake wrote:
> My experience is always with a project file; the user explicitly selects
> which project file is active. That can be multi-language or
> single-language.
>
> So I'm not familiar with the project system trying to guess a project on
> the fly, as the current code does.
I'm not discussing a particular project backend here, just the API. And
you can use the user-selected backend everywhere, if there's an
appropriate element at the head of project-find-functions.
If I recall, you've mentioned a concept of project file listing
different search-paths for different languages. Without
project-search-path taking the language into account, you'll have to
append all paths together.
And what about jdee? Would you treat lisp/.dir-locals.el as the project
file for Elisp, and ask the user to select it?
> As you point out, there are use cases for using the major mode to
> influence the project, and others for not doing that.
I've only listed the implementation difficulties. If there are any
particular problems with the concept of dispatching on the language, I'd
like to hear about those.
> I don't know which would be better as the default; the only general
> solution is to require a project file.
How does that translate to API requirements?
> Ede also uses the (require 'path/file) style; not surprising, since it
> was written by the CEDET team.
It does. This doesn't seem relevant.
> I don't see any implementations of ede-source-paths, and only one use of
> it, so it's hard to say what the purpose of the mode parameter is.
That's true, unfortunately. The only implementations I could find are in
malabar-mode (another third-party project attempting to provide a Java
editing environment).
Eric, could you tell if we're missing anything? Are there other places
where ede-source-paths is defined or used?
Any reason why symref isn't using it?
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-08-01 12:40 ` Dmitry Gutov
@ 2015-08-01 14:15 ` Stephen Leake
2015-08-01 15:09 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-08-01 14:15 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 08/01/2015 12:50 PM, Stephen Leake wrote:
>
>> This could be implemented with a root projects type:
>>
>> (cl-defstruct (projects)
>> user-search-path ;; list of directories added to project-search-path
>> user-ignores ;; list of ignores added to project-ignores
>> recursive-search-path
>> ;; if non-nil, project-search-path is treated as recursive in all usees
>> ...
>> )
>
> That doesn't tell me what values project-search-path would return, for
> every kind of configuration. That's the important part.
(cl-defgeneric project-search-path (project)
"..."
(project--prune-directories
(nconc (project-user-search-path project)
(funcall project-search-path-function)
;; Include these, because we don't know any better.
;; But a specialized implementation may include only some of
;; the project's subdirectories, if there are no source
;; files at the top level.
(project-directories project))))
>> Is that "ugly"?
>
> Depends on how you feel about double negatives. There should be just
> one way to express whether a given directory should be traversed
> recursively.
There is; the boolean flag recursive-search-path.
> (cl-defstruct (projects)
> flat-search-path
> recursive-search-path
>
I'm guessing that here "flat-search-path" is a list of directories,
intended to be used non-recursively. A comment would really help;
guessing is dangerous in this discussion.
And then "recursive-search-path" is a list of directories, intended to
be used recursively.
So they contain redundant information; they are each a cache of the
result of the current `project-search-path' (with and without pruning).
> would be one option for the implementation, but what would it mean for
> the API? If we have both project-flat-search-path and
> project-recursive-search-path as accessors, there's high probability
> that some functions will only look up and handle one of them. That's
> bad.
Yes, it would be nice if we could agree that either flat or recursive is
the baseline; any function that needs the other can convert.
So far, I think we have these arguments on each side:
- elisp load-path is already mostly flat
it needs to be expanded to handle path/file style requires
- Ada project paths are flat
This is actually AdaCore project paths; they support C++ and C as well.
- Some other project paths are recursive
- Converting between them (while also converting the corresponding
includes) is possible
- Using find-grep and similar external programs allows recursive search
to display first results immediately
>>> I've just started to write about this... Yes indeed, I think every
>>> flat structure can be translated into path/to/dir, plus path/to/dir/*/
>>> in the ignores. The translation function shouldn't take too much work
>>> to write.
>>
>> The reverse is also true; it is possible to convert recursive
>> path/ignores into flat path/ignores.
>
> Not without drawbacks. First, it's more computationally intensive
> (Emacs would have to traverse the actual directory tree).
You have to traverse the tree at some point; either during the use of
the recursive path, or to convert the path.
Converting it once is cheaper overall.
But, as I've said before, this is an optimization issue. If it turns out
that the performance either way is acceptable, it's not important.
> Second, like
> I mentioned before, that loses information: in particular, we won't
> know which directories were the root ones.
If that is useful information, obviously it can be saved. That's what
project-roots is for.
>>> There's no need to produce the list of ignores manually.
>>
>> Only if you ignore the use cases I posted. There are situations where
>> the algorithm will not do what the user wants.
>
> What algorithm?
The algorithm that automatically produces the list of ignores.
> I don't see why you can't write an algorithm that will
> both handle flat directories via ignores, as well as honor the user's
> wishes.
The user has to have a way of expressing those wishes. That's what
project-add-ignores is for.
>> The fact that you don't like them does not mean they are not valid.
>>
>> Would it be such a problem to provide 'project-add-ignores'?
>
> I don't understand why you would state this question here? It doesn't
> seem relevant.
I'll take that as agreeing to add the functions project-add-search-path
and project-add-ignores to project.el
Given all of the above, I'll make an attempt to write Ada and JDEE
project backends, using recursive paths. We'll see how that goes.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-08-01 14:15 ` Stephen Leake
@ 2015-08-01 15:09 ` Dmitry Gutov
2015-08-01 19:04 ` Stephen Leake
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-01 15:09 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 08/01/2015 05:15 PM, Stephen Leake wrote:
> (cl-defgeneric project-search-path (project)
> "..."
> (project--prune-directories
> (nconc (project-user-search-path project)
> (funcall project-search-path-function)
> ;; Include these, because we don't know any better.
> ;; But a specialized implementation may include only some of
> ;; the project's subdirectories, if there are no source
> ;; files at the top level.
> (project-directories project))))
That doesn't use recursive-search-path, and doesn't allow to treat only
some elements recursively. Which would be natural.
And if we allowed /**, the recursive-search-path flag would be redundant.
>> (cl-defstruct (projects)
>> flat-search-path
>> recursive-search-path
>>
>
> I'm guessing that here "flat-search-path" is a list of directories,
> intended to be used non-recursively.
> ...
> And then "recursive-search-path" is a list of directories, intended to
> be used recursively.
Yes.
> So they contain redundant information; they are each a cache of the
> result of the current `project-search-path' (with and without pruning).
This is backwards. There wouldn't be project-search-path function at
all, and neither flat-search-path, nor recursive-search-path would be
the result of pruning.
> Yes, it would be nice if we could agree that either flat or recursive is
> the baseline; any function that needs the other can convert.
That's the direction I'm inclined to take, too.
> So far, I think we have these arguments on each side:
>
> - elisp load-path is already mostly flat
For out purposes, it's plainly recursive, even if that feature isn't in
use in most projects. That's highly unlikely to ever change.
> it needs to be expanded to handle path/file style requires
Same as any other language with "recursive" search path that I know of.
> - Ada project paths are flat
>
> This is actually AdaCore project paths; they support C++ and C as well.
>
> - Some other project paths are recursive
Most of them, AFAIK. I've given several examples; there have been no
further counter-examples mentioned.
> - Converting between them (while also converting the corresponding
> includes) is possible
I think you mean the corresponding ignores. And converting a certain
class of ignores from recursive to flat paths seems non-trivial.
> - Using find-grep and similar external programs allows recursive search
> to display first results immediately
That's a good point, and I think it hasn't been stated clearly before.
>> Second, like
>> I mentioned before, that loses information: in particular, we won't
>> know which directories were the root ones.
>
> If that is useful information, obviously it can be saved. That's what
> project-roots is for.
project-roots are different. This would have to be a separate accessor,
because the search path roots outside of the project are just as (if not
more) important.
>> I don't see why you can't write an algorithm that will
>> both handle flat directories via ignores, as well as honor the user's
>> wishes.
>
> The user has to have a way of expressing those wishes. That's what
> project-add-ignores is for.
The user will express their wishes via the project file and .gitignore.
There's nothing to discuss there; obviously, a project implementation
will have access to all information the user will give it.
> I'll take that as agreeing to add the functions project-add-search-path
> and project-add-ignores to project.el
Not really. I don't think we should mess with a project backend's
internals. So unless we see a use case for third-party code to modify
project's search-path and ignores, these functions should stay out of
the API.
> Given all of the above, I'll make an attempt to write Ada and JDEE
> project backends, using recursive paths. We'll see how that goes.
Hopefully, you'll make that attempt anyway.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-01 14:12 ` Dmitry Gutov
@ 2015-08-01 18:57 ` Stephen Leake
2015-08-02 0:25 ` Dmitry Gutov
2015-08-02 2:29 ` Eric Ludlam
1 sibling, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-08-01 18:57 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
> On 08/01/2015 01:43 PM, Stephen Leake wrote:
>
> If I recall, you've mentioned a concept of project file listing
> different search-paths for different languages. Without
> project-search-path taking the language into account, you'll have to
> append all paths together.
I don't know what you mean by "all paths" here; one project file defines
one source path.
In the case of a multilanguage project, that search path includes files
in all languages.
I often mix languages in one directory anyway; the file extension
indicates the language, so there's no need for it in the directory name.
> And what about jdee? Would you treat lisp/.dir-locals.el as the
> project file for Elisp, and ask the user to select it?
I think you are asking
"what project would you use while editing elisp files that implement
jdee"
not
"what project file does jdee use for Java projects".
Yes, serious users will want to explicitly select the active project.
If I'm debugging jdee, I want the search path to be elisp load-path,
whether I'm in a notes.text file, a jdee .el file, or a test .java file.
So I don't want Emacs to guess what the active project is, nor determine
the search path based on the mode.
When I'm testing jdee's use of the project API, I will explicitly select
a Java project as the active project.
I've just written this:
(cl-defstruct (elisp-project)
;; no slots yet
)
(cl-defmethod project-search-path ((type elisp-project))
(project--prune-directories load-path))
(cl-defmethod project-root ((type elisp-project))
;; no meaningful root
nil)
;; more overrides to come
(setq project-explicit-prj (make-elisp-project))
See below for project-explicit-prj.
In my mind, there is only one elisp project; all files currently on
load-path.
There will be a similar cl-defstruct for jdee-project.
>> I don't know which would be better as the default; the only general
>> solution is to require a project file.
>
> How does that translate to API requirements?
- Provide a mechanism for the user to explicitly set a project file.
For example (now in my local project-patches.el):
(defvar project-explicit-prj nil
"Project explicitly set by user.")
(defun project-explicit-prj (dir)
"Return the project the user has set in `project-explicit-prj'."
project-explicit-prj)
(setq project-find-functions 'project-explicit-prj)
A menu or completion list of previously selected projects would be
very useful here (I'm working on that).
- Provide a way for functions to require a particular type of project file
This is mostly there; all functions that only work with projects of
some type should be cl-defmethods for that type.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-08-01 15:09 ` Dmitry Gutov
@ 2015-08-01 19:04 ` Stephen Leake
2015-08-01 22:33 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Stephen Leake @ 2015-08-01 19:04 UTC (permalink / raw)
To: emacs-devel
Dmitry Gutov <dgutov@yandex.ru> writes:
>> I'll take that as agreeing to add the functions project-add-search-path
>> and project-add-ignores to project.el
>
> Not really. I don't think we should mess with a project backend's
> internals.
Well, yes; these should not be user level functions. But it would be
nice if any backend could use them to build the project object.
As you keep saying, "elisp does not have a project backend", so the
user is forced to kludges for elisp projects.
>> Given all of the above, I'll make an attempt to write Ada and JDEE
>> project backends, using recursive paths. We'll see how that goes.
>
> Hopefully, you'll make that attempt anyway.
Yes. I'll write what I need, and if there are functions/data structures
used in both JDEE and Ada project backends, I'll propose them as
additions to the project API.
It turns out I'm also writing a more complete elisp backend as well.
--
-- Stephe
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Unified project interface
2015-08-01 19:04 ` Stephen Leake
@ 2015-08-01 22:33 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-01 22:33 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 08/01/2015 10:04 PM, Stephen Leake wrote:
> Yes. I'll write what I need, and if there are functions/data structures
> used in both JDEE and Ada project backends, I'll propose them as
> additions to the project API.
Great. That should be the way to go.
Please be advised: I've added a second argument to project-ignores, like
discussed before.
> It turns out I'm also writing a more complete elisp backend as well.
Maybe we could add it to Emacs as well.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-01 18:57 ` Stephen Leake
@ 2015-08-02 0:25 ` Dmitry Gutov
0 siblings, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-02 0:25 UTC (permalink / raw)
To: Stephen Leake, emacs-devel
On 08/01/2015 09:57 PM, Stephen Leake wrote:
> I often mix languages in one directory anyway; the file extension
> indicates the language, so there's no need for it in the directory name.
All right. That's fine if you don't need it.
>> And what about jdee? Would you treat lisp/.dir-locals.el as the
>> project file for Elisp, and ask the user to select it?
>
> I think you are asking
>
> "what project would you use while editing elisp files that implement
> jdee"
>
> ...
>
> Yes, serious users will want to explicitly select the active project.
Okay. I could add that different languages can have interdependencies
(like templates can references to "normal" source code), so it's not
always easy to separate them. But I guess you can use their search paths
together, and rely on filtering by extension.
> If I'm debugging jdee, I want the search path to be elisp load-path,
> whether I'm in a notes.text file, a jdee .el file, or a test .java file.
> So I don't want Emacs to guess what the active project is, nor determine
> the search path based on the mode.
Fair enough.
> (setq project-find-functions 'project-explicit-prj)
When the package is ready, this should be added via add-hook, when a
certain minor mode is enabled.
>
> A menu or completion list of previously selected projects would be
> very useful here (I'm working on that).
That minor mode could provide a relevant command.
> - Provide a way for functions to require a particular type of project file
>
> This is mostly there; all functions that only work with projects of
> some type should be cl-defmethods for that type.
That's fine, but it's better is more functions are universal.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-01 14:12 ` Dmitry Gutov
2015-08-01 18:57 ` Stephen Leake
@ 2015-08-02 2:29 ` Eric Ludlam
2015-08-02 8:57 ` Nix
2015-08-03 1:21 ` Dmitry Gutov
1 sibling, 2 replies; 87+ messages in thread
From: Eric Ludlam @ 2015-08-02 2:29 UTC (permalink / raw)
To: Dmitry Gutov, Stephen Leake, emacs-devel; +Cc: Eric Ludlam
Hi,
Sorry for a long silence- it was a busy month. ;)
On 08/01/2015 10:12 AM, Dmitry Gutov wrote:
> On 08/01/2015 01:43 PM, Stephen Leake wrote:
>
>> My experience is always with a project file; the user explicitly selects
>> which project file is active. That can be multi-language or
>> single-language.
>>
>> So I'm not familiar with the project system trying to guess a project on
>> the fly, as the current code does.
>
> I'm not discussing a particular project backend here, just the API.
> And you can use the user-selected backend everywhere, if there's an
> appropriate element at the head of project-find-functions.
WRT EDE as an existing model to consider ...
EDE will detect projects based on a key file, or you can ask it to load
a project file with the ede command. It defines a base implementation
of what a project is for handling menus and keybindings, and depends on
different project backends to implement features used by the menus and
keybindings.
EDE does not have a list of 'find' type functions. Instead, each
project type declares a small structure with functions for identifying
the dominating file, which classes to implement, and for some complex
cases, how to identify where projects are based on the contents of some
config file. For example, Arduino has a config file that says where
your sketches are. I used the structures because the allow project
detection without autoloading the supporting back-end until that project
type is actually used.
> If I recall, you've mentioned a concept of project file listing
> different search-paths for different languages. Without
> project-search-path taking the language into account, you'll have to
> append all paths together.
EDE was not originally intended to be a way to search for files, but
this feature was needed for semantic to find headers and referenced
files using short names. It does this with 'ede-expand-filename'. For
projects that include an exhaustive list of files as part of the
configuration (such as Automake) searches just go through the known
files using major-mode to filter if needed.
Other projects that do not have all the files ready, such as EDE's
android backend instead implement a method for `ede-expand-filename'
that indicates where to start searching based on simple patterns, such
as the java code in src, and resource xml files in res. Using methods
is important so backend authors can do checking before blindly creating
path names.
For particularly large projects with a lot of source files, you can bind
in an external tools such as Global or locate to help you find files
when the basics don't work out.
> I've only listed the implementation difficulties. If there are any
> particular problems with the concept of dispatching on the language,
> I'd like to hear about those.
EDE had to manage the concept of derived modes via the mode-local tool
in order to handle cases where C++ was like C with a few more details.
If you intend to dispatch behavior based on mode, I strongly recommend
mode local as a way to make it declarative and traceable.
>
>> I don't see any implementations of ede-source-paths, and only one use of
>> it, so it's hard to say what the purpose of the mode parameter is.
>
> That's true, unfortunately. The only implementations I could find are
> in malabar-mode (another third-party project attempting to provide a
> Java editing environment).
>
> Eric, could you tell if we're missing anything? Are there other places
> where ede-source-paths is defined or used?
ede-source-paths is implemented for several java flavors, such as
ant/maven and android. It is then used by semantic's javap database
backend, and SRecode's template based code generator.
It works well for java because class names and path names line up
nicely. It hasn't been used for anything else, even though the comment
mentions C++.
I suspect what your source-paths concept better matches EDE's targets
concept. EDE breaks a project into targets, matching a Makefile style
target if that metadata is available. Each target represents some set
of source files, usually all of the same mode.
EDE also tracks include paths, which are locations where you might
include code from some other location, matching the C/C++ concept. It
works ok for some other languages, but primarily it is for that. Java
has it's own classpath notion to better match what it does.
> Any reason why symref isn't using it?
Symref style searching, which I assume is like your xref concept,
doesn't line up with the various paths in EDE, as EDE doesn't really
need those types of paths for anything. Even the semantic tool, which
used to deal with lists of project roots dropped the concept in favor of
EDE's project detection as it was superior for all use cases.
Every EDE project marks the top directory of a source tree, with
.ede-ignore enabling breaks for nested projects. Using that top path,
you can use find/grep, GNU Global, and various other tools to do your
searching for you. The nice thing when you start using Global or CScope
is that your config for the tool in question does the mode type
filtering for you.
My experience with EDE and other CEDET pieces is that I need to do
symref type searches quite rarely. Usually the various tag based
searches do a fine job getting me the data I need, the exception being
finding uses of some function. In that case, symref, along with Global
or cscope is nice because it does a good job organizing the results so
it is easy to find what you want. I had intended symref to be where
various refactoring tools would be based, as it allows you to select
various hits, and perform operations on only selected hits. I stopped
developing code as my job before I got to those features.
Eric
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-02 2:29 ` Eric Ludlam
@ 2015-08-02 8:57 ` Nix
2015-08-02 17:14 ` Michael Heerdegen
2015-08-02 23:07 ` Dmitry Gutov
2015-08-03 1:21 ` Dmitry Gutov
1 sibling, 2 replies; 87+ messages in thread
From: Nix @ 2015-08-02 8:57 UTC (permalink / raw)
To: Eric Ludlam; +Cc: michael_heerdegen, emacs-devel, Stephen Leake, Dmitry Gutov
On 2 Aug 2015, Eric Ludlam stated:
> Every EDE project marks the top directory of a source tree, with
> .ede-ignore enabling breaks for nested projects. Using that top path,
> you can use find/grep, GNU Global, and various other tools to do your
> searching for you. The nice thing when you start using Global or
> CScope is that your config for the tool in question does the mode type
> filtering for you.
Allow me to interject here that this feature had me applauding when I
discovered it. It's an example of helpful automagic: if you have a large
project, you're going to want a tagging system anyway: having Semantic
automatically exploit it as soon as it appears is just gravy.
Speaking personally, I don't need an extra project system: as far as I'm
concerned Emacs already has one. I wrote one, as everyone does, and then
threw it away because I never use it. EDE and GNU GLOBAL are sufficient,
once you figure out how to use them: they know what files are part of
the project, how to build it, how to find things in it with a degree of
context-sensitivity that xref can only dream of (though its
multi-backend feature means that presumably it can learn to *use*
Semantic to provide context-sensitive search), and can handle things
like projects with handwritten Makefiles that don't match its automagic
project-root- searching features easily:
(ede-cpp-root-project "DTrace-CTF" :file "/usr/src/libdtrace-ctf/Makerules"
:include-path '("/include" "/build")
:system-include-path '("/usr/include")
:spp-table '(("_LITTLE_ENDIAN" . "") ("_GNU_SOURCE" . "")))
That, combined with a GTAGS file, gives me most of EDE's features, and
all the features proposed for the project system. For projects like
Emacs or the Linux kernel or anything using Automake I don't even need
that much. EDE is an underappreciated jewel. Most of this project
discussion and the invasive code piling into modes all across Emacs
feels to me like an attempt to reimplement part of EDE, but without
things like the grammar-sensitive searching that makes EDE so capable.
(What I'd really like is something like the el-search pcase-based
searcher Michael Heerdegen recently proposed, only for everything
supported by Semantic, but with most language grammars not being very
amenable to ML-style pattern-matching I doubt this is possible...)
--
NULL && (void)
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-02 8:57 ` Nix
@ 2015-08-02 17:14 ` Michael Heerdegen
2015-08-02 23:09 ` Eric Ludlam
2015-08-02 23:07 ` Dmitry Gutov
1 sibling, 1 reply; 87+ messages in thread
From: Michael Heerdegen @ 2015-08-02 17:14 UTC (permalink / raw)
To: Nix; +Cc: Eric Ludlam, emacs-devel, Stephen Leake, Dmitry Gutov
Nix <nix@esperi.org.uk> writes:
> (What I'd really like is something like the el-search pcase-based
> searcher Michael Heerdegen recently proposed, only for everything
> supported by Semantic, but with most language grammars not being very
> amenable to ML-style pattern-matching I doubt this is possible...)
I agree that would be very useful. But I don't know much about
Semantic, and the info pages seem to be outdated.
Is there some function in semantic that parses the expression at point
and returns a parse tree represented as (Lisp) list? The outdated info
page mentions ast.el which seems to do that, but I can't find it
anymore.
I guess one could try the following, more or less: go through the buffer
and parse it on the fly to ast expressions. The search command prompts
for a "pattern" using the syntax of the buffer's language plus
placeholders. The input is parsed and converted to a an ast as well.
Then the matching could be done in Elisp like el-search does.
Michael.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-02 8:57 ` Nix
2015-08-02 17:14 ` Michael Heerdegen
@ 2015-08-02 23:07 ` Dmitry Gutov
2015-08-03 10:24 ` Nix
1 sibling, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-02 23:07 UTC (permalink / raw)
To: Nix, Eric Ludlam; +Cc: michael_heerdegen, Stephen Leake, emacs-devel
On 08/02/2015 11:57 AM, Nix wrote:
> Allow me to interject here that this feature had me applauding when I
> discovered it. It's an example of helpful automagic: if you have a large
> project, you're going to want a tagging system anyway: having Semantic
> automatically exploit it as soon as it appears is just gravy.
If EDE works for you, that's great. You should continue using it.
> Speaking personally, I don't need an extra project system: as far as I'm
> concerned Emacs already has one.
The point is other tools can use some information about your current
project without being aware that you're using EDE. Or something else.
> the project, how to build it, how to find things in it with a degree of
> context-sensitivity that xref can only dream of (though its
> multi-backend feature means that presumably it can learn to *use*
> Semantic to provide context-sensitive search)
Indeed, it can. *And* it's also useful for languages which Semantic
doesn't support.
> Most of this project
> discussion and the invasive code piling into modes all across Emacs
> feels to me like an attempt to reimplement part of EDE, but without
> things like the grammar-sensitive searching that makes EDE so capable.
What code piling?
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-02 17:14 ` Michael Heerdegen
@ 2015-08-02 23:09 ` Eric Ludlam
2015-08-02 23:39 ` Michael Heerdegen
0 siblings, 1 reply; 87+ messages in thread
From: Eric Ludlam @ 2015-08-02 23:09 UTC (permalink / raw)
To: Michael Heerdegen, Nix; +Cc: emacs-devel, Stephen Leake, Dmitry Gutov
On 08/02/2015 01:14 PM, Michael Heerdegen wrote:
> Nix <nix@esperi.org.uk> writes:
>
>> (What I'd really like is something like the el-search pcase-based
>> searcher Michael Heerdegen recently proposed, only for everything
>> supported by Semantic, but with most language grammars not being very
>> amenable to ML-style pattern-matching I doubt this is possible...)
> I agree that would be very useful. But I don't know much about
> Semantic, and the info pages seem to be outdated.
Most of the doc is about using some of the user facing tools. The
sections about the lex stage should be good.
> Is there some function in semantic that parses the expression at point
> and returns a parse tree represented as (Lisp) list? The outdated info
> page mentions ast.el which seems to do that, but I can't find it
> anymore.
>
> I guess one could try the following, more or less: go through the buffer
> and parse it on the fly to ast expressions. The search command prompts
> for a "pattern" using the syntax of the buffer's language plus
> placeholders. The input is parsed and converted to a an ast as well.
> Then the matching could be done in Elisp like el-search does.
You can use the lexer from semantic to get the list of parsed tokens.
The current parsers output tags instead of an AST. Making an AST would
be handy for langauge sensitive tooling past tagging parsers, but would
be a bit slow. Large parts of the buffer aren't parsed at all right now
as the parser skips over large { bracketed } blocks of code which keeps
thing faster.
It may be that the specifics you are looking for are part of the local
context parser, which includes functions such as -get-local-variables,
-get-local-arguments, and -current-symbol, which gets the whole symbols
in a dotted expression. This is in semantic/ctxt.el.
Eric
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-02 23:09 ` Eric Ludlam
@ 2015-08-02 23:39 ` Michael Heerdegen
2015-08-03 11:33 ` Eric Ludlam
0 siblings, 1 reply; 87+ messages in thread
From: Michael Heerdegen @ 2015-08-02 23:39 UTC (permalink / raw)
To: Eric Ludlam; +Cc: Nix, emacs-devel, Stephen Leake, Dmitry Gutov
Eric Ludlam <ericludlam@gmail.com> writes:
> It may be that the specifics you are looking for are part of the local
> context parser, which includes functions such as -get-local-variables,
> -get-local-arguments, and -current-symbol, which gets the whole
> symbols in a dotted expression. This is in semantic/ctxt.el.
Thanks for the explanations.
I gave these functions a quick try, evaluating them via M-:
(the-function), with not much luck. In Elisp, all calls always returned
nil. In a C file, I even got an error:
Debugger entered--Lisp error: (wrong-type-argument syntax-table-p nil)
set-syntax-table(nil)
(progn (set-syntax-table semantic-lex-syntax-table) (save-excursion (cond ((looking-at "\\w\\|\\s_") (forward-sexp 1)) ((looking-at fieldsep1) nil) ((save-excursion (and (condition-case nil (progn ... ... t) (error nil)) (looking-at fieldsep1))) (setq symlist (list "")) (forward-sexp -1) (while (looking-at "\\s(") (forward-sexp -1)) (forward-sexp 1))) (setq end (point)) (condition-case nil (while (save-excursion (forward-char -1) (looking-at "\\w\\|\\s_")) (forward-sexp -1) (setq symlist (cons (buffer-substring-no-properties (point) end) symlist)) (let ((cp (point))) (forward-sexp -1) (forward-sexp 1) (if (<= cp (point)) (error nil))) (if (looking-at fieldsep) (progn (forward-sexp -1) (while (and ... ...) (forward-sexp -1)) (forward-sexp 1) (setq end (point))) (error nil))) (error nil)))
symlist)
(unwind-protect (progn (set-syntax-table semantic-lex-syntax-table) (save-excursion (cond ((looking-at "\\w\\|\\s_") (forward-sexp 1)) ((looking-at fieldsep1) nil) ((save-excursion (and (condition-case nil ... ...) (looking-at fieldsep1))) (setq symlist (list "")) (forward-sexp -1) (while (looking-at "\\s(") (forward-sexp -1)) (forward-sexp 1))) (setq end (point)) (condition-case nil (while (save-excursion (forward-char -1) (looking-at "\\w\\|\\s_")) (forward-sexp -1) (setq symlist (cons (buffer-substring-no-properties ... end) symlist)) (let ((cp ...)) (forward-sexp -1) (forward-sexp 1) (if (<= cp ...) (error nil))) (if (looking-at fieldsep) (progn (forward-sexp -1) (while ... ...) (forward-sexp 1) (setq end ...)) (error nil))) (error nil))) symlist) (save-current-buffer (set-buffer #:b
uffer) (set-syntax-table #:table)))
(let ((#2=#:table (syntax-table)) (#1=#:buffer (current-buffer))) (unwind-protect (progn (set-syntax-table semantic-lex-syntax-table) (save-excursion (cond ((looking-at "\\w\\|\\s_") (forward-sexp 1)) ((looking-at fieldsep1) nil) ((save-excursion (and ... ...)) (setq symlist (list "")) (forward-sexp -1) (while (looking-at "\\s(") (forward-sexp -1)) (forward-sexp 1))) (setq end (point)) (condition-case nil (while (save-excursion (forward-char -1) (looking-at "\\w\\|\\s_")) (forward-sexp -1) (setq symlist (cons ... symlist)) (let (...) (forward-sexp -1) (forward-sexp 1) (if ... ...)) (if (looking-at fieldsep) (progn ... ... ... ...) (error nil))) (error nil))) symlist) (save-current-buffer (set-buffer #1#) (set-syntax-table #2#))))
(with-syntax-table semantic-lex-syntax-table (save-excursion (cond ((looking-at "\\w\\|\\s_") (forward-sexp 1)) ((looking-at fieldsep1) nil) ((save-excursion (and (condition-case nil (progn ... ... t) (error nil)) (looking-at fieldsep1))) (setq symlist (list "")) (forward-sexp -1) (while (looking-at "\\s(") (forward-sexp -1)) (forward-sexp 1))) (setq end (point)) (condition-case nil (while (save-excursion (forward-char -1) (looking-at "\\w\\|\\s_")) (forward-sexp -1) (setq symlist (cons (buffer-substring-no-properties (point) end) symlist)) (let ((cp (point))) (forward-sexp -1) (forward-sexp 1) (if (<= cp (point)) (error nil))) (if (looking-at fieldsep) (progn (forward-sexp -1) (while (and ... ...) (forward-sexp -1)) (forward-sexp 1) (setq end (point))) (error nil))) (error nil))) symlis
t)
(let* ((fieldsep1 (mapconcat (lambda (a) (regexp-quote a)) semantic-type-relation-separator-character "\\|")) (fieldsep (concat "[ \n.]*\\(" fieldsep1 "\\)[ \n.]*\\(\\w\\|\\s_\\)")) (case-fold-search semantic-case-fold) (symlist nil) end) (with-syntax-table semantic-lex-syntax-table (save-excursion (cond ((looking-at "\\w\\|\\s_") (forward-sexp 1)) ((looking-at fieldsep1) nil) ((save-excursion (and (condition-case nil ... ...) (looking-at fieldsep1))) (setq symlist (list "")) (forward-sexp -1) (while (looking-at "\\s(") (forward-sexp -1)) (forward-sexp 1))) (setq end (point)) (condition-case nil (while (save-excursion (forward-char -1) (looking-at "\\w\\|\\s_")) (forward-sexp -1) (setq symlist (cons (buffer-substring-no-properties ... end) symlist)) (let ((cp ...)) (forward-sexp -1) (f
orward-sexp 1) (if (<= cp ...) (error nil))) (if (looking-at fieldsep) (progn (forward-sexp -1) (while ... ...) (forward-sexp 1) (setq end ...)) (error nil))) (error nil))) symlist))
(save-excursion (if point (goto-char point)) (let* ((fieldsep1 (mapconcat (lambda (a) (regexp-quote a)) semantic-type-relation-separator-character "\\|")) (fieldsep (concat "[ \n.]*\\(" fieldsep1 "\\)[ \n.]*\\(\\w\\|\\s_\\)")) (case-fold-search semantic-case-fold) (symlist nil) end) (with-syntax-table semantic-lex-syntax-table (save-excursion (cond ((looking-at "\\w\\|\\s_") (forward-sexp 1)) ((looking-at fieldsep1) nil) ((save-excursion (and ... ...)) (setq symlist (list "")) (forward-sexp -1) (while (looking-at "\\s(") (forward-sexp -1)) (forward-sexp 1))) (setq end (point)) (condition-case nil (while (save-excursion (forward-char -1) (looking-at "\\w\\|\\s_")) (forward-sexp -1) (setq symlist (cons ... symlist)) (let (...) (forward-sexp -1) (forward-sexp 1) (if ... ...)) (if (looking
-at fieldsep) (progn ... ... ... ...) (error nil))) (error nil))) symlist)))
semantic-ctxt-current-symbol-default()
...
because semantic-lex-syntax-table is bound to nil which
`set-syntax-table' doesn't like.
Michael.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-02 2:29 ` Eric Ludlam
2015-08-02 8:57 ` Nix
@ 2015-08-03 1:21 ` Dmitry Gutov
1 sibling, 0 replies; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-03 1:21 UTC (permalink / raw)
To: Eric Ludlam, Stephen Leake, emacs-devel; +Cc: Eric Ludlam
Hi Erc,
On 08/02/2015 05:29 AM, Eric Ludlam wrote:
> EDE does not have a list of 'find' type functions. Instead, each
> project type declares a small structure with functions for identifying
> the dominating file, which classes to implement, and for some complex
"which classes to implement" - what does that mean? Is this something
interesting, in the context of this discussion?
> EDE was not originally intended to be a way to search for files, but
> this feature was needed for semantic to find headers and referenced
> files using short names.
I think it's quite natural for a project to be able to list the
directories where related files could be found. For instance, so that
someone could implement a find-file-in-project command. Or search across
those files.
> It does this with 'ede-expand-filename'. For
> projects that include an exhaustive list of files as part of the
> configuration (such as Automake) searches just go through the known
> files using major-mode to filter if needed.
This seems useful enough, although maybe I'd untie the expand-filename
logic from a given project implementation and make it dependent on the
major mode. For instance, no matter if it's a common Java project or an
Android project, whether it's based on Ant, or Maven, or Gradle, the
logic for finding the file based on the class name is the same: you just
need the list of directories to look in.
Which project-search-path might correspond to.
> For particularly large projects with a lot of source files, you can bind
> in an external tools such as Global or locate to help you find files
> when the basics don't work out.
Global seems fine to search inside the project (for certain languages,
at least), but what about the system libraries? Headers, etc. If they're
used in the current project, being able to search in them would be
useful, too.
> EDE had to manage the concept of derived modes via the mode-local tool
> in order to handle cases where C++ was like C with a few more details.
> If you intend to dispatch behavior based on mode, I strongly recommend
> mode local as a way to make it declarative and traceable.
That's interesting, but it doesn't sidestep the issue of different users
possibly using alternative major modes for the same files.
So far as I see it, in most cases the project implementation could
easily write something ad-hoc like that that would be good enough. IOW,
I'm more concerned about the basics right now.
> ede-source-paths is implemented for several java flavors, such as
> ant/maven and android. It is then used by semantic's javap database
> backend, and SRecode's template based code generator.
I see, thank you. I've only been looking in CEDET inside Emacs.
> It works well for java because class names and path names line up
> nicely. It hasn't been used for anything else, even though the comment
> mentions C++.
In Ruby, class and module names also often map to file paths. Only by
convention, though, so there's some variability there. The same happens,
to certain degrees, in other languages too.
> I suspect what your source-paths concept better matches EDE's targets
> concept. EDE breaks a project into targets, matching a Makefile style
> target if that metadata is available. Each target represents some set
> of source files, usually all of the same mode.
Maybe? I'm not sure how the concept of targets maps to dynamic
languages, for example.
> EDE also tracks include paths, which are locations where you might
> include code from some other location, matching the C/C++ concept. It
> works ok for some other languages, but primarily it is for that. Java
> has it's own classpath notion to better match what it does.
Indeed. project-search-path, as it's currently imagined, is a merge of
ede-source-paths and ede-system-include-path.
So xref-find-regexp and xref-find-references will, by default, search
the includes path too. Something that symref doesn't do.
>> Any reason why symref isn't using it?
>
> Symref style searching, which I assume is like your xref concept,
> doesn't line up with the various paths in EDE, as EDE doesn't really
> need those types of paths for anything. Even the semantic tool, which
> used to deal with lists of project roots dropped the concept in favor of
> EDE's project detection as it was superior for all use cases.
Yes, we also try to reuse semantic-symref-find-references-by-name in the
default implementation of xref-find-references.
The two main drawbacks now are: it doesn't handle the project's ignored
directories (which should be relevant for symref/grep.el), and
semantic-symref-filepattern-alist needs to have correct entries for
every language.
> Every EDE project marks the top directory of a source tree, with
> .ede-ignore enabling breaks for nested projects. Using that top path,
> you can use find/grep, GNU Global, and various other tools to do your
> searching for you. The nice thing when you start using Global or CScope
> is that your config for the tool in question does the mode type
> filtering for you.
I suppose. As long as the tool supports all kinds of operators (search
for references, regexp, listing files), and supports your target
language. Though it probably won't help with related code outside of
your project's tree anyway.
Maybe xref-find-references should also always search all project-roots
anyway (while filtering by file names and ignoring non-interesting
directories), and project-search-path should focus on "source roots"
(inside and outside of the project tree) exclusively. So even if the
project has a build file written in the same language as the rest of it
(and it lies in the root directory), project-search-path still would
only include, say, project/lib and project/app.
The main use case for those entries, then, would be locating the file
for a given class name. Which could serve as a basis for a naive
find-definition implementation.
> My experience with EDE and other CEDET pieces is that I need to do
> symref type searches quite rarely. Usually the various tag based
> searches do a fine job getting me the data I need, the exception being
> finding uses of some function.
"Finding uses" is an operation I have to do quite frequently.
>I had intended symref to be where
> various refactoring tools would be based, as it allows you to select
> various hits, and perform operations on only selected hits.
Yup, support for renaming is something I'm interested in as well. It's
generally accepted that Emacs's refactoring support is lacking/nonexistent.
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-02 23:07 ` Dmitry Gutov
@ 2015-08-03 10:24 ` Nix
2015-08-03 10:35 ` Dmitry Gutov
0 siblings, 1 reply; 87+ messages in thread
From: Nix @ 2015-08-03 10:24 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: michael_heerdegen, Eric Ludlam, Stephen Leake, emacs-devel
On 3 Aug 2015, Dmitry Gutov spake thusly:
> On 08/02/2015 11:57 AM, Nix wrote:
>> Most of this project
>> discussion and the invasive code piling into modes all across Emacs
>> feels to me like an attempt to reimplement part of EDE, but without
>> things like the grammar-sensitive searching that makes EDE so capable.
>
> What code piling?
There seem to be rather a lot of references to project-* landing in
unrelated files (language modes, etc). Or so it seems to me, anyway.
(But it's much less invasive than, say, the curved-quotes stuff, and
much more useful ;) )
--
NULL && (void)
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-03 10:24 ` Nix
@ 2015-08-03 10:35 ` Dmitry Gutov
2015-08-07 15:25 ` Nix
0 siblings, 1 reply; 87+ messages in thread
From: Dmitry Gutov @ 2015-08-03 10:35 UTC (permalink / raw)
To: Nix; +Cc: michael_heerdegen, Eric Ludlam, Stephen Leake, emacs-devel
On 08/03/2015 01:24 PM, Nix wrote:
> There seem to be rather a lot of references to project-* landing in
> unrelated files (language modes, etc). Or so it seems to me, anyway.
^
└ citation needed
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-02 23:39 ` Michael Heerdegen
@ 2015-08-03 11:33 ` Eric Ludlam
0 siblings, 0 replies; 87+ messages in thread
From: Eric Ludlam @ 2015-08-03 11:33 UTC (permalink / raw)
To: Michael Heerdegen; +Cc: Nix, emacs-devel, Stephen Leake, Dmitry Gutov
On 08/02/2015 07:39 PM, Michael Heerdegen wrote:
> Eric Ludlam <ericludlam@gmail.com> writes:
>
>> It may be that the specifics you are looking for are part of the local
>> context parser, which includes functions such as -get-local-variables,
>> -get-local-arguments, and -current-symbol, which gets the whole
>> symbols in a dotted expression. This is in semantic/ctxt.el.
> Thanks for the explanations.
>
> I gave these functions a quick try, evaluating them via M-:
> (the-function), with not much luck. In Elisp, all calls always returned
> nil. In a C file, I even got an error:
>
> Debugger entered--Lisp error: (wrong-type-argument syntax-table-p nil)
> set-syntax-table(nil)
> (progn (set-syntax-table semantic-lex-syntax-table) (save-excursion (cond ((looking-at "\\w\\|\\s_")
> because semantic-lex-syntax-table is bound to nil which
> `set-syntax-table' doesn't like.
>
Everything in the semantic system needs `semantic-mode' enabled. That
sets up all the parsing context needed for the various tools, and keeps
it up to date. There is a lot of little features needed to handle
parsing, so the mode enables the hooks to get it setup as you load
files, and then the tagging parser parsers you buffers, which the
context parser uses to identify the outer bounds of different contexts.
Once you do that, you could try 'semantic-analyze-current-context' which
will give you some high level data about where point is. The local
-ctxt- functions can be used to do something unrelated smart completion.
Eric
^ permalink raw reply [flat|nested] 87+ messages in thread
* Re: Per-language project-search-path, was: Re: Unified project interface
2015-08-03 10:35 ` Dmitry Gutov
@ 2015-08-07 15:25 ` Nix
0 siblings, 0 replies; 87+ messages in thread
From: Nix @ 2015-08-07 15:25 UTC (permalink / raw)
To: Dmitry Gutov; +Cc: michael_heerdegen, Eric Ludlam, Stephen Leake, emacs-devel
On 3 Aug 2015, Dmitry Gutov verbalised:
> On 08/03/2015 01:24 PM, Nix wrote:
>
>> There seem to be rather a lot of references to project-* landing in
>> unrelated files (language modes, etc). Or so it seems to me, anyway.
> ^
> └ citation needed
None available because it was predicated on a bunch of misunderstandings
of how tightly things like xref tie into it. :) sorry!
--
NULL && (void)
^ permalink raw reply [flat|nested] 87+ messages in thread
end of thread, other threads:[~2015-08-07 15:25 UTC | newest]
Thread overview: 87+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-06-04 11:43 Unified project interface Dmitry Gutov
2015-06-04 14:40 ` Stephen Leake
2015-06-05 0:08 ` Dmitry Gutov
2015-06-05 10:08 ` Stephen Leake
2015-06-05 13:03 ` Stephen Leake
2015-06-05 13:14 ` Dmitry Gutov
2015-06-08 1:24 ` Stephen Leake
2015-06-09 18:16 ` Dmitry Gutov
2015-06-09 18:21 ` Eli Zaretskii
2015-06-09 18:49 ` Dmitry Gutov
2015-06-09 19:03 ` Eli Zaretskii
2015-06-07 23:22 ` Dmitry Gutov
2015-06-08 1:35 ` [SPAM UNSURE] " Stephen Leake
2015-06-09 19:04 ` Dmitry Gutov
2015-06-07 23:15 ` Dmitry Gutov
2015-06-08 1:59 ` Stephen Leake
2015-06-09 22:31 ` Dmitry Gutov
2015-06-10 7:13 ` Steinar Bang
2015-07-08 0:25 ` Dmitry Gutov
2015-07-11 13:43 ` Bozhidar Batsov
2015-07-11 14:17 ` Dmitry Gutov
2015-07-12 14:42 ` Dmitry Gutov
2015-07-13 8:49 ` Bozhidar Batsov
2015-07-13 10:23 ` Dmitry Gutov
2015-07-24 23:43 ` Dmitry Gutov
2015-07-25 0:55 ` Stephen Leake
2015-07-25 7:29 ` Eli Zaretskii
2015-07-26 2:12 ` Dmitry Gutov
2015-07-26 2:45 ` Eli Zaretskii
2015-07-26 11:25 ` Stephen Leake
2015-07-26 2:11 ` Dmitry Gutov
2015-07-26 11:22 ` Stephen Leake
2015-07-26 17:23 ` Dmitry Gutov
2015-07-26 18:57 ` Stephen Leake
2015-07-26 23:56 ` John Yates
2015-07-27 1:49 ` Dmitry Gutov
2015-07-27 11:12 ` Stephen Leake
2015-07-27 11:27 ` Dmitry Gutov
2015-07-27 13:00 ` Dmitry Gutov
2015-07-27 13:02 ` Dmitry Gutov
2015-07-28 1:21 ` Stephen Leake
2015-07-28 11:05 ` Stephen Leake
2015-07-28 14:33 ` Dmitry Gutov
2015-07-28 15:45 ` Stephen Leake
2015-07-28 16:25 ` Dmitry Gutov
2015-07-29 1:36 ` Stephen Leake
2015-07-29 2:10 ` Dmitry Gutov
2015-07-28 14:18 ` Dmitry Gutov
2015-07-28 16:15 ` Stephen Leake
2015-07-28 18:44 ` Dmitry Gutov
2015-07-29 2:27 ` Stephen Leake
2015-07-29 22:51 ` Dmitry Gutov
2015-07-30 8:17 ` Stephen Leake
2015-07-31 0:15 ` Dmitry Gutov
2015-07-31 16:13 ` Stephen Leake
2015-08-01 0:57 ` Dmitry Gutov
2015-08-01 9:50 ` Stephen Leake
2015-08-01 10:51 ` Stephen Leake
2015-08-01 12:42 ` Dmitry Gutov
2015-08-01 12:40 ` Dmitry Gutov
2015-08-01 14:15 ` Stephen Leake
2015-08-01 15:09 ` Dmitry Gutov
2015-08-01 19:04 ` Stephen Leake
2015-08-01 22:33 ` Dmitry Gutov
2015-08-01 1:14 ` Per-language project-search-path, was: " Dmitry Gutov
2015-08-01 10:43 ` Stephen Leake
2015-08-01 14:12 ` Dmitry Gutov
2015-08-01 18:57 ` Stephen Leake
2015-08-02 0:25 ` Dmitry Gutov
2015-08-02 2:29 ` Eric Ludlam
2015-08-02 8:57 ` Nix
2015-08-02 17:14 ` Michael Heerdegen
2015-08-02 23:09 ` Eric Ludlam
2015-08-02 23:39 ` Michael Heerdegen
2015-08-03 11:33 ` Eric Ludlam
2015-08-02 23:07 ` Dmitry Gutov
2015-08-03 10:24 ` Nix
2015-08-03 10:35 ` Dmitry Gutov
2015-08-07 15:25 ` Nix
2015-08-03 1:21 ` Dmitry Gutov
2015-07-29 23:11 ` xref display and multiple locations, " Dmitry Gutov
2015-06-06 10:20 ` Bozhidar Batsov
2015-06-06 10:29 ` Dmitry Gutov
2015-06-06 12:32 ` Eric Ludlam
2015-06-06 18:44 ` Dmitry Gutov
2015-06-06 19:28 ` Eli Zaretskii
2015-06-07 22:29 ` Dmitry Gutov
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.