running EDE from a file that is not under a project root dir

running EDE from a file that is not under a project root dir

[Stephen Leake]

I often have "notes" files that are outside a project directory. For
example, my Emacs notes are not in the emacs/master directory; it is
controlled by git, and I don't want to commit my notes to that, nor
clutter .gitignore.

At the same time, I'd like to be able to invoke "search for symbol at
point" from the notes text buffer; I have notes on functions/vars I'm
using at the moment.

Currently, I'm trying to use xref-find-references on the elisp function
temp-buffer-window-show (defined in window.el).

That delegates to semantic-symref-find-references-by-name, which is
where EDE comes in.

This uses semantic-symref-detect-symref-tool to get the search tool; it
picks grep (meaning grep-find), which is fine.

But then it uses semantic-symref-derive-find-filepatterns to get the
file patterns to pass to grep. Since I'm starting from a text-mode
buffer, that returns (*.letter *.article *.te?xt). Which is _not_ what I
want; I want *.el.

Creating an EDE project would not help, because
semantic-symref-derive-find-filepatterns doesn't use the EDE project,
only semantic-symref-filepattern-alist and auto-mode-alist.


A related use case is searching for C identifiers from a LaTeX
documentation file in a C project; then EDE will find the right root and
project, but still return the wrong file patterns.


My question is this; what is the minimum change I should make to EDE to
support these use cases?

I think I need three things:

- add a "file patterns" element to the EDE project (or target?) data
structure, and have semantic-symref-derive-find-filepatterns check that
before the alists.

- add an ede-global-project variable that I can set to my current
project, to bypass the project root detection.

    I've proposed the same change to project.el; there, it can be used
    by one of the project-find functions.

- some way to set up the EDE project object for elisp, especially the
file patterns.

    The EDE info manual in Emacs master mentions ede-proj-elisp.el in
    section 8 Extending EDE, but I don't see that file in Emacs master
    git.
    
-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Dmitry Gutov]

On 08/04/2015 09:13 PM, Stephen Leake wrote:

> But then it uses semantic-symref-derive-find-filepatterns to get the
> file patterns to pass to grep. Since I'm starting from a text-mode
> buffer, that returns (*.letter *.article *.te?xt). Which is _not_ what I
> want; I want *.el.

If we don't want to change too much code, I suppose 
xref-collect-references could bind semantic-symref-filepattern-alist to 
a list of one element:

`((,major-mode "*"))

Though that will search in all files. To refine it further, indeed we 
could make the choice based on what the project tells us.

> My question is this; what is the minimum change I should make to EDE to
> support these use cases?
>
> I think I need three things:
>
> - add a "file patterns" element to the EDE project (or target?) data
> structure, and have semantic-symref-derive-find-filepatterns check that
> before the alists.

That's cool, but won't help xref-find-references if EDE isn't used. A 
hard dependency on EDE could even break the current usage of 
semantic-symref-find-references-by-name in xref-find-references.

On the other hand, if we introduce project-source-file-patterns, 
xref-find-references could use it in the 
semantic-symref-filepattern-alist binding.

And ede-source-file-patterns could be used by semantic-symref-symbol and 
semantic-symref-regexp.

That issue aside, xref-find-references currently doesn't use 
project-ignores, nor does it use grep-find-ignored-directories and 
grep-find-ignored-files. Thought it still works reasonably quickly as 
long as the "uninteresting" files don't match the file name pattern we 
search for.

> - add an ede-global-project variable that I can set to my current
> project, to bypass the project root detection.
>
>      I've proposed the same change to project.el; there, it can be used
>      by one of the project-find functions.

project.el doesn't need that variable. The variable can just as well 
live in the package you're writing.

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/04/2015 09:13 PM, Stephen Leake wrote:
>
>> But then it uses semantic-symref-derive-find-filepatterns to get the
>> file patterns to pass to grep. Since I'm starting from a text-mode
>> buffer, that returns (*.letter *.article *.te?xt). Which is _not_ what I
>> want; I want *.el.
>
> If we don't want to change too much code, I suppose
> xref-collect-references could bind semantic-symref-filepattern-alist
> to a list of one element:
>
> `((,major-mode "*"))
>
> Though that will search in all files. To refine it further, indeed we
> could make the choice based on what the project tells us.
>
>> My question is this; what is the minimum change I should make to EDE to
>> support these use cases?
>>
>> I think I need three things:
>>
>> - add a "file patterns" element to the EDE project (or target?) data
>> structure, and have semantic-symref-derive-find-filepatterns check that
>> before the alists.
>
> That's cool, but won't help xref-find-references if EDE isn't used. 

Right; I'm exploring using an EDE elisp project via the project.el API.

I'm also implementing a separate elisp project that supports only the
project.el API.

> On the other hand, if we introduce project-source-file-patterns,
> xref-find-references could use it in the
> semantic-symref-filepattern-alist binding.

Yes, I think that would be appropriate.

> And ede-source-file-patterns 

That's not a currently defined function. Did you mean
project-source-file-patterns? Or are you agreeing with me that EDE needs
to add this?

> could be used by semantic-symref-symbol and semantic-symref-regexp.
>
> That issue aside, xref-find-references currently doesn't use
> project-ignores, 

Right; it should.

> nor does it use grep-find-ignored-directories and
> grep-find-ignored-files. 

It should not use these directly; only via project-ignores.

>> - add an ede-global-project variable that I can set to my current
>> project, to bypass the project root detection.
>>
>>      I've proposed the same change to project.el; there, it can be used
>>      by one of the project-find functions.
>
> project.el doesn't need that variable. The variable can just as well
> live in the package you're writing.

I don't understand. Surely the concept "a project that is active in all
buffers" is independent of the actual project type?

I will want it for elisp, Ada, Java, ...

If it's not in project.el, I'll have to add it to every new project type
I write; that will be a name conflict, at the very least.

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Dmitry Gutov]

On 08/05/2015 10:48 AM, Stephen Leake wrote:

>> And ede-source-file-patterns
>
> That's not a currently defined function. Did you mean
> project-source-file-patterns? Or are you agreeing with me that EDE needs
> to add this?

Needs to add.

>> nor does it use grep-find-ignored-directories and
>> grep-find-ignored-files.
>
> It should not use these directly; only via project-ignores.

As long as we're reusing semantic-symref-find-references-by-name, we 
need some way to make it use the ignores. Probably not by making it 
aware of project.el.

> I don't understand. Surely the concept "a project that is active in all
> buffers" is independent of the actual project type?

That's a trivial concept and piece of code. There's no reason to insist 
on reusing it.

> If it's not in project.el, I'll have to add it to every new project type
> I write; that will be a name conflict, at the very least.

Not at all. The variable should be named according to each particular 
package it's in.

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/05/2015 10:48 AM, Stephen Leake wrote:
>
>>> nor does it use grep-find-ignored-directories and
>>> grep-find-ignored-files.
>>
>> It should not use these directly; only via project-ignores.
>
> As long as we're reusing semantic-symref-find-references-by-name, we
> need some way to make it use the ignores. Probably not by making it
> aware of project.el.

Why not? that seems to fit perfectly with "provide project information
to other elisp programs"

>> I don't understand. Surely the concept "a project that is active in all
>> buffers" is independent of the actual project type?
>
> That's a trivial concept and piece of code. There's no reason to
> insist on reusing it.

There's a very good reason; there should be _one_ global variable that
stores the _one_ currently active project.

I don't care what backend provides the project; I just care that there
is exactly one.

>> If it's not in project.el, I'll have to add it to every new project type
>> I write; that will be a name conflict, at the very least.
>
> Not at all. The variable should be named according to each particular
> package it's in.

You are completely missing the point of this functionality.

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Dmitry Gutov]

On 08/06/2015 11:01 AM, Stephen Leake wrote:

>> As long as we're reusing semantic-symref-find-references-by-name, we
>> need some way to make it use the ignores. Probably not by making it
>> aware of project.el.
>
> Why not? that seems to fit perfectly with "provide project information
> to other elisp programs"

Sure, if you want to rewrite it. Should probably get Eric's agreement first.

> There's a very good reason; there should be _one_ global variable that
> stores the _one_ currently active project.

You seem to be arguing from some ideological standpoint. There's no 
practical need for this, for you to get the desired behavior, AFAICT.

> You are completely missing the point of this functionality.

How so? As long as your project-find-functions element knows which 
variable to look up, you should be all set.

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/06/2015 11:01 AM, Stephen Leake wrote:
>
>>> As long as we're reusing semantic-symref-find-references-by-name, we
>>> need some way to make it use the ignores. Probably not by making it
>>> aware of project.el.
>>
>> Why not? that seems to fit perfectly with "provide project information
>> to other elisp programs"
>
> Sure, if you want to rewrite it. Should probably get Eric's agreement first.
>
>> There's a very good reason; there should be _one_ global variable that
>> stores the _one_ currently active project.
>
> You seem to be arguing from some ideological standpoint. There's no
> practical need for this, for you to get the desired behavior, AFAICT.
>
>> You are completely missing the point of this functionality.
>
> How so? As long as your project-find-functions element knows which
> variable to look up, you should be all set.

How would it know that?

Suppose I have a gpr project backend for AdaCore gprbuild projects, and
a gradle backend for Android gradle projects.

There are corresponding global variables; gpr-global-project,
gradle-global-project, each with a project-find function.

I set project-find-functions to '(gpr-find-project gradle-find-project).

I first open a gpr project, which sets gpr-global-project, then later
open a gradle project, which sets gradle-global-project.

Next time (project-current) is  called, it returns gpr-global-project,
which is wrong.

I'm proposing to fix this by having one global variable with one
find-function.

You seem to have some other solution in mind; what would it be?

Hmm. Perhaps the "select project" function could change
project-find-functions as well, using delq and add-to-list. But that
messes with the order of the list, which the user may have carefully
set. For example, I currently have it set to '(project-try-ede
project-find-prj-root project-explicit-prj).

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Dmitry Gutov]

On 08/06/2015 05:39 PM, Stephen Leake wrote:

>> How so? As long as your project-find-functions element knows which
>> variable to look up, you should be all set.
>
> How would it know that?
>
> Suppose I have a gpr project backend for AdaCore gprbuild projects, and
> a gradle backend for Android gradle projects.

I was assuming your project backends would share a codebase. But if they 
don't, they'll need to depend on a third package (which can be very small).

> There are corresponding global variables; gpr-global-project,
> gradle-global-project, each with a project-find function.

The third package would contain: variable global-project-current, 
function global-project-visit, function global-project-try-it.

global-project-set will be called from gpr and gradle upon visiting a 
project, and will set global-project-current to the project passed in to 
it. global-project-try-it will be added to project-find-functions 
(probably as soon as global-project is loaded), and will use 
global-project-current. The variable could as well be made private.

> I'm proposing to fix this by having one global variable with one
> find-function.

So I take it you're asking the above global-project to be moved to the 
core. I'm hesitant, but it's better than what I originally understood 
your request to be (add the variable, and make it higher priority than 
project-find-functions).

I wonder if the concept of the global project will be very popular.

> You seem to have some other solution in mind; what would it be?
>
> Hmm. Perhaps the "select project" function could change
> project-find-functions as well, using delq and add-to-list. But that
> messes with the order of the list, which the user may have carefully
> set. For example, I currently have it set to '(project-try-ede
> project-find-prj-root project-explicit-prj).

If you're willing to sacrifice the "order of the list carefully set by 
the user", the delq + add-to-list (or add-hook; this should be carefully 
considered) solution can be pretty elegant. It's not what I had in mind, 
but I like it.

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/06/2015 05:39 PM, Stephen Leake wrote:
>
>>> How so? As long as your project-find-functions element knows which
>>> variable to look up, you should be all set.
>>
>> How would it know that?
>>
>> Suppose I have a gpr project backend for AdaCore gprbuild projects, and
>> a gradle backend for Android gradle projects.
>
> I was assuming your project backends would share a codebase. But if
> they don't, they'll need to depend on a third package (which can be
> very small).

Right. That "third package" is project.el. What's wrong with that?

>> There are corresponding global variables; gpr-global-project,
>> gradle-global-project, each with a project-find function.
>
> The third package would contain: variable global-project-current,
> function global-project-visit, function global-project-try-it.

How is that different from declaring these in project.el?

>> I'm proposing to fix this by having one global variable with one
>> find-function.
>
> So I take it you're asking the above global-project to be moved to the
> core. 

Right.

> I'm hesitant, but it's better than what I originally understood
> your request to be (add the variable, and make it higher priority than
> project-find-functions).

It has the same priority as anything else in project-find-functions; if
the user wants to give it top priority, they put it first (or only).

> I wonder if the concept of the global project will be very popular.

It's in many current IDEs; Android Studio, Arduino, AdaCore GPS.

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Eric Ludlam]

On 08/04/2015 02:13 PM, Stephen Leake wrote:
> I often have "notes" files that are outside a project directory. For
> example, my Emacs notes are not in the emacs/master directory; it is
> controlled by git, and I don't want to commit my notes to that, nor
> clutter .gitignore.
>
> At the same time, I'd like to be able to invoke "search for symbol at
> point" from the notes text buffer; I have notes on functions/vars I'm
> using at the moment.

...

> My question is this; what is the minimum change I should make to EDE to
> support these use cases?
>
> I think I need three things:
>
> - add a "file patterns" element to the EDE project (or target?) data
> structure, and have semantic-symref-derive-find-filepatterns check that
> before the alists.

I'm not sure what matching file patterns has to do with core project 
functionality.  Your use case sounds like something that would want a 
specialized tool, the same way there are tools that jump between 'doc' 
and 'code'.

For example, if you keep notes in org mode (I assume everyone here uses 
org mode these days) then perhaps what is needed is tooling that knows 
how to keep TODO files in an org file tied to a project and the desired 
language, and then the project would form the root of any symref call.

I suspect symref needs a tweak so that the major mode in question is 
passed along far enough to get into the pattern matcher.  Once that is 
done, a tool that goes from your notes to src could call symref however 
it is needed.

> - add an ede-global-project variable that I can set to my current
> project, to bypass the project root detection.
>
>      I've proposed the same change to project.el; there, it can be used
>      by one of the project-find functions.

I find the global project concept scary.  I can't say how many times 
I've edited Emacs code that was wasn't on my load path because I had 
multiple checkouts of the same code.  Mostly just too many times.

> - some way to set up the EDE project object for elisp, especially the
> file patterns.

I don't think you need to do anything project specific here.  Symref 
doesn't do anything with EDE past getting the root.

>      The EDE info manual in Emacs master mentions ede-proj-elisp.el in
>      section 8 Extending EDE, but I don't see that file in Emacs master
>      git.

That is a piece of the large Make/Automake project system that builds 
makefiles for you.  Symref only uses EDE to identify a starting place 
for the search, or a place to find tooling caches (like GTAGS).  EDE is 
then used by the semantic part that builds up the output buffer by 
parsing tags which needs various include paths depending on language.

Eric

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Eric Ludlam <eric@siege-engine.com> writes:

> On 08/04/2015 02:13 PM, Stephen Leake wrote:
>> I often have "notes" files that are outside a project directory. For
>> example, my Emacs notes are not in the emacs/master directory; it is
>> controlled by git, and I don't want to commit my notes to that, nor
>> clutter .gitignore.
>>
>> At the same time, I'd like to be able to invoke "search for symbol at
>> point" from the notes text buffer; I have notes on functions/vars I'm
>> using at the moment.
>
> ...
>
>> My question is this; what is the minimum change I should make to EDE to
>> support these use cases?
>>
>> I think I need three things:
>>
>> - add a "file patterns" element to the EDE project (or target?) data
>> structure, and have semantic-symref-derive-find-filepatterns check that
>> before the alists.
>
> I'm not sure what matching file patterns has to do with core project
> functionality.  Your use case sounds like something that would want a
> specialized tool, the same way there are tools that jump between 'doc'
> and 'code'.

If I'm debugging the elisp code that implements Emacs Ada mode, I will
often want to search for an elisp symbol while in an Ada mode buffer. So
I'd like an elisp project that is simply global; active in all buffers.

> For example, if you keep notes in org mode (I assume everyone here
> uses org mode these days) 

Not me; plain text suits me just fine.

> then perhaps what is needed is tooling that knows how to keep TODO
> files in an org file tied to a project and the desired language, and
> then the project would form the root of any symref call.

Ok; consider my request one implementation of such tooling. It happens
to work in any buffer, not just org (or LaTeX or markdown or ...).

> I suspect symref needs a tweak so that the major mode in question is
> passed along far enough to get into the pattern matcher.  Once that is
> done, a tool that goes from your notes to src could call symref
> however it is needed.

"the major mode in question" is elisp, here; that's the kind of file
this project is managing (at least, that's my intent). But other
projects could easily manage more than one major mode; I have projects
that contain Ada, C, Makefile, LaTeX, texinfo. I guess the major mode
could be a list.

>> - add an ede-global-project variable that I can set to my current
>> project, to bypass the project root detection.
>>
>>      I've proposed the same change to project.el; there, it can be used
>>      by one of the project-find functions.
>
> I find the global project concept scary.  I can't say how many times
> I've edited Emacs code that was wasn't on my load path because I had
> multiple checkouts of the same code.  Mostly just too many times.

You misunderstand my point. I don't want a project that includes all
files on the disk (which is what I think you are refering to); I want a
single project that is active in all buffers.

Most other IDEs I've used have the notion of a single active project;
the splash screen asks for the project to open.

That's how Emacs Ada mode works; the user selects the single active
project. They can later select another one.

>> - some way to set up the EDE project object for elisp, especially the
>> file patterns.
>
> I don't think you need to do anything project specific here.  Symref
> doesn't do anything with EDE past getting the root.

No, it also gets the file pattern list.

>>      The EDE info manual in Emacs master mentions ede-proj-elisp.el in
>>      section 8 Extending EDE, but I don't see that file in Emacs master
>>      git.
>
> That is a piece of the large Make/Automake project system that builds
> makefiles for you.

I think you are saying the file ede-proj-elisp.el is in some CEDET
repository, but not in Emacs core? I'll go look.

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Eric Ludlam <eric@siege-engine.com> writes:
>
>>>      The EDE info manual in Emacs master mentions ede-proj-elisp.el in
>>>      section 8 Extending EDE, but I don't see that file in Emacs master
>>>      git.
>>
>> That is a piece of the large Make/Automake project system that builds
>> makefiles for you.
>
> I think you are saying the file ede-proj-elisp.el is in some CEDET
> repository, but not in Emacs core? I'll go look.

It's emacs/master/lisp/cedet/ede/proj-elisp.el

Now I just need to figure out how to enable it; I guess (require
'proj-elisp)?

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Wed, 05 Aug 2015 02:20:07 -0500
> 
> It's emacs/master/lisp/cedet/ede/proj-elisp.el
> 
> Now I just need to figure out how to enable it; I guess (require
> 'proj-elisp)?

No, I think you want (require 'ede/proj-elisp).

Re: running EDE from a file that is not under a project root dir

[Eric Ludlam]

On 08/05/2015 02:43 AM, Stephen Leake wrote:
>> >I find the global project concept scary.  I can't say how many times
>> >I've edited Emacs code that was wasn't on my load path because I had
>> >multiple checkouts of the same code.  Mostly just too many times.
> You misunderstand my point. I don't want a project that includes all
> files on the disk (which is what I think you are refering to); I want a
> single project that is active in all buffers.
>
> Most other IDEs I've used have the notion of a single active project;
> the splash screen asks for the project to open.
>
> That's how Emacs Ada mode works; the user selects the single active
> project. They can later select another one.

I have used IDEs with "global" projects, and I consider Emacs an IDE 
with a single global project for "emacs lisp" code (among other things, 
of course. :) It has handy functions like 'find-function' and 
'find-library' that lets you move to definitions and files.

The reason I don't like global projects is that if you are in one 
resource (for Emacs any old .el, .txt, etc) and call `find-function', it 
will jump off to the code, but unless you pay attention, you might end 
up somewhere you don't expect.  For example, if I have two versions of 
my source code open, such as ede.el in Emacs vs ede.el in CEDET's 
repository, and I blindly jump from a symbol, I'll could end up in the 
wrong place based on what my load path is.

In my mind, all files should be deterministically bound to a project so 
when you perform an operation, you always know where you're going.  Some 
files are "in" projects.  In your case, a way to associate a notes file 
outside a project to a project sounds important.

Having a 'global' project could be useful if you only ever edit one 
project at a time in a given emacs instance, and you want everything to 
behave as if it were.  I don't picture most Emacs users operating like 
that.  If it is feature to be added, it should be optional.

Eric

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Eric Ludlam <eric@siege-engine.com> writes:

> On 08/05/2015 02:43 AM, Stephen Leake wrote:
>>> >I find the global project concept scary.  I can't say how many times
>>> >I've edited Emacs code that was wasn't on my load path because I had
>>> >multiple checkouts of the same code.  Mostly just too many times.
>> You misunderstand my point. I don't want a project that includes all
>> files on the disk (which is what I think you are refering to); I want a
>> single project that is active in all buffers.
>>
>> Most other IDEs I've used have the notion of a single active project;
>> the splash screen asks for the project to open.
>>
>> That's how Emacs Ada mode works; the user selects the single active
>> project. They can later select another one.
>
> I have used IDEs with "global" projects, and I consider Emacs an IDE
> with a single global project for "emacs lisp" code (among other
> things, of course. :) It has handy functions like 'find-function' and
> 'find-library' that lets you move to definitions and files.

I use Emacs as an IDE for all kinds of things, not just elisp. I assume
you do as well; that's what EDE does, after all.

> The reason I don't like global projects is that if you are in one
> resource (for Emacs any old .el, .txt, etc) and call `find-function',
> it will jump off to the code, but unless you pay attention, you might
> end up somewhere you don't expect.  For example, if I have two
> versions of my source code open, such as ede.el in Emacs vs ede.el in
> CEDET's repository, and I blindly jump from a symbol, I'll could end
> up in the wrong place based on what my load path is.

I find that an odd definition of "wrong". Since you know 'find-function'
searches load-path, why would you expect to go somewhere that is not on
load-path?

I guess you mean something like "I forget to change load-path when I
switch to thinking about a different project".

That makes sense; Emacs does not provide a menu for a set of load-path
settings.

That's one thing an elisp project backend should do; similar to setting
the source file path for a gcc project. There should be a menu of
possible projects to choose from, so it becomes easy to change
load-path.

Of course, since Emacs doesn't support unloading things, you can't take
this very far; at some point, you have to quit and restart Emacs. But
then the menu can still be there to choose what project to work on.

> In my mind, all files should be deterministically bound to a project
> so when you perform an operation, you always know where you're going.

Many files are in core sub-projects that reused in higher-level
projects; most Emacs code uses window.el and other core files, some
Emacs code uses ELPA libraries; all C code uses the C
standard library, some C code uses many other libraries.

So I don't see how you can definitively determine the search path from
looking at a single file.

In addition, just because I'm in an Ada file doesn't mean I want to
search for other Ada files; sometimes I do, sometimes I don't.

That's _exactly_ why I prefer a single globally active project that I
explicitly select; then I know what the search path is, no matter what
file I'm in.

And when I decide to start thinking about a different project, I tell
Emacs that by selecting that project from the menu. Then Emacs _knows_
what I'm thinking about; it doesn't have to guess.

> Some files are "in" projects. In your case, a way to associate a notes
> file outside a project to a project sounds important.
>
> Having a 'global' project could be useful if you only ever edit one
> project at a time in a given emacs instance, 

Yes.

> and you want everything
> to behave as if it were.  

Were what?

If you mean "Emacs should treat all files on the disk as if they were
part of the current project", then no.

For example, if you are in an Ada file that is not part of the current
project, and invoke 'ada-find-other-file' (to move from body to
specification), you will get an error "file is not in the current
project". That prevents the sort of error you describe above.

But all commands that use project information should get it from the
single current project, yes.

> I don't picture most Emacs users operating
> like that.  

I have no idea who "most Emacs users" are, or how they use it. I do know
a few users, and they use it like I do (mostly because I taught them :).

I have heard complaints that "emacs is not like other IDEs". So when I
see _every_ other IDE providing a single global project, it seems
obvious that Emacs should support that as well.

> If it is feature to be added, it should be optional.

Of course; that is the Emacs Prime Directive :).

There are two levels of "optional" here; `ede-current-project' could
always check the global var first; people who don't want that style just
leave it set to nil.

Or the single global variable could be one of a user-configurable list
of things that ede-current-project checks.

I haven't read the ede code carefully enough yet to say which might be
better.

Hmm. In addition, I would expect the Development | Load Project menu
command to set the global variable, but others won't. So that needs to
be configurable somehow. Ah; if ede-current-project is set to not check
the global variable, it doesn't matter if it gets set. That was easy :).

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[David Engster]

Stephen Leake writes:
> I have heard complaints that "emacs is not like other IDEs". So when I
> see _every_ other IDE providing a single global project, it seems
> obvious that Emacs should support that as well.

The usual way I know is to load some sort of container which contains
any number of projects. In Eclipse it's the 'workspace', in Visual
Studio it's the 'solution'. Working with many projects at the same time
and seamlessly switching between them is what I would consider to be the
norm in other IDEs. Everything else would feel exotic to me.

-David

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

David Engster <deng@randomsample.de> writes:

> Stephen Leake writes:
>> I have heard complaints that "emacs is not like other IDEs". So when I
>> see _every_ other IDE providing a single global project, it seems
>> obvious that Emacs should support that as well.
>
> The usual way I know is to load some sort of container which contains
> any number of projects. In Eclipse it's the 'workspace', in Visual
> Studio it's the 'solution'. Working with many projects at the same time
> and seamlessly switching between them is what I would consider to be the
> norm in other IDEs. Everything else would feel exotic to me.

Ok, it's been a (long) while since I used Eclipse or Visual *; last time
I did, there was only one active project. My recent experience is
Arduino (admittedly a very simple environment by design), Android
Studio, AdaCore GPS.

In Eclipse and Visual Studio with more than one project open, is the
"current project" deduced from the current file? If so how do they
handle the issue of project hierarchy that I raised?

I could see identifying the lowest-level project that contains the file
(ie, the C standard library project contains stdio.h). But that doesn't
give me the full search path, because I don't know which higher level
project the user is working on.

In Emacs Ada mode, the top level project is the current project; the lower
level projects are present as far as gprbuild is concerned, but the
Emacs code doesn't need to know about them. There are still many
top-level Ada projects that share lower-level code, so if more than one
top-level project was open, there would not be a one-to-one mapping from
file to project.

If Eclipse and Visual Studio are like that, so the collection of
projects contains only top level projects, then _if_ they have disjoint
file sets, you can get the project from a file. But that seems unlikely,
and highly use case dependent.

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[David Engster]

Stephen Leake writes:
> David Engster <deng@randomsample.de> writes:
>> Stephen Leake writes:
>>> I have heard complaints that "emacs is not like other IDEs". So when I
>>> see _every_ other IDE providing a single global project, it seems
>>> obvious that Emacs should support that as well.
>>
>> The usual way I know is to load some sort of container which contains
>> any number of projects. In Eclipse it's the 'workspace', in Visual
>> Studio it's the 'solution'. Working with many projects at the same time
>> and seamlessly switching between them is what I would consider to be the
>> norm in other IDEs. Everything else would feel exotic to me.
>
> Ok, it's been a (long) while since I used Eclipse or Visual *; last time
> I did, there was only one active project. My recent experience is
> Arduino (admittedly a very simple environment by design), Android
> Studio, AdaCore GPS.
>
> In Eclipse and Visual Studio with more than one project open, is the
> "current project" deduced from the current file?

Yes.

> If so how do they handle the issue of project hierarchy that I raised?

Things like standard headers are simply not associated with any
project. If you open a file like stdio.h, it will be temporarily
associated with currently active project for things like the 'Build'
button. But that's just an ad-hoc thing. Technically, stdio.h is not
part of any project, and in the 'Project' menu, things like 'Build
Project' are greyed out when you select that file.

> I could see identifying the lowest-level project that contains the file
> (ie, the C standard library project contains stdio.h).

System includes should always be treated specially. They do not change
and are at least in C/C++ a special search path (<string.h> vs
"string.h"). Eclipse determines them automatically by asking the
compiler from the configured toolchain. The system include *paths* can
of course change for a project when I switch configurations and choose a
different toolchain (for instance, when I switch from x86 to
PowerPC). This will also change lots of preprocessor symbols, of course,
and Eclipse parses those automatically as well. Semantic does something
similar, but only for plain gcc, unfortunately.

> But that doesn't give me the full search path, because I don't know
> which higher level project the user is working on.
>
> In Emacs Ada mode, the top level project is the current project; the lower
> level projects are present as far as gprbuild is concerned, but the
> Emacs code doesn't need to know about them. There are still many
> top-level Ada projects that share lower-level code, so if more than one
> top-level project was open, there would not be a one-to-one mapping from
> file to project.
>
> If Eclipse and Visual Studio are like that, so the collection of
> projects contains only top level projects, then _if_ they have disjoint
> file sets, you can get the project from a file. But that seems unlikely,
> and highly use case dependent.

In Eclipse, you simply cannot share files or folders between projects,
at least not with the same path. You either copy them, or you symlink
them from one project into another. In the latter case, you can open the
same file twice within two different projects, but since they are in
different paths, the associated project is clear.

I'm not exactly sure how it works in VS, as I'm currently on paternal
leave and I don't use it at home... :-)

-David

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

David Engster <deng@randomsample.de> writes:

> Stephen Leake writes:
>
>> But that doesn't give me the full search path, because I don't know
>> which higher level project the user is working on.
>>
>> In Emacs Ada mode, the top level project is the current project; the lower
>> level projects are present as far as gprbuild is concerned, but the
>> Emacs code doesn't need to know about them. There are still many
>> top-level Ada projects that share lower-level code, so if more than one
>> top-level project was open, there would not be a one-to-one mapping from
>> file to project.
>>
>> If Eclipse and Visual Studio are like that, so the collection of
>> projects contains only top level projects, then _if_ they have disjoint
>> file sets, you can get the project from a file. But that seems unlikely,
>> and highly use case dependent.
>
> In Eclipse, you simply cannot share files or folders between projects,
> at least not with the same path. 

Ok, so the file sets are (nominally) disjoint between projects (except
for system files, which are special). Apparently that's more
likely/common than I realized.

And if you are working on several top level projects that share core
code subprojects, you have to manually switch between the top level
projects, possibly by quitting that instance of Eclipse and starting
another.

So we still need that ability in Emacs, preferably without requiring
quiting Emacs :).

> You either copy them, or you symlink them from one project into
> another.

Yuck. That totally defeats CM!

<flame>
This all sounds like a desperate effort to pretend that we are _not_
working on projects that have an inherent hierarchical structure, with
shared code at the base. Apparently that would be too hard to
understand, or something. It does require the top level teams to
actually talk to each other to manage the core code; maybe that's the
problem (software engineers are not good at talking to each other?).

I prefer to embrace the hierarchy, and develop tools to let me manage it
efficiently.
</flame>

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[David Engster]

Stephen Leake writes:
> David Engster <deng@randomsample.de> writes:
>
>> Stephen Leake writes:
>>
>>> But that doesn't give me the full search path, because I don't know
>>> which higher level project the user is working on.
>>>
>>> In Emacs Ada mode, the top level project is the current project; the lower
>>> level projects are present as far as gprbuild is concerned, but the
>>> Emacs code doesn't need to know about them. There are still many
>>> top-level Ada projects that share lower-level code, so if more than one
>>> top-level project was open, there would not be a one-to-one mapping from
>>> file to project.
>>>
>>> If Eclipse and Visual Studio are like that, so the collection of
>>> projects contains only top level projects, then _if_ they have disjoint
>>> file sets, you can get the project from a file. But that seems unlikely,
>>> and highly use case dependent.
>>
>> In Eclipse, you simply cannot share files or folders between projects,
>> at least not with the same path. 
>
> Ok, so the file sets are (nominally) disjoint between projects (except
> for system files, which are special). Apparently that's more
> likely/common than I realized.
>
> And if you are working on several top level projects that share core
> code subprojects, you have to manually switch between the top level
> projects, possibly by quitting that instance of Eclipse and starting
> another.

You can of course load another workspace/solution into Eclipse/VS
without quitting.

>> You either copy them, or you symlink them from one project into
>> another.
>
> Yuck. That totally defeats CM!

What CM? Content management? Configuration management? Cyanogenmod? ;-)

> <flame>
> This all sounds like a desperate effort to pretend that we are _not_
> working on projects that have an inherent hierarchical structure, with
> shared code at the base. Apparently that would be too hard to
> understand, or something. It does require the top level teams to
> actually talk to each other to manage the core code; maybe that's the
> problem (software engineers are not good at talking to each other?).
>
> I prefer to embrace the hierarchy, and develop tools to let me manage it
> efficiently.
> </flame>

Not sure I understand you here. The usual way of code sharing I know is
to create libraries. Those have their own project, and if your project
depends on a library, you load its project into your workspace and
define a dependency on it. What's wrong with that? Would you prefer to
add it as some kind of sub-project into your project? That would be a
much stronger coupling.

-David

Re: running EDE from a file that is not under a project root dir

[Steinar Bang]

>>>>> Stephen Leake <stephen_leake@stephe-leake.org>:

> In Eclipse and Visual Studio with more than one project open, is the
> "current project" deduced from the current file?

Yes.

> If so how do they handle the issue of project hierarchy that I raised?

I don't know what the issue was, but eclipse doesn't have a project
hierarchy.

When importing projects from a system that is hierarchical (e.g. maven)
the project structure is flattened and the projects live side by side in
the workspace.

I have never used hierarchical/nested projects in Visual Studio so I
don't know how that behaves.

Re: running EDE from a file that is not under a project root dir

[Eric Ludlam]

On 08/06/2015 04:37 AM, Stephen Leake wrote:
> For example, if you are in an Ada file that is not part of the current
> project, and invoke 'ada-find-other-file' (to move from body to
> specification), you will get an error "file is not in the current
> project". That prevents the sort of error you describe above.
>
> But all commands that use project information should get it from the
> single current project, yes.

Ah, this is the distinction I was missing.  The global project only 
affects unaffiliated files.  I think I could agree with that.

> There are two levels of "optional" here; `ede-current-project' could
> always check the global var first; people who don't want that style just
> leave it set to nil.
>
> Or the single global variable could be one of a user-configurable list
> of things that ede-current-project checks.

The logic I would expect is that the buffer local variable wins, and if 
it is empty, it checks the global project.  That would prevent mistakes 
and provide flexibility.  If you want the global value to win against 
the local value, that could be an option which I would avoid using. ;)

I tend to work across multiple projects at the same time because CEDET 
is a support library, so I test it with different other projects. 
Having to constantly select what is active would be a PITA for me.

> Hmm. In addition, I would expect the Development | Load Project menu
> command to set the global variable, but others won't. So that needs to
> be configurable somehow. Ah; if ede-current-project is set to not check
> the global variable, it doesn't matter if it gets set. That was easy :).

The load menu just makes sure the project has been loaded in, then you 
can use project browsers and such.  Selecting the active project seems 
like a different feature.  I'd want to see the suite of options before 
recommending what goes where.

Eric

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Eric Ludlam <eric@siege-engine.com> writes:

> On 08/06/2015 04:37 AM, Stephen Leake wrote:
>> For example, if you are in an Ada file that is not part of the current
>> project, and invoke 'ada-find-other-file' (to move from body to
>> specification), you will get an error "file is not in the current
>> project". That prevents the sort of error you describe above.
>>
>> But all commands that use project information should get it from the
>> single current project, yes.
>
> Ah, this is the distinction I was missing.  The global project only
> affects unaffiliated files.  I think I could agree with that.

Only a few project-related commands work from files that are not part of
any project; xref-find-identifier would be one of the ones that does.

In files that are part of a project, it is still the single global
project that is used.

>> There are two levels of "optional" here; `ede-current-project' could
>> always check the global var first; people who don't want that style just
>> leave it set to nil.
>>
>> Or the single global variable could be one of a user-configurable list
>> of things that ede-current-project checks.
>
> The logic I would expect is that the buffer local variable wins, and
> if it is empty, it checks the global project.  That would prevent
> mistakes and provide flexibility.  If you want the global value to win
> against the local value, that could be an option which I would avoid
> using. ;)

As long as I can avoid having the buffer local variable ever get set,
that would work for me. From the little I've seen, that seems like major
surgery on the current EDE project-detect code.

> I tend to work across multiple projects at the same time because CEDET
> is a support library, so I test it with different other projects.
> Having to constantly select what is active would be a PITA for me.

Ok, that's a good use case. Hence the Emacs Prime Directive :).

>> Hmm. In addition, I would expect the Development | Load Project menu
>> command to set the global variable, but others won't. So that needs to
>> be configurable somehow. Ah; if ede-current-project is set to not check
>> the global variable, it doesn't matter if it gets set. That was easy :).
>
> The load menu just makes sure the project has been loaded in, then you
> can use project browsers and such.  Selecting the active project seems
> like a different feature.  

It could be a separate menu item, as long as they are both clearly
visible at the same time. 

So "load project" would add the project to the list of projects that
"select project" offers.

> I'd want to see the suite of options before
> recommending what goes where.

Right; the devil is in the details.

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Eric Ludlam]

On 08/05/2015 02:43 AM, Stephen Leake wrote:
>>> >>      The EDE info manual in Emacs master mentions ede-proj-elisp.el in
>>> >>      section 8 Extending EDE, but I don't see that file in Emacs master
>>> >>      git.
>> >
>> >That is a piece of the large Make/Automake project system that builds
>> >makefiles for you.
> I think you are saying the file ede-proj-elisp.el is in some CEDET
> repository, but not in Emacs core? I'll go look.

Sorry, I wasn't clear.

You can think of EDE as being broken into some different groups.  There 
is project detection, for finding projects on disk, and there is 'core' 
EDE functionality that just manages file names, menus, keybindings and 
the like.  There are also project implementations, such as a project for 
emacs, linux, and a project type that builds Makefiles for you based on 
meta data you provide.

ede/proj-elisp.el is part of a bigger project that creates Project.ede 
files.  The proj-elisp.el part is for managing Emacs Lisp code in that 
project type, and has code for creating makefile targets.

Anyone who digs into any of the proj-*.el files will find code that 
starts dealing with file name patterns.  These are used for building up 
make rules.  There are some abstractions in there for 'source' files in 
ede/source.el for supporting that.

I suppose that infrastructure could be used to build file patterns based 
on your current target, but it isn't bound to other project types.  I'll 
also guess you won't want EDE dropping Project.ede files or creating 
Makefiles as you go along, so some some development of EDE would be 
needed to start bring the concepts across between the feature sets.

Other project type also have emacs lisp targets, such as the Emacs 
project type.  To use, just enabled ede with

(global-ede-mode 1)

And go edit some Emacs.  Everything you might want to tweak that is 
project specific is in ede/emacs.el

You could also consider

(ede-enable-generic-projects)

to get other basic projects to start being used.  It will track .el 
files, but doesn't have any special features for it.


Eric

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Eric Ludlam <eric@siege-engine.com> writes:

> ede/proj-elisp.el is part of a bigger project that creates Project.ede
> files.  The proj-elisp.el part is for managing Emacs Lisp code in that
> project type, and has code for creating makefile targets.

Ok.

> Anyone who digs into any of the proj-*.el files will find code that
> starts dealing with file name patterns.  These are used for building
> up make rules.  There are some abstractions in there for 'source'
> files in ede/source.el for supporting that.
>
> I suppose that infrastructure could be used to build file patterns
> based on your current target, but it isn't bound to other project
> types.  

Ok. It could be added, so symref-grep can use it. find-file could also
use it.

> I'll also guess you won't want EDE dropping Project.ede files
> or creating Makefiles as you go along, 

A single Project.ede for each project is fine, as long as it tells me
that it's being created, and where. I expect a project manager to need a
project definition file.

Makefiles are not fine; many projects have no need of Makefiles (they
use other build tools). An explicit command to create a Makefile (or
other build specification file) would be fine.

> so some some development of EDE would be needed to start bring the
> concepts across between the feature sets.

Right. I'm willing to help do that development.

> Other project type also have emacs lisp targets, such as the Emacs
> project type.  To use, just enabled ede with
>
> (global-ede-mode 1)
>
> And go edit some Emacs.  

Is that supposed to work for random elisp code, or only for code in the
core Emacs repository?

I don't see any effect in random elisp code; in particular, M-:
(ede-current-project) returns nil.

Testing; same result in emacs/master/lisp/xml.el.

So I'm missing something.

> Everything you might want to tweak that is
> project specific is in ede/emacs.el

Hmm. No mention of load-path, so no tools for setting it. And some of
the functions are specific to the Emacs repository, but others are fully
general (ede-emacs-find-in-directories).

> You could also consider
>
> (ede-enable-generic-projects)
>
> to get other basic projects to start being used.  It will track .el
> files, but doesn't have any special features for it.

I had found that; still gives nil for (ede-current-project) in a .el
file, even after I did M-x ede-new, which created a Project.ede (without
telling me the file name or location) and an entry in my ~/.emacs
Customize (again without telling me).

I did some digging; it seems I didn't create a target. But Development |
Target Options | New Target is greyed out. It seems I created an
Automake project (it was the only choice even after
ede-enable-generic-projects), and that doesn't support creating targets.

Still missing something.

-- 
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Stephen Leake]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Eric Ludlam <eric@siege-engine.com> writes:
>
>> Other project type also have emacs lisp targets, such as the Emacs
>> project type.  To use, just enabled ede with
>>
>> (global-ede-mode 1)
>>
>> And go edit some Emacs.
>
> Is that supposed to work for random elisp code, or only for code in the
> core Emacs repository?
>
> I don't see any effect in random elisp code; in particular, M-:
> (ede-current-project) returns nil.
>
> Testing; same result in emacs/master/lisp/xml.el.
>
> So I'm missing something.

I tracked this down. To get this to work, I need the following:

(require 'ede)
(require 'ede/emacs)

(global-ede-mode 1)

Then when I edit a file in an Emacs git checkout, (ede-current-project)
returns the emacs project.

That project doesn't support 'compile', which is a little surprising. I
was wondering if it could handle out-of-tree builds, which I use.


Along the way, I discovered that the ede code in Emacs core is a little
broken by the recent changes in eieio/cl-generic.

I see this is fixed (at least partly?) in the current EDE SourceForge
git. So I'll use that code base for further work.

--
-- Stephe

Re: running EDE from a file that is not under a project root dir

[Eric Ludlam]

On 08/08/2015 11:26 AM, Stephen Leake wrote:
> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> Eric Ludlam <eric@siege-engine.com> writes:
>>
>>> Other project type also have emacs lisp targets, such as the Emacs
>>> project type.  To use, just enabled ede with
>>>
>>> (global-ede-mode 1)
>>>
>>> And go edit some Emacs.
>>
>> Is that supposed to work for random elisp code, or only for code in the
>> core Emacs repository?
>>
>> I don't see any effect in random elisp code; in particular, M-:
>> (ede-current-project) returns nil.
>>
>> Testing; same result in emacs/master/lisp/xml.el.
>>
>> So I'm missing something.
>
> I tracked this down. To get this to work, I need the following:
>
> (require 'ede)
> (require 'ede/emacs)
>
> (global-ede-mode 1)

The way it should work is to just enable global-ede-mode, and everything 
else should be brought in.  If it isn't there is probably something 
broken in the way it was merged with Emacs.  The setup to make EDE 
detect Emacs is in ede/emacs.el with an autoload cookie, but it isn't in 
loaddefs.   Probably because it requires eieio ??

Looks like a new mechanism is needed to make sure it bootstraps 
correctly in core Emacs.

> Then when I edit a file in an Emacs git checkout, (ede-current-project)
> returns the emacs project.
>
> That project doesn't support 'compile', which is a little surprising. I
> was wondering if it could handle out-of-tree builds, which I use.

I don't work on Emacs proper, so I never implemented it.  It wouldn't be 
hard to do.  ede/linux has examples of project-compile-project and 
-target that may be similar.  The Emacs version would probably be 
easier.   The C and Lisp and Texi targets could compile just those piece 
of Emacs.

> Along the way, I discovered that the ede code in Emacs core is a little
> broken by the recent changes in eieio/cl-generic.
>
> I see this is fixed (at least partly?) in the current EDE SourceForge
> git. So I'll use that code base for further work.

The sourceforge version uses older eieio, or eieio compatibility.  I 
haven't been able to get a clean Emacs/git build lately to try anything 
out. :(

Eric

Re: running EDE from a file that is not under a project root dir

[Dmitry Gutov]

On 08/05/2015 04:57 AM, Eric Ludlam wrote:

>> - add a "file patterns" element to the EDE project (or target?) data
>> structure, and have semantic-symref-derive-find-filepatterns check that
>> before the alists.
>
> I'm not sure what matching file patterns has to do with core project
> functionality.

A project could/should report on which kinds of source files there are 
in it. That can be done via file patterns.