Re: progmodes/project.el and search paths

[Eric Ludlam <ericludlam@gmail.com>]

On 08/02/2015 01:17 PM, Dmitry Gutov wrote:
> Hi Eric,
>
> On 08/02/2015 04:52 PM, Eric Ludlam wrote:
>
>> project.el and the current discussion seems to revolve almost entirely
>> around search paths and xref.  EDE does project management, but has not
>> needed to track search paths the way project.el does.  It certainly has
>> ways to find files, and has include paths and such needed by source
>> code, but not generic search paths.  I think project.el should be recast
>> as defining search paths for search tools.  EDE could then plug itself
>> in to provide some paths if asked.  I think calling it a "project" is
>> overstating what project.el does.
>
> The intention is to provide a generic API that other packages can use. 
> The roots and search-path looked to be the most important things to 
> know about a project, so far.
>
> If you can propose better ways to do the same things, or something 
> else worth adding, please do.

I apologize for not being up on the full thread.  It was very long and I 
didn't really get what the goals were.

One of my hopes for Emacs is that it will keep tabs on what I'm up to to 
provide useful context relevant feedback when I ask for it. My 
experience has been that tools that depend on a project to ask what the 
current project is do so a lot.  As such, I cache the current project, 
and current root project as buffer local variables.  Those project 
structures then have relevant meta data as well.  Another tactic is 
initializing each buffer into a project as it is created.  That moves 
search time into a different timing that is less noticeable than 
on-demand when the user makes a request.

Also, identifying projects should be a service of the core project 
system.  Dependent projects need to provide detection data without 
having Emacs load the full project definition.  That way you can have 
lots of projects types without forcing Emacs to load piles of code that 
you don't need.  My opinion is that all the projects should be available 
for detection without the user having to custom add anything about which 
they want.  Externally supplied projects (those not part of core emacs) 
can have a user provide a small snippet for detection or could do a 
trick with autoloads.

By way of example, EDE's autoloader does this well, and is wrapped in a 
complex test suite to make sure things don't get loaded when detecting 
projects of a different type.  I re-wrote most of it last year and it is 
much better than it was previously.  Use of 'locate-dominating-file' was 
a big win.

ede/detect.el is relatively independent of most of EDE's other 
functionality.  It does depend on ede/auto.el which defines a data 
structure for detecting projects.  Note that the important class is 
ede-project-autoload which provides basic matching data.  For more 
complex matching tasks, ede-project-autoload-dirmatch provides a little 
more.  Then over in ede/generic.el, you will find 
ede-enable-generic-projects, which has a simplified method for creating 
autoloads for the most basic of project types, including various vc 
types.  Your vc detector looks like it uses some nicer infrastructure 
than the simple file system matcher in generic.  The point is simply a 
simplified matcher construct.

If you'd like to adopt EDE's detector and rename bits of it 'project', 
that would work well for EDE (obviously) and would get you a very 
robust, flexible, and tested project detection system.  I don't think 
you can re-use EDE's caching without wholesale adoption of ede since it 
depends on ede's project classes.

Anyway, all that is just about matching projects.  Projects then also 
have meta data.  Things like compilation configuration, compile 
commands, release commands, include paths, classpaths, and whatnot. I 
don't know what you intend with project meta data.  If there was a 
thread about that, I missed it, sorry.

>> In the meantime, attached is a small patch to enable EDE to provide
>> roots to project.el.  The new function works for me with 24.3, but I
>> didn't try it in project.el
>
> I've implemented EDE support similarly to your patch initially, but 
> then Stefan opined it's better to dispatch on the actual EIEIO class. 
> This implementation lives near the bottom of ede.el in Emacs master. 
> Please take a look and try it sometime.

Thanks for the pointer.  My Emacs25 didn't compile when I pulled from 
git today, so I didn't get to try it or go very far beyond reading the 
latest in project.el.

For your implementation in ede.el, you are welcome to use what I wrote.  
If EDE is active, you can just reach out with those functions to get the 
same directory names.  For a subset of projects you can get other roots 
for different targets too.  I wasn't sure what the use case was for that 
was.

>> Perhaps all the work you're doing could just hang off ede since it only
>> takes 2 lines of code, and if it is missing some sort of pruning tricks,
>> it could just be added to ede.
>
> project.el defines a generic API. EDE is one implementation.

EDE used to be an implementation of a project management system that 
could edit Makefiles and Automakefiles.  The demands of other tools like 
semantic forced it to switch to a generic system that can detect 
arbitrary patterns of files that indicate projects created an maintained 
by other systems.  It then wraps those with detection code, and when 
possible pushes useful features such as compile commands, debugging, 
configuration options and more out to the user.

I think EDE has a generic API for defining a project.  It probably looks 
big on the outside if you just read the doc where it starts with 
"ede-new", but for most cases, if you enable EDE, it will find your 
Emacs src, Linux src, Cmake based build, etc, and it will magically 
provide that to Semantic's parser, to location functions (if enabled) 
and to taggers like global (if enabled.)   The reason it looks big is 
because those are the things that were needed while I was working on 
other tools that needed project support.  I would guess your system will 
grow to a similar state if you keep adding features needed by tools that 
want a project.  It is a really big task.

> One that a lot of Emacs users still avoid using (hard to tell exactly 
> why; 

Someone pointed out on the cedet mailing list recently that the doc (and 
probably the website) is too focused on how EDE can create makefiles 
which most people don't want.  Most questions on my mailing list ask how 
to get some other tool like smart completion in semantic working, and 
when they find out that they just need to turn on EDE, and not use the 
full project management system, they are pretty happy.

> probably because it doesn't match every kind of project). Projectile 
> seems to the most popular "project implementation" out there.

Projectile is a nice interface for moving around between files.  It 
happens to have a project detector in it.  Most of projectile is nice 
stuff that could fit easily on top of EDE's project detection without 
overlapping features of either tool.  Projectile has definitions for 
projects EDE does not have because various CEDET developers haven't 
worked with those tools.  The opposite is probably true too.

> Anyway, the suggestion to replace an agnostic API with a specific 
> implementation doesn't make a lot of sense.

If EDE is considered specific because it is called ede, or because it 
uses eieio, that would be too bad.  It's purpose is to generically 
detect projects such that anyone could create a new project definition, 
find project metadata, and make it accessible to other tools.  This has 
already happened many times.  Supporting EDE is more difficult than just 
providing a dominating file name because it also wants to find extra 
meta data.  I think that is a pretty generic task also.

One way EDE could be made more generic is to make the 'target' concept 
optional.  Simple ede projects also have to define targets as part of 
the detection and matching system, so making them optional would 
simplify creating new projects types in EDE.

A second way for EDE to be more generic is to better support 
ede/generic.el, or perhaps have something like ede/generic.el that 
leaves off all the extra meta-data handling.

Another way EDE could be made more generic is by having built-in tools 
like the 'compile' command ask EDE how to compile, or by having flymake 
use EDE to autodetect how to run make, or perhaps etags could ask EDE 
where to run etags from, and a host of other options.  I had pulled 
those features out into ede named features to keep things independent 
when CEDET was an independent tool.  I don't see why naming conventions 
couldn't change to make things better.

Eric