* progmodes/project.el and search paths @ 2015-08-02 13:52 Eric Ludlam 2015-08-02 17:17 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Eric Ludlam @ 2015-08-02 13:52 UTC (permalink / raw) To: Emacs Development [-- Attachment #1: Type: text/plain, Size: 1384 bytes --] I have a some thoughts related to the project.el project being discussed in the mailing list. When it was first proposed as a set of core functions to generically describe a project, I thought it could be a good idea, since there are a several 'projects' out there, such as EDE, projectile, jdee, and a few others I know less about. It would save some tools the trouble of knowing how to integrate all those possibilities if all they needed was some random piece of project metadata. project.el and the current discussion seems to revolve almost entirely around search paths and xref. EDE does project management, but has not needed to track search paths the way project.el does. It certainly has ways to find files, and has include paths and such needed by source code, but not generic search paths. I think project.el should be recast as defining search paths for search tools. EDE could then plug itself in to provide some paths if asked. I think calling it a "project" is overstating what project.el does. In the meantime, attached is a small patch to enable EDE to provide roots to project.el. The new function works for me with 24.3, but I didn't try it in project.el Perhaps all the work you're doing could just hang off ede since it only takes 2 lines of code, and if it is missing some sort of pruning tricks, it could just be added to ede. Eric [-- Attachment #2: project.el.patch --] [-- Type: text/x-patch, Size: 1040 bytes --] diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el index d849f93..3fd1358 100644 --- a/lisp/progmodes/project.el +++ b/lisp/progmodes/project.el @@ -27,7 +27,8 @@ (require 'cl-generic) -(defvar project-find-functions (list #'project-try-vc +(defvar project-find-functions (list #'project-try-ede + #'project-try-vc #'project-ask-user) "Special hook to find the project containing a given directory. Each functions on this hook is called in turn with one @@ -99,6 +100,13 @@ an element of `project-search-path'." (vc-call-backend backend 'root dir))))) (and root (cons 'vc root)))) +(defun project-try-ede (dir) + "Use EDE to determine if DIR is a project directory." + (when (featurep 'ede) + (let* ((eproj (ede-current-project dir)) + (eroot (ede-project-root-directory eproj))) + (cons 'ede eroot)))) + (cl-defmethod project-roots ((project (head vc))) (list (cdr project))) ^ permalink raw reply related [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-02 13:52 progmodes/project.el and search paths Eric Ludlam @ 2015-08-02 17:17 ` Dmitry Gutov 2015-08-03 1:19 ` Eric Ludlam ` (2 more replies) 0 siblings, 3 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-02 17:17 UTC (permalink / raw) To: Eric Ludlam, Emacs Development Hi Eric, On 08/02/2015 04:52 PM, Eric Ludlam wrote: > project.el and the current discussion seems to revolve almost entirely > around search paths and xref. EDE does project management, but has not > needed to track search paths the way project.el does. It certainly has > ways to find files, and has include paths and such needed by source > code, but not generic search paths. I think project.el should be recast > as defining search paths for search tools. EDE could then plug itself > in to provide some paths if asked. I think calling it a "project" is > overstating what project.el does. The intention is to provide a generic API that other packages can use. The roots and search-path looked to be the most important things to know about a project, so far. If you can propose better ways to do the same things, or something else worth adding, please do. > In the meantime, attached is a small patch to enable EDE to provide > roots to project.el. The new function works for me with 24.3, but I > didn't try it in project.el I've implemented EDE support similarly to your patch initially, but then Stefan opined it's better to dispatch on the actual EIEIO class. This implementation lives near the bottom of ede.el in Emacs master. Please take a look and try it sometime. > Perhaps all the work you're doing could just hang off ede since it only > takes 2 lines of code, and if it is missing some sort of pruning tricks, > it could just be added to ede. project.el defines a generic API. EDE is one implementation. One that a lot of Emacs users still avoid using (hard to tell exactly why; probably because it doesn't match every kind of project). Projectile seems to the most popular "project implementation" out there. Anyway, the suggestion to replace an agnostic API with a specific implementation doesn't make a lot of sense. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-02 17:17 ` Dmitry Gutov @ 2015-08-03 1:19 ` Eric Ludlam 2015-08-03 16:16 ` Stephen Leake 2015-08-03 22:47 ` Dmitry Gutov 2015-08-03 13:49 ` David Engster 2015-08-05 1:29 ` Eric Ludlam 2 siblings, 2 replies; 101+ messages in thread From: Eric Ludlam @ 2015-08-03 1:19 UTC (permalink / raw) To: Dmitry Gutov, Eric Ludlam, Emacs Development On 08/02/2015 01:17 PM, Dmitry Gutov wrote: > Hi Eric, > > On 08/02/2015 04:52 PM, Eric Ludlam wrote: > >> project.el and the current discussion seems to revolve almost entirely >> around search paths and xref. EDE does project management, but has not >> needed to track search paths the way project.el does. It certainly has >> ways to find files, and has include paths and such needed by source >> code, but not generic search paths. I think project.el should be recast >> as defining search paths for search tools. EDE could then plug itself >> in to provide some paths if asked. I think calling it a "project" is >> overstating what project.el does. > > The intention is to provide a generic API that other packages can use. > The roots and search-path looked to be the most important things to > know about a project, so far. > > If you can propose better ways to do the same things, or something > else worth adding, please do. I apologize for not being up on the full thread. It was very long and I didn't really get what the goals were. One of my hopes for Emacs is that it will keep tabs on what I'm up to to provide useful context relevant feedback when I ask for it. My experience has been that tools that depend on a project to ask what the current project is do so a lot. As such, I cache the current project, and current root project as buffer local variables. Those project structures then have relevant meta data as well. Another tactic is initializing each buffer into a project as it is created. That moves search time into a different timing that is less noticeable than on-demand when the user makes a request. Also, identifying projects should be a service of the core project system. Dependent projects need to provide detection data without having Emacs load the full project definition. That way you can have lots of projects types without forcing Emacs to load piles of code that you don't need. My opinion is that all the projects should be available for detection without the user having to custom add anything about which they want. Externally supplied projects (those not part of core emacs) can have a user provide a small snippet for detection or could do a trick with autoloads. By way of example, EDE's autoloader does this well, and is wrapped in a complex test suite to make sure things don't get loaded when detecting projects of a different type. I re-wrote most of it last year and it is much better than it was previously. Use of 'locate-dominating-file' was a big win. ede/detect.el is relatively independent of most of EDE's other functionality. It does depend on ede/auto.el which defines a data structure for detecting projects. Note that the important class is ede-project-autoload which provides basic matching data. For more complex matching tasks, ede-project-autoload-dirmatch provides a little more. Then over in ede/generic.el, you will find ede-enable-generic-projects, which has a simplified method for creating autoloads for the most basic of project types, including various vc types. Your vc detector looks like it uses some nicer infrastructure than the simple file system matcher in generic. The point is simply a simplified matcher construct. If you'd like to adopt EDE's detector and rename bits of it 'project', that would work well for EDE (obviously) and would get you a very robust, flexible, and tested project detection system. I don't think you can re-use EDE's caching without wholesale adoption of ede since it depends on ede's project classes. Anyway, all that is just about matching projects. Projects then also have meta data. Things like compilation configuration, compile commands, release commands, include paths, classpaths, and whatnot. I don't know what you intend with project meta data. If there was a thread about that, I missed it, sorry. >> In the meantime, attached is a small patch to enable EDE to provide >> roots to project.el. The new function works for me with 24.3, but I >> didn't try it in project.el > > I've implemented EDE support similarly to your patch initially, but > then Stefan opined it's better to dispatch on the actual EIEIO class. > This implementation lives near the bottom of ede.el in Emacs master. > Please take a look and try it sometime. Thanks for the pointer. My Emacs25 didn't compile when I pulled from git today, so I didn't get to try it or go very far beyond reading the latest in project.el. For your implementation in ede.el, you are welcome to use what I wrote. If EDE is active, you can just reach out with those functions to get the same directory names. For a subset of projects you can get other roots for different targets too. I wasn't sure what the use case was for that was. >> Perhaps all the work you're doing could just hang off ede since it only >> takes 2 lines of code, and if it is missing some sort of pruning tricks, >> it could just be added to ede. > > project.el defines a generic API. EDE is one implementation. EDE used to be an implementation of a project management system that could edit Makefiles and Automakefiles. The demands of other tools like semantic forced it to switch to a generic system that can detect arbitrary patterns of files that indicate projects created an maintained by other systems. It then wraps those with detection code, and when possible pushes useful features such as compile commands, debugging, configuration options and more out to the user. I think EDE has a generic API for defining a project. It probably looks big on the outside if you just read the doc where it starts with "ede-new", but for most cases, if you enable EDE, it will find your Emacs src, Linux src, Cmake based build, etc, and it will magically provide that to Semantic's parser, to location functions (if enabled) and to taggers like global (if enabled.) The reason it looks big is because those are the things that were needed while I was working on other tools that needed project support. I would guess your system will grow to a similar state if you keep adding features needed by tools that want a project. It is a really big task. > One that a lot of Emacs users still avoid using (hard to tell exactly > why; Someone pointed out on the cedet mailing list recently that the doc (and probably the website) is too focused on how EDE can create makefiles which most people don't want. Most questions on my mailing list ask how to get some other tool like smart completion in semantic working, and when they find out that they just need to turn on EDE, and not use the full project management system, they are pretty happy. > probably because it doesn't match every kind of project). Projectile > seems to the most popular "project implementation" out there. Projectile is a nice interface for moving around between files. It happens to have a project detector in it. Most of projectile is nice stuff that could fit easily on top of EDE's project detection without overlapping features of either tool. Projectile has definitions for projects EDE does not have because various CEDET developers haven't worked with those tools. The opposite is probably true too. > Anyway, the suggestion to replace an agnostic API with a specific > implementation doesn't make a lot of sense. If EDE is considered specific because it is called ede, or because it uses eieio, that would be too bad. It's purpose is to generically detect projects such that anyone could create a new project definition, find project metadata, and make it accessible to other tools. This has already happened many times. Supporting EDE is more difficult than just providing a dominating file name because it also wants to find extra meta data. I think that is a pretty generic task also. One way EDE could be made more generic is to make the 'target' concept optional. Simple ede projects also have to define targets as part of the detection and matching system, so making them optional would simplify creating new projects types in EDE. A second way for EDE to be more generic is to better support ede/generic.el, or perhaps have something like ede/generic.el that leaves off all the extra meta-data handling. Another way EDE could be made more generic is by having built-in tools like the 'compile' command ask EDE how to compile, or by having flymake use EDE to autodetect how to run make, or perhaps etags could ask EDE where to run etags from, and a host of other options. I had pulled those features out into ede named features to keep things independent when CEDET was an independent tool. I don't see why naming conventions couldn't change to make things better. Eric ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 1:19 ` Eric Ludlam @ 2015-08-03 16:16 ` Stephen Leake 2015-08-03 22:56 ` Dmitry Gutov 2015-08-08 13:07 ` Nix 2015-08-03 22:47 ` Dmitry Gutov 1 sibling, 2 replies; 101+ messages in thread From: Stephen Leake @ 2015-08-03 16:16 UTC (permalink / raw) To: emacs-devel Eric Ludlam <ericludlam@gmail.com> writes: > I apologize for not being up on the full thread. It was very long and > I didn't really get what the goals were. I still don't get what the goals are ;). > If you'd like to adopt EDE's detector and rename bits of it 'project', > that would work well for EDE (obviously) and would get you a very > robust, flexible, and tested project detection system. I don't think > you can re-use EDE's caching without wholesale adoption of ede since > it depends on ede's project classes. That makes sense to me. In the context of project.el, this detection would return both the "project backend" (the elisp code to load) and any relevant "project file", if the backend uses one. I agree the result of that detection should be cached, rather than recomputed each time it is needed. I prefer to cache it in a global variable; that way, I have only one project active at a time, for all buffers. So if I'm debugging the elisp code that implements Ada mode, I get elisp search and completion paths from Ada mode buffers, as well as elisp buffers. > Anyway, all that is just about matching projects. Projects then also > have meta data. Things like compilation configuration, compile > commands, release commands, include paths, classpaths, and whatnot. I think that level of metadata belongs on a backend, not the core. Some projects don't need all of that data (elisp and Ada mode for example). > EDE used to be an implementation of a project management system that > could edit Makefiles and Automakefiles. The demands of other tools > like semantic forced it to switch to a generic system that can detect > arbitrary patterns of files that indicate projects created an > maintained by other systems. It then wraps those with detection code, > and when possible pushes useful features such as compile commands, > debugging, configuration options and more out to the user. I'll see if I can implement Ada, JDEE, and elisp projects as EDE backends; that will allow a detailed comparison to the project backends I'm also writing. > I think EDE has a generic API for defining a project. It probably > looks big on the outside if you just read the doc where it starts with > "ede-new", but for most cases, if you enable EDE, it will find your > Emacs src, Linux src, Cmake based build, etc, and it will magically > provide that to Semantic's parser, to location functions (if enabled) > and to taggers like global (if enabled.) Ada mode has its own parser and xref tool, and never uses CMake. So at first glance it seems like there is too much in the EDE core. Similarly for JDEE. > The reason it looks big is because those are the things that > were needed while I was working on other tools that needed project > support. I would guess your system will grow to a similar state if you > keep adding features needed by tools that want a project. It is a > really big task. Right. And we should leverage the work that has gone into EDE. I suspect we are really talking about refactoring EDE so the core is smaller. That is really just an optimization, and may not be worth it, if the overhead the present core imposes is not noticeable. >> One that a lot of Emacs users still avoid using (hard to tell >> exactly why; > > Someone pointed out on the cedet mailing list recently that the doc > (and probably the website) is too focused on how EDE can create > makefiles which most people don't want. Yes; that is the first doc I saw (I don't remember where now, sorry). The intro at https://www.gnu.org/software/emacs/manual/html_node/emacs/EDE.html is better. Although it could mention that it is possible to add new project types (I guess that's just assumed with Emacs :). >> probably because it doesn't match every kind of project). Projectile >> seems to the most popular "project implementation" out there. > > Projectile is a nice interface for moving around between files. It > happens to have a project detector in it. Most of projectile is nice > stuff that could fit easily on top of EDE's project detection without > overlapping features of either tool. Projectile has definitions for > projects EDE does not have because various CEDET developers haven't > worked with those tools. The opposite is probably true too. Which is one reason to have an "independent" project core; perhaps the projectile team will be more amenable to adopting a core API that isn't named "ede"? >> Anyway, the suggestion to replace an agnostic API with a specific >> implementation doesn't make a lot of sense. > > If EDE is considered specific because it is called ede, or because it > uses eieio, that would be too bad. Right. "EDE" is "Emacs Development Environment"; that's a good generic name. eieio is apparently a little more problematic; some people are still alergic to object oriented programming. For example, projects.el and xref.el use several random function variables, in addition to dispatching via cl-defmethod. There are also restrictions for elisp, since it is preloaded. I think the current rule is cl-lib can be preloaded, but not eieio? > It's purpose is to generically > detect projects such that anyone could create a new project > definition, find project metadata, and make it accessible to other > tools. This has already happened many times. Supporting EDE is more > difficult than just providing a dominating file name because it also > wants to find extra meta data. I think that is a pretty generic task > also. If it _insists_ on collecting meta data that is simply not defined (for example "the location of the CMake file in this Ada project"), and can't handle either nil or signal as a result, that's a problem. > One way EDE could be made more generic is to make the 'target' concept > optional. Simple ede projects also have to define targets as part of > the detection and matching system, so making them optional would > simplify creating new projects types in EDE. That would help; anything to reduce the burden of getting a new project backend working is good. > Another way EDE could be made more generic is by having built-in tools > like the 'compile' command ask EDE how to compile, or by having > flymake use EDE to autodetect how to run make, or perhaps etags could > ask EDE where to run etags from, and a host of other options. I had > pulled those features out into ede named features to keep things > independent when CEDET was an independent tool. I don't see why > naming conventions couldn't change to make things better. Yes. xref is going the same root as EDE took; there is xref-find-regexp. I agree it would be better (if projects are enabled) for grep to use project information for the search path. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 16:16 ` Stephen Leake @ 2015-08-03 22:56 ` Dmitry Gutov 2015-08-08 13:07 ` Nix 1 sibling, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 22:56 UTC (permalink / raw) To: Stephen Leake, emacs-devel On 08/03/2015 07:16 PM, Stephen Leake wrote: > Which is one reason to have an "independent" project core; perhaps the > projectile team will be more amenable to adopting a core API that isn't > named "ede"? I'm sure any reservations developers have about EDE are not related to the name. > There are also restrictions for elisp, since it is preloaded. I think > the current rule is cl-lib can be preloaded, but not eieio? eieio is not preloaded, but cl-generic already is. That means you can use the generic functions and cl-structs, but not EIEIO classes, in the preloaded code. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 16:16 ` Stephen Leake 2015-08-03 22:56 ` Dmitry Gutov @ 2015-08-08 13:07 ` Nix 2015-08-09 5:18 ` Stephen Leake 1 sibling, 1 reply; 101+ messages in thread From: Nix @ 2015-08-08 13:07 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel On 3 Aug 2015, Stephen Leake told this: > I prefer to cache it in a global variable; that way, I have only one > project active at a time, for all buffers. That doesn't work very well for those of us working on multi-component projects. If I'm working in the Linux kernel I do *not* want identifier search or file lookup wandering into glibc or Emacs, even though they share some identifiers (with completely divergent definitions) and even though I may have quite a lot of buffers open on those projects. (I have fairly often worked on things that affect glibc, binutils, GCC, and the kernel all at once. Those are different projects with distinct roots, identifier sets, and tags tables...) -- NULL && (void) ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-08 13:07 ` Nix @ 2015-08-09 5:18 ` Stephen Leake 2015-08-09 12:17 ` David Engster 0 siblings, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-09 5:18 UTC (permalink / raw) To: emacs-devel Nix <nix@esperi.org.uk> writes: > On 3 Aug 2015, Stephen Leake told this: >> I prefer to cache it in a global variable; that way, I have only one >> project active at a time, for all buffers. > > That doesn't work very well for those of us working on multi-component > projects. Fine; of course this behavior should be user-configurable. At the same time, I work on multi-component projects all the time, and one active project works for me. > If I'm working in the Linux kernel I do *not* want identifier > search or file lookup wandering into glibc or Emacs, even though they > share some identifiers (with completely divergent definitions) and even > though I may have quite a lot of buffers open on those projects. > (I have fairly often worked on things that affect glibc, binutils, GCC, > and the kernel all at once. Those are different projects with distinct > roots, identifier sets, and tags tables...) But if you are in glibc, looking at an identifier for a function that happens to be defined in the linux kernel, you want to be able to find the corresponding definition; the linux kernel is a subproject of glibc (which is a subproject of higher level projects like Emacs). So the Emacs project tools should reflect that. For example, file name completion must handle duplicate file names. That's a problem right now in Emacs for elisp files; there are both lisp/cedet/ede/dired.el and lisp/dired.el in Emacs core. 'load' and 'require' handle this by having only lisp/cedet in load-path, not lisp/cedet/ede. So if I do file name completion with just load-path, I have to know to type ede/d<tab> when searching for ede/dired.el, which is _not_ friendly. But if I do file name completion with an expanded load path that includes cedet/ede, it shows only one dired.el (it happens to be ede/dired.el). Which is also not friendly; I'll be filing a bug report on this later today. For identifier search, you can use a cross reference tool that understands name overloading; AdaCore asistant is one - it uses cross reference information output by the compiler, which obviously handles the duplicate names correctly. The multi-component projects I work on are in Ada, using the AdaCore tools. Both make multi-component projects easier to work with, and Emacs Ada mode understands the full project structure. Recently we added a large C++ component to one product, and it suffered from some of the above problems. So part of the solution is using a better programming language (not always a choice, but it will never happen if we don't ask for it!). In addition, sometimes the user wants to search all subprojects, sometimes a subset. So we need a good way of specifying subsets of subprojects at search time. Emacs Ada mode doesn't support this yet (I just put up with the unwanted hits in searches; better than false negaitives). Hardcoding the searchable subset to only the top-level project is overly restrictive. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-09 5:18 ` Stephen Leake @ 2015-08-09 12:17 ` David Engster 2015-08-09 15:55 ` Stephen Leake 2015-08-10 17:12 ` Nix 0 siblings, 2 replies; 101+ messages in thread From: David Engster @ 2015-08-09 12:17 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel Stephen Leake writes: > Nix <nix@esperi.org.uk> writes: > >> On 3 Aug 2015, Stephen Leake told this: >>> I prefer to cache it in a global variable; that way, I have only one >>> project active at a time, for all buffers. >> >> That doesn't work very well for those of us working on multi-component >> projects. > > Fine; of course this behavior should be user-configurable. > > At the same time, I work on multi-component projects all the time, and > one active project works for me. > >> If I'm working in the Linux kernel I do *not* want identifier >> search or file lookup wandering into glibc or Emacs, even though they >> share some identifiers (with completely divergent definitions) and even >> though I may have quite a lot of buffers open on those projects. > >> (I have fairly often worked on things that affect glibc, binutils, GCC, >> and the kernel all at once. Those are different projects with distinct >> roots, identifier sets, and tags tables...) > > But if you are in glibc, looking at an identifier for a function that > happens to be defined in the linux kernel, you want to be able to find > the corresponding definition; the linux kernel is a subproject of > glibc (which is a subproject of higher level projects like Emacs). I must say I find this to be a very unusual view. Just because a program depends on libc, the libc is not a "subproject". Instead, you have a dependency on a library which has a fixed API. The only thing you need are the declarations of the API and the needed type definitions, which are given through the system headers, but those hardly qualify as a "subproject"; they simply state how libc functions must be called. I don't care how printf is implemented; in fact, I might have to switch to musl or ulibc if my target system can't handle glibc's size. You can of course have the full glibc and Emacs sources loaded as projects at the same time, but they are separate projects on the same level in the project hierarchy. > So the Emacs project tools should reflect that. For example, file name > completion must handle duplicate file names. That's a problem right now > in Emacs for elisp files; there are both lisp/cedet/ede/dired.el and > lisp/dired.el in Emacs core. This is a different problem. You are within one project here; of course a project system should be able to handle this. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-09 12:17 ` David Engster @ 2015-08-09 15:55 ` Stephen Leake 2015-08-10 11:29 ` David Engster 2015-08-10 17:12 ` Nix 1 sibling, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-09 15:55 UTC (permalink / raw) To: emacs-devel David Engster <deng@randomsample.de> writes: > Stephen Leake writes: >> >> But if you are in glibc, looking at an identifier for a function that >> happens to be defined in the linux kernel, you want to be able to find >> the corresponding definition; the linux kernel is a subproject of >> glibc (which is a subproject of higher level projects like Emacs). > > I must say I find this to be a very unusual view. Just because a program > depends on libc, the libc is not a "subproject". We are arguing about the definition of the word "subproject". The Oxford English Dictonary doesn't define it. Wiktionary says: A project within a larger project. I agree libc is not "within" Emacs, so apparently I'm using the word wrong. > Instead, you have a > dependency on a library which has a fixed API. "dependency" is fine with me. "fixed API"; only sometimes. If I have control over both the top level project and the dependency, it is sometimes advantageous to change the API to better implement some feature. > The only thing you need are the declarations of the API and the needed > type definitions, Sometimes. There were _many_ times at NASA when I wished we had paid the contractor to deliver the full source code, rather than an API and user guide. Certainly in Emacs the doc string is often not enough, and I need to read the body of the function to find out what's actually going on. I often feel the need to do that in Android, but I haven't developed the search/cross-reference skills to be able to yet. > which are given through the system headers, Not all libraries are "system", but libraries do provide headers, yes. Whether and where a library is "installed" is a packaging/distribution detail; it should not impact this discussion. > I don't care how printf is implemented; There are some libraries you trust (hopefully the ones you are paying for), and others not so much. >> So the Emacs project tools should reflect that. For example, file name >> completion must handle duplicate file names. That's a problem right now >> in Emacs for elisp files; there are both lisp/cedet/ede/dired.el and >> lisp/dired.el in Emacs core. > > This is a different problem. Ok, now we are getting somewhere; this is one example of the kind of problems I'm trying to address. > You are within one project here; No, 'cedet' is a library as far as I'm concerned; it has an API, I try to treat it as a black box. The fact that it is in Emacs core is just a packaging issue; it doesn't change the logical structure. The fact that Emacs locate-library searches within it just indicates that Emacs got it right from my point of view :). ELPA packages are also libraries; they are included in load-path when I need them, and not otherwise. The same problem occurs with any language library projects; if I'm working on top_project, and need to read the impelementation for a function that happens to be in library_project_1, they both need to be in the file completion search path. Same for identifier completion, find parent/child declarations, etc. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-09 15:55 ` Stephen Leake @ 2015-08-10 11:29 ` David Engster 2015-08-10 16:43 ` Stephen Leake 0 siblings, 1 reply; 101+ messages in thread From: David Engster @ 2015-08-10 11:29 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel Stephen Leake writes: > David Engster <deng@randomsample.de> writes: > >> Stephen Leake writes: >>> >>> But if you are in glibc, looking at an identifier for a function that >>> happens to be defined in the linux kernel, you want to be able to find >>> the corresponding definition; the linux kernel is a subproject of >>> glibc (which is a subproject of higher level projects like Emacs). >> >> I must say I find this to be a very unusual view. Just because a program >> depends on libc, the libc is not a "subproject". > > We are arguing about the definition of the word "subproject". > > The Oxford English Dictonary doesn't define it. > > Wiktionary says: > > A project within a larger project. > > I agree libc is not "within" Emacs, so apparently I'm using the word > wrong. Actually, EDE has support for subprojects. For instance, EDE itself is a subproject of CEDET, just like Semantic, SRecode, and Cogre. Together, they form CEDET. While this is nice in theory, this generates more problems than it is worth. For instance, EDE will generate a recursive Makefile for this, which I'm really not very fond of... >> The only thing you need are the declarations of the API and the needed >> type definitions, > > Sometimes. There were _many_ times at NASA when I wished we had paid the > contractor to deliver the full source code, rather than an API and user > guide. > > Certainly in Emacs the doc string is often not enough, and I need to > read the body of the function to find out what's actually going on. > > I often feel the need to do that in Android, but I haven't developed the > search/cross-reference skills to be able to yet. We are in violent agreement here. Of course you should *have* the source. But I would not put it into a subproject. I want a flat project hierarchy. >>> So the Emacs project tools should reflect that. For example, file name >>> completion must handle duplicate file names. That's a problem right now >>> in Emacs for elisp files; there are both lisp/cedet/ede/dired.el and >>> lisp/dired.el in Emacs core. >> >> This is a different problem. > > Ok, now we are getting somewhere; this is one example of the kind of > problems I'm trying to address. > >> You are within one project here; > > No, 'cedet' is a library as far as I'm concerned; it has an API, I try > to treat it as a black box. Sorry, but CEDET is not a library. You might think that the lisp/cedet directory is CEDET, but that's not true (unfortunately, as it makes merging pretty tedious). The files are scattered all over the Emacs sources, which is also why it doesn't make sense to treat lisp/cedet as a "subproject". The grammars are in admin/grammars, SRecode files are in etc/srecode, documentation is in doc/misc, EIEIO is in lisp/emacs-lisp. They are all part of CEDET (though EIEIO is now maintained in Emacs). Originally there was even more (speedbar, for instance). > The fact that it is in Emacs core is just a packaging issue; it doesn't > change the logical structure. First and foremost, we simply have files with the same name scattered across the Emacs source tree. dired.el, custom.el, speedbar.el, shell.el, and many more. This is a very common thing in larger projects, and a project system must be able to handle this. Technically, since lisp/cedet/ede is not in load-path, those two are different files (dired.el and ede/dired.el). Also, all the symbols defined in those files live in different namespaces (dired- and ede-, resp.). So really, the only issue here is when you say "give me the file dired.el from the Emacs project", in which case the project system should tell you that it exists twice and asks you which one to load. But it doesn't matter when you search for symbols. > The same problem occurs with any language library projects; if I'm > working on top_project, and need to read the impelementation for a > function that happens to be in library_project_1, they both need to be > in the file completion search path. Say I really want to see the implementation of 'foo' in glibc. I simply load the glibc project into my workspace (to use the Eclipse lingo); it is not a subproject, but simply another project parallel to all the others I have loaded. I can now jump from some Emacs C source file to the 'foo' definition. My guess your issue is that another library might have defined a symbol 'foo' as well, and its project is also loaded in my workspace - then it depends on the #include's in your source file. So here, a project system must ask something like Semantic for the search path of the current file. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-10 11:29 ` David Engster @ 2015-08-10 16:43 ` Stephen Leake 2015-08-12 10:10 ` David Engster 0 siblings, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-10 16:43 UTC (permalink / raw) To: emacs-devel David Engster <deng@randomsample.de> writes: > Stephen Leake writes: >> The same problem occurs with any language library projects; if I'm >> working on top_project, and need to read the impelementation for a >> function that happens to be in library_project_1, they both need to be >> in the file completion search path. > > Say I really want to see the implementation of 'foo' in glibc. I simply > load the glibc project into my workspace (to use the Eclipse lingo); it > is not a subproject, but simply another project parallel to all the > others I have loaded. I can now jump from some Emacs C source file to > the 'foo' definition. Do you have to load glibc manually? I would think that would be done for you by the declared dependency. So Eclipse implicitly adds all the projects to a single "source search path". That is what I'm asking for in Emacs project-related searches. load-path does that for elisp. compilation-search-path does that for most language modes. In Ada mode lingo, that group of projects is one hierarchical project; the project definition files define the project hierarchy. Ada mode supports having several distict hierarchical projects loaded, but only one active. So that is the source of my confusion. > My guess your issue is that another library might have defined a symbol > 'foo' as well, and its project is also loaded in my workspace - then it > depends on the #include's in your source file. So here, a project system > must ask something like Semantic for the search path of the current > file. Depends on the use case; the user might want to see: - "all (overloaded) definitions of "foo", in all projects " Perhaps to check whether current occurance could be using a different definition. Or to see if the different definitions could be combined, or further refactored. This is more useful in languages that explicitly support overloaded symbols (Ada has many definitions of "Put" in its standard library). - "the definition for _this_ use of "foo", in some project " To see what it actually does. - "all methods for this generic function" Similar to the first use case, but limited to one generic function. So the question now is: what does EDE do for these use cases? Ie, what functions does the user invoke in each case, and what is the result? The definition of xref-find-definitions is agnostic with respect to these use cases; the "identifier" returned by xref--read-identifier can be: - a simple string for the first use case - a more complex structure (in text properties) that identifies this occurance for the second and third use case In addition, the backend can use the extra info or ignore it. But the xref-find-definitions UI doesn't allow the user to express a desire for one these uses. In any case, the "source search path" used by xref-find-definitions should include all of the relevant projects, as defined by the project dependencies. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-10 16:43 ` Stephen Leake @ 2015-08-12 10:10 ` David Engster 2015-08-12 13:49 ` Stephen Leake 0 siblings, 1 reply; 101+ messages in thread From: David Engster @ 2015-08-12 10:10 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel Stephen Leake writes: > David Engster <deng@randomsample.de> writes: > >> Stephen Leake writes: >>> The same problem occurs with any language library projects; if I'm >>> working on top_project, and need to read the impelementation for a >>> function that happens to be in library_project_1, they both need to be >>> in the file completion search path. >> >> Say I really want to see the implementation of 'foo' in glibc. I simply >> load the glibc project into my workspace (to use the Eclipse lingo); it >> is not a subproject, but simply another project parallel to all the >> others I have loaded. I can now jump from some Emacs C source file to >> the 'foo' definition. > > Do you have to load glibc manually? Yes. You can either create a separate project for glibc and add this to your workspace, or you add the sources of glibc directly to your project as a special "library source". The latter is what you usually do, unless you are a glibc developer. > I would think that would be done for you by the declared dependency. Since you only need the headers for compilation and the library object for linking, there's no need for Eclipse to force you to add the sources. It also cannot add them automatically, because they are usually not installed in the first place. My current Emacs binary depends on 73 libraries, and AFAIK I have sources for exactly none of them installed. > So Eclipse implicitly adds all the projects to a single "source search > path". Not implicitly; you have to configure that. > That is what I'm asking for in Emacs project-related searches. > > load-path does that for elisp. compilation-search-path does that for > most language modes. > > In Ada mode lingo, that group of projects is one hierarchical project; > the project definition files define the project hierarchy. Ada mode > supports having several distict hierarchical projects loaded, but only > one active. > > So that is the source of my confusion. I see. >> My guess your issue is that another library might have defined a symbol >> 'foo' as well, and its project is also loaded in my workspace - then it >> depends on the #include's in your source file. So here, a project system >> must ask something like Semantic for the search path of the current >> file. > > Depends on the use case; the user might want to see: > > - "all (overloaded) definitions of "foo", in all projects " > > Perhaps to check whether current occurance could be using a > different definition. Or to see if the different definitions could > be combined, or further refactored. > > This is more useful in languages that explicitly support overloaded > symbols (Ada has many definitions of "Put" in its standard library). > > - "the definition for _this_ use of "foo", in some project " > > To see what it actually does. > > - "all methods for this generic function" > > Similar to the first use case, but limited to one generic function. > > So the question now is: what does EDE do for these use cases? AFAIK, you can only search in the current project. However, your use cases involve more than just a project system. If you want to search for overloads or methods, you need a parser as well. This also applies to the most important feature (at least for me): show all definitions of 'foo' that are currently in scope. EDE has no notions of symbols, scope, overloading, methods, etc. It gets much more complicated when you're not only interested in definitions, but any occurance of a symbol. For instance, you want all places where 'foo' gets called, but not any 'foo', but the method 'foo' from class 'bar' and not from any other class. > The definition of xref-find-definitions is agnostic with respect to > these use cases; the "identifier" returned by xref--read-identifier can > be: > > - a simple string for the first use case > > - a more complex structure (in text properties) that identifies this > occurance for the second and third use case > > In addition, the backend can use the extra info or ignore it. > > But the xref-find-definitions UI doesn't allow the user to express a > desire for one these uses. I can't really say much about xref since I haven't really worked with it yet. The semantic-symref feature is also pretty limited in this regard. So in the end, I often ended up calling grep or global directly. All this stuff is highly dependent on the language you're using. > In any case, the "source search path" used by xref-find-definitions > should include all of the relevant projects, as defined by the project > dependencies. I agree. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-12 10:10 ` David Engster @ 2015-08-12 13:49 ` Stephen Leake 2015-08-12 15:36 ` David Engster 0 siblings, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-12 13:49 UTC (permalink / raw) To: emacs-devel David Engster <deng@randomsample.de> writes: > Stephen Leake writes: >> Depends on the use case; the user might want to see: >> >> - "all (overloaded) definitions of "foo", in all projects " >> >> Perhaps to check whether current occurance could be using a >> different definition. Or to see if the different definitions could >> be combined, or further refactored. >> >> This is more useful in languages that explicitly support overloaded >> symbols (Ada has many definitions of "Put" in its standard library). >> >> - "the definition for _this_ use of "foo", in some project " >> >> To see what it actually does. >> >> - "all methods for this generic function" >> >> Similar to the first use case, but limited to one generic function. >> >> So the question now is: what does EDE do for these use cases? > > AFAIK, you can only search in the current project. However, your use > cases involve more than just a project system. If you want to search for > overloads or methods, you need a parser as well. Yes. > This also applies to the most important feature (at least for me): > show all definitions of 'foo' that are currently in scope. EDE has no > notions of symbols, scope, overloading, methods, etc. Yes, I'm beginning to realize that. Semantic provides those things. And it can use different backends, so it can serve as the Emacs standard interface to such things (although I'd want to make significant changes to use it for Ada mode). In that sense, Semantic competes with xref in the same way EDE competes with project.el. > It gets much more complicated when you're not only interested in > definitions, but any occurance of a symbol. For instance, you want all > places where 'foo' gets called, but not any 'foo', but the method 'foo' > from class 'bar' and not from any other class. Right; that's name resolution; best done by the compiler. So hopefully it can output cross reference information (or provide access to it in some way). > I can't really say much about xref since I haven't really worked with it > yet. The semantic-symref feature is also pretty limited in this > regard. So in the end, I often ended up calling grep or global > directly. Right. > All this stuff is highly dependent on the language you're using. The backend may be, but at the API level, there are some things that all object oriented languages support: - Search for all implementations of this method. That's the language used by elisp cl-generic and eieio defclass. I just made this the default behavior for xref-find-definitions. Although that's not tested on eieio classes; I'll add that to my list. In Ada, it's called "functions that override primitive operations". In C++, it's "functions that override a virtual function". The terminology is different, but the underlying notion of runtime dispatch is the same. We might want to customize the menu entry depending on the current programming language. - Search for the parents of this type. "Type" is the Ada word; other languages use "class" here. - Search for all functions that overload this one. Ada and C++ (at least) support this. - Search for all references to this function. In C, these searches are simpler; the first three only return one result. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-12 13:49 ` Stephen Leake @ 2015-08-12 15:36 ` David Engster 2015-08-13 11:53 ` Nix 0 siblings, 1 reply; 101+ messages in thread From: David Engster @ 2015-08-12 15:36 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel Stephen Leake writes: > David Engster <deng@randomsample.de> writes: >> This also applies to the most important feature (at least for me): >> show all definitions of 'foo' that are currently in scope. EDE has no >> notions of symbols, scope, overloading, methods, etc. > > Yes, I'm beginning to realize that. Semantic provides those things. And > it can use different backends, so it can serve as the Emacs standard > interface to such things (although I'd want to make significant changes > to use it for Ada mode). > > In that sense, Semantic competes with xref in the same way EDE competes > with project.el. There is some overlap, especially in the GUI area for presenting references. But at least I would be happy to give that up in favor of xref, since CEDET was always planned to be more on the backend side of things, and getting the GUI right is nasty, and maintaining it takes a lot of time I'd rather spend elsewhere. So I'm happy that Dmitry is working on this (also with maintaining company-mode, which IMO should become part of core Emacs rather sooner than later). With respect to project.el, I'm still not sure where this is going. >> It gets much more complicated when you're not only interested in >> definitions, but any occurance of a symbol. For instance, you want all >> places where 'foo' gets called, but not any 'foo', but the method 'foo' >> from class 'bar' and not from any other class. > > Right; that's name resolution; best done by the compiler. Yeah well, we had that discussion. I hope I find the time to work on that. Currently, there are more urgent issue, like getting CEDET mergeable again after the switch to git and the extensive EIEIO refactoring... >> All this stuff is highly dependent on the language you're using. > > The backend may be, but at the API level, there are some things that all > object oriented languages support: > > - Search for all implementations of this method. > > That's the language used by elisp cl-generic and eieio defclass. I > just made this the default behavior for xref-find-definitions. > Although that's not tested on eieio classes; I'll add that to my > list. > > In Ada, it's called "functions that override primitive operations". > > In C++, it's "functions that override a virtual function". > > The terminology is different, but the underlying notion of runtime > dispatch is the same. > > We might want to customize the menu entry depending on the current > programming language. As always, it is more complicated in C++ (overload vs override, namespaces, argument dependent lookup, etc.), but currently I don't worry too much about details. Getting the usual dispatch right would cover a lot of ground. > - Search for the parents of this type. > > "Type" is the Ada word; other languages use "class" here. > > - Search for all functions that overload this one. > > Ada and C++ (at least) support this. > > - Search for all references to this function. Yes. > In C, these searches are simpler; the first three only return one > result. Aah, the joys of ANSI C. Our parser is actually pretty good for that. :-) -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-12 15:36 ` David Engster @ 2015-08-13 11:53 ` Nix 2015-08-13 12:05 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Nix @ 2015-08-13 11:53 UTC (permalink / raw) To: David Engster; +Cc: Stephen Leake, emacs-devel On 12 Aug 2015, David Engster verbalised: > Stephen Leake writes: >> - Search for all implementations of this method. >> >> That's the language used by elisp cl-generic and eieio defclass. I >> just made this the default behavior for xref-find-definitions. >> Although that's not tested on eieio classes; I'll add that to my >> list. >> >> In Ada, it's called "functions that override primitive operations". >> >> In C++, it's "functions that override a virtual function". >> >> The terminology is different, but the underlying notion of runtime >> dispatch is the same. >> >> We might want to customize the menu entry depending on the current >> programming language. > > As always, it is more complicated in C++ (overload vs override, > namespaces, argument dependent lookup, etc.), but currently I don't > worry too much about details. Getting the usual dispatch right would > cover a lot of ground. Quite. Given that name lookup can be literally arbitrarily complicated in C++, the only way to get it guaranteed right there involves writing a C++ compiler in elisp or compiler assistance -- and even that is kind of hopeless, because of course CEDET has to routinely work with uncompilable and only-partially-written buffers. (This is a fairly strong argument *against* relying on the compiler, in fact -- we'll always want something like Semantic, because we can tell it to be less pedantic in its parsing, recover as much as possible from every parse error, and so on.) >> In C, these searches are simpler; the first three only return one >> result. > > Aah, the joys of ANSI C. Our parser is actually pretty good for > that. :-) Now, of course, we have C11's _Generic to make things more interesting :) -- NULL && (void) ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-13 11:53 ` Nix @ 2015-08-13 12:05 ` Dmitry Gutov 2015-08-14 11:52 ` Eric Ludlam 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-13 12:05 UTC (permalink / raw) To: Nix, David Engster; +Cc: Stephen Leake, emacs-devel On 08/13/2015 02:53 PM, Nix wrote: > Quite. Given that name lookup can be literally arbitrarily complicated > in C++, the only way to get it guaranteed right there involves writing a > C++ compiler in elisp or compiler assistance -- and even that is kind of > hopeless, because of course CEDET has to routinely work with > uncompilable and only-partially-written buffers. There are compilers written with editing and incomplete code in mind. Clang, of course, performs some at least error recovery, or else it would be useless for code completion. > (This is a fairly > strong argument *against* relying on the compiler, in fact -- we'll > always want something like Semantic, because we can tell it to be less > pedantic in its parsing, recover as much as possible from every parse > error, and so on.) Can we? I'm guessing Semantic has at most one level of being "less pedantic", and implementing several of them, as well as support for those in grammars, wouldn't be trivial. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-13 12:05 ` Dmitry Gutov @ 2015-08-14 11:52 ` Eric Ludlam 2015-08-14 22:30 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Eric Ludlam @ 2015-08-14 11:52 UTC (permalink / raw) To: Dmitry Gutov, Nix, David Engster; +Cc: Stephen Leake, emacs-devel On 08/13/2015 08:05 AM, Dmitry Gutov wrote: > On 08/13/2015 02:53 PM, Nix wrote: >> (This is a fairly >> strong argument *against* relying on the compiler, in fact -- we'll >> always want something like Semantic, because we can tell it to be less >> pedantic in its parsing, recover as much as possible from every parse >> error, and so on.) > > Can we? > > I'm guessing Semantic has at most one level of being "less pedantic", > and implementing several of them, as well as support for those in > grammars, wouldn't be trivial. While Semantic does have an implementation of Bison as it's core for some languages, it is quite unlike most compilers. You can think of it has having 4 ways of handling errors the grammar author doesn't have to put effort into: 1) Lexical 'block' symbols The lexer can identify blocks, which would normally be handled by the parser. This is because Emacs has built in block matching that is quite good. This lets the parser completely ignore anything in { code } blocks or ( paramater lists ) unless it is important. As long as your block indicators match up, it doesn't matter how crappy your code is. 2) Parsers don't check types, etc as they go, and are flexible about the symbols they accept. This means you can get irrational tags. This is for a combination of handling incomplete code, speed (less to do) and because most folks just go an run their compiler at some point anyway. 3) Iterative parsing, so each invocation of the parser only executes across a single tag. This means that if parsing fails, we just move forward one lexical token and try again. This lets the parser just wash over anything it doesn't get. 4) Incremental parser Emacs watches you edit, tracks your changes, and matches it up against previously parsed tags. If things are 'broken' in the code you edit, but you didn't touch the boundaries of the other nearby tags, it lets it be until next time. This means all the previously captured data is available for use, including the active broken tag you might be editing. It also has the typical bison error handling, but that tends not to be necessary. In summary; Semantic can extract useful data out of pretty badly trashed code, and track your changes while you edit in a robust way. Eric ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-14 11:52 ` Eric Ludlam @ 2015-08-14 22:30 ` Dmitry Gutov 2015-08-15 0:48 ` Eric Ludlam 2015-08-15 7:05 ` Eli Zaretskii 0 siblings, 2 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-14 22:30 UTC (permalink / raw) To: Eric Ludlam, Nix, David Engster; +Cc: Stephen Leake, emacs-devel On 08/14/2015 02:52 PM, Eric Ludlam wrote: > 2) Parsers don't check types, etc as they go, and are flexible about the > symbols they accept. > > This means you can get irrational tags. This is for a combination of > handling incomplete code, speed (less to do) and because most folks just > go an run their compiler at some point anyway. Doesn't that mean that code completion at the end of a chain of calls can be inaccurate (with e.g. lots of false positives), even if code is otherwise valid? > 3) Iterative parsing, so each invocation of the parser only executes > across a single tag. > > This means that if parsing fails, we just move forward one lexical token > and try again. This lets the parser just wash over anything it doesn't > get. AFAIK, parsers in general try to ignore errors, or else you wouldn't be able to get any useful information, after compilation fails, about any errors after the first one. > 4) Incremental parser > > Emacs watches you edit, tracks your changes, and matches it up against > previously parsed tags. If things are 'broken' in the code you edit, > but you didn't touch the boundaries of the other nearby tags, it lets it > be until next time. This means all the previously captured data is > available for use, including the active broken tag you might be editing. That sounds useful. > It also has the typical bison error handling, but that tends not to be > necessary. > > In summary; Semantic can extract useful data out of pretty badly trashed > code, and track your changes while you edit in a robust way. It seems I stand corrected, thanks. However, it would be interesting to do a comparison on some real file, against a widely used compiler, such as Clang. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-14 22:30 ` Dmitry Gutov @ 2015-08-15 0:48 ` Eric Ludlam 2015-08-15 7:05 ` Eli Zaretskii 1 sibling, 0 replies; 101+ messages in thread From: Eric Ludlam @ 2015-08-15 0:48 UTC (permalink / raw) To: Dmitry Gutov, Eric Ludlam, Nix, David Engster; +Cc: Stephen Leake, emacs-devel On 08/14/2015 06:30 PM, Dmitry Gutov wrote: > On 08/14/2015 02:52 PM, Eric Ludlam wrote: > >> 2) Parsers don't check types, etc as they go, and are flexible about the >> symbols they accept. >> >> This means you can get irrational tags. This is for a combination of >> handling incomplete code, speed (less to do) and because most folks just >> go an run their compiler at some point anyway. > > Doesn't that mean that code completion at the end of a chain of calls > can be inaccurate (with e.g. lots of false positives), even if code is > otherwise valid? While the parser generally ignores data types, it does record the declarations. When smart completion is invoked, only the types currently relevant are resolved in order to make the completions accurate. Thus: Parsing phase: Just record the data as declared, don't check. Completion phase: Look up data types as needed for symbols in the completion string. If point of fact, Semantic's tagging parser doesn't parse the bodies of functions, and just skips over them. The body is only parsed if a completion is needed since the symbols within are locally scoped and don't affect other tags. On a side note, I've had a lot of complaints that completion was too pedantic, and if it can't find a pedantic answer, perhaps it could let loose a little and find a vaguely close answer. >> >>> In summary; Semantic can extract useful data out of pretty badly >>> trashed >>> code, and track your changes while you edit in a robust way. >> >> It seems I stand corrected, thanks. However, it would be interesting >> to do a comparison on some real file, against a widely used compiler, >> such as Clang. >> That would be interesting. What would the goal be? There are lots of things Semantic doesn't do, nor was intended for. Like many Emacs tools, the Semantic parser is a bit liberal in the assumptions it makes to get a good enough answer. Eric ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-14 22:30 ` Dmitry Gutov 2015-08-15 0:48 ` Eric Ludlam @ 2015-08-15 7:05 ` Eli Zaretskii 1 sibling, 0 replies; 101+ messages in thread From: Eli Zaretskii @ 2015-08-15 7:05 UTC (permalink / raw) To: Dmitry Gutov; +Cc: nix, emacs-devel, stephen_leake, deng, eric > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Sat, 15 Aug 2015 01:30:41 +0300 > Cc: Stephen Leake <stephen_leake@stephe-leake.org>, emacs-devel@gnu.org > > > 3) Iterative parsing, so each invocation of the parser only executes > > across a single tag. > > > > This means that if parsing fails, we just move forward one lexical token > > and try again. This lets the parser just wash over anything it doesn't > > get. > > AFAIK, parsers in general try to ignore errors, or else you wouldn't be > able to get any useful information, after compilation fails, about any > errors after the first one. Compiler parsers indeed do that, but AFAIK not at lexical token granularity. I think they throw away much larger portions of the parse tree before trying to resync. IME, when a compiler reports an error, it also frequently reports additional bogus errors about unrelated symbols. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-09 12:17 ` David Engster 2015-08-09 15:55 ` Stephen Leake @ 2015-08-10 17:12 ` Nix 1 sibling, 0 replies; 101+ messages in thread From: Nix @ 2015-08-10 17:12 UTC (permalink / raw) To: David Engster; +Cc: Stephen Leake, emacs-devel On 9 Aug 2015, David Engster outgrape: > Stephen Leake writes: >> Nix <nix@esperi.org.uk> writes: >>> On 3 Aug 2015, Stephen Leake told this: >>>> I prefer to cache it in a global variable; that way, I have only one >>>> project active at a time, for all buffers. >>> >>> That doesn't work very well for those of us working on multi-component >>> projects. >> >> Fine; of course this behavior should be user-configurable. >> >> At the same time, I work on multi-component projects all the time, and >> one active project works for me. >> >>> If I'm working in the Linux kernel I do *not* want identifier >>> search or file lookup wandering into glibc or Emacs, even though they >>> share some identifiers (with completely divergent definitions) and even >>> though I may have quite a lot of buffers open on those projects. >> >>> (I have fairly often worked on things that affect glibc, binutils, GCC, >>> and the kernel all at once. Those are different projects with distinct >>> roots, identifier sets, and tags tables...) >> >> But if you are in glibc, looking at an identifier for a function that >> happens to be defined in the linux kernel, you want to be able to find >> the corresponding definition; the linux kernel is a subproject of >> glibc (which is a subproject of higher level projects like Emacs). > > I must say I find this to be a very unusual view. In this specific case, I'd call it strictly incorrect. I picked those examples for a reason: they're an example of two sorta-related projects which define identical identifiers which relate to *different things* in each project: the kernel is in effect a freestanding environment and *does not* depend on glibc (indeed, the opposite is true, glibc depends on the kernel, even though it cannot resolve any symbols in it and no identifier lookups should ever traverse from the one to the other). e.g. the kernel defines many of the *printf() functions, and so does glibc -- but they define them in different places. If something inside glibc references snprintf(), the identifier lookup should proceed within glibc alone: if something inside the kernel references snprintf(), it had better find the copy in the kernel's lib/vsprintf.c. (Meanwhile, there are a lot of projects that really *do* depend on each other and symbol resolution in one can look up symbols in the other: it would be nice if identifier search could follow that, but it is very far from crucial.) >> So the Emacs project tools should reflect that. For example, file name >> completion must handle duplicate file names. That's a problem right now >> in Emacs for elisp files; there are both lisp/cedet/ede/dired.el and >> lisp/dired.el in Emacs core. > > This is a different problem. You are within one project here; of course > a project system should be able to handle this. Yes indeed. -- NULL && (void) ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 1:19 ` Eric Ludlam 2015-08-03 16:16 ` Stephen Leake @ 2015-08-03 22:47 ` Dmitry Gutov 2015-08-04 11:52 ` Eric Ludlam 1 sibling, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 22:47 UTC (permalink / raw) To: Eric Ludlam, Emacs Development Sorry for the late reply. On 08/03/2015 04:19 AM, Eric Ludlam wrote: > I apologize for not being up on the full thread. It was very long and I > didn't really get what the goals were. I would be lying if I said I got everything figured out. One "unified project interface" proposal surfaced a while back, and eventually fizzled out. I decided to revisit the subject when I was writing a couple of commands that needed some information about the current project, and here we are. Those turned out to be pretty useful, by the way. > One of my hopes for Emacs is that it will keep tabs on what I'm up to to > provide useful context relevant feedback when I ask for it. That sounds like something Flymake or Flycheck would do. Or a completion system. > My > experience has been that tools that depend on a project to ask what the > current project is do so a lot. As such, I cache the current project, > and current root project as buffer local variables. Those project > structures then have relevant meta data as well. Another tactic is > initializing each buffer into a project as it is created. That moves > search time into a different timing that is less noticeable than > on-demand when the user makes a request. IME, (vc-call-backend backend 'root dir) takes a negligible amount of time, so we might as well call it once per command. Of course, it might be different when using Tramp, but both Tramp and VC do some caching of their own. Other project implementations are free to employ their own caching methods (might be buffer-local, or if the implementation considers that only one project might be enabled at a time, simply in its own variables). > Also, identifying projects should be a service of the core project > system. Dependent projects need to provide detection data without > having Emacs load the full project definition. That way you can have > lots of projects types without forcing Emacs to load piles of code that > you don't need. Each project-find-functions element is free to only contain the bare necessary logic to identify its project type, and autoload the rest only as required. > By way of example, EDE's autoloader does this well, and is wrapped in a > complex test suite to make sure things don't get loaded when detecting > projects of a different type. I re-wrote most of it last year and it is > much better than it was previously. Use of 'locate-dominating-file' was > a big win. From a quick glance, it defines some restrictions on how a project could be detected, and then adds some reusable logic to optimize that. I'm not really clear on what those restrictions are. Maybe we'll want them, maybe not, we'll have to see. > Anyway, all that is just about matching projects. Projects then also > have meta data. Things like compilation configuration, compile > commands, release commands, include paths, classpaths, and whatnot. I > don't know what you intend with project meta data. If there was a > thread about that, I missed it, sorry. Probably a generic function called project-atrributes that returns a hash. Some keys could be standardized. includes path and class path roughtly correspond to project-search-path. > Thanks for the pointer. My Emacs25 didn't compile when I pulled from > git today, so I didn't get to try it or go very far beyond reading the > latest in project.el. That's odd, I didn't have any particular problems compiling it recently. > For your implementation in ede.el, you are welcome to use what I wrote. I'm not sure I understand. There's an implementation there already. Is your code better? How? > I think EDE has a generic API for defining a project. It probably looks > big on the outside if you just read the doc where it starts with > "ede-new", There is a lot to read there. One of the value propositions of project.el is that it's easier to grasp. Another distinction is that project.el is not an API for defining a project. It's an API for doing something with an already-defined project. > but for most cases, if you enable EDE, it will find your > Emacs src, Linux src, Cmake based build, etc, and it will magically > provide that to Semantic's parser, to location functions (if enabled) > and to taggers like global (if enabled.) The reason it looks big is > because those are the things that were needed while I was working on > other tools that needed project support. I would guess your system will > grow to a similar state if you keep adding features needed by tools that > want a project. It is a really big task. Maybe. So far, out of that list, the "information for taggers" seems like something that could be exposed in the API. The information that Semantic needs is probably not generic enough. The rest are just project types, with different source directories. > Someone pointed out on the cedet mailing list recently that the doc (and > probably the website) is too focused on how EDE can create makefiles > which most people don't want. Most questions on my mailing list ask how > to get some other tool like smart completion in semantic working, and > when they find out that they just need to turn on EDE, and not use the > full project management system, they are pretty happy. That's good. In my experience, the CEDET tools are quite dependent on each other. Semantic doesn't work for the languages I use, so a lot of value proposition goes out of the window. >> probably because it doesn't match every kind of project). Projectile >> seems to the most popular "project implementation" out there. > > Projectile is a nice interface for moving around between files. Some other commands it defines: projectile-run-shell-command-in-root projectile-find-tag projectile-multi-occur projectile-replace projectile-regenerate-tags projectile-grep projectile-test-project projectile-run-project I figure most of them could be re-implemented on top of a generic API. > Projectile has definitions for > projects EDE does not have because various CEDET developers haven't > worked with those tools. The opposite is probably true too. Probably. Projectile is also 2675 lines long, in total. > If EDE is considered specific because it is called ede, or because it > uses eieio, that would be too bad. It's purpose is to generically > detect projects such that anyone could create a new project definition, > find project metadata, and make it accessible to other tools. This has > already happened many times. Supporting EDE is more difficult than just > providing a dominating file name because it also wants to find extra > meta data. I think that is a pretty generic task also. All I know for sure is that I find EDE's code hard to read (and CEDET as a whole, to be honest), and there are not too many third-party packages that have decided to build on EDE. Maybe a simpler API will sway a larger crowd. > One way EDE could be made more generic is to make the 'target' concept > optional. Simple ede projects also have to define targets as part of > the detection and matching system, so making them optional would > simplify creating new projects types in EDE. It's pretty hard to take something out of an existing API, or a class hierarchy. And you can say these methods are optional, but they still take up space, and possible consumers are still going to see them. > Another way EDE could be made more generic is by having built-in tools > like the 'compile' command ask EDE how to compile, or by having flymake > use EDE to autodetect how to run make, or perhaps etags could ask EDE > where to run etags from, and a host of other options. I had pulled > those features out into ede named features to keep things independent > when CEDET was an independent tool. I don't see why naming conventions > couldn't change to make things better. These sounds like good possible applications for project.el. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 22:47 ` Dmitry Gutov @ 2015-08-04 11:52 ` Eric Ludlam 2015-08-04 16:09 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Eric Ludlam @ 2015-08-04 11:52 UTC (permalink / raw) To: Dmitry Gutov, Eric Ludlam, Emacs Development On 08/03/2015 06:47 PM, Dmitry Gutov wrote: >> My >> experience has been that tools that depend on a project to ask what the >> current project is do so a lot. As such, I cache the current project, >> and current root project as buffer local variables. Those project >> structures then have relevant meta data as well. Another tactic is >> initializing each buffer into a project as it is created. That moves >> search time into a different timing that is less noticeable than >> on-demand when the user makes a request. > > IME, (vc-call-backend backend 'root dir) takes a negligible amount of > time, so we might as well call it once per command. Of course, it > might be different when using Tramp, but both Tramp and VC do some > caching of their own. Other project implementations are free to employ > their own caching methods (might be buffer-local, or if the > implementation considers that only one project might be enabled at a > time, simply in its own variables). That is true now for project.el. Once you support 20+ project definitions as in EDE where vc is just one of them, all that negligible time adds up. Asking all 20+ project definitions to do their own caching is something EDE used to do, and it was a pain to maintain. I recommend against it. Eric ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 11:52 ` Eric Ludlam @ 2015-08-04 16:09 ` Dmitry Gutov 0 siblings, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-04 16:09 UTC (permalink / raw) To: Eric Ludlam, Emacs Development On 08/04/2015 02:52 PM, Eric Ludlam wrote: > That is true now for project.el. Once you support 20+ project > definitions as in EDE where vc is just one of them, all that negligible > time adds up. I honestly expect that project-find-functions, for each individual user, will almost never reach even 10 items, let alone 20. > Asking all 20+ project definitions to do their own caching is something > EDE used to do, and it was a pain to maintain. I recommend against it. You were able to do caching thanks to allowing some restrictions on how the current project depends on the current buffer (basically, along the lines of locate-dominating-file). Just a day or two ago, Stephen Leake posted that he was writing a project implementation that's active everywhere, as soon as the user visits the project. Caching is probably still possible, even in view of that, but there are more things to consider. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-02 17:17 ` Dmitry Gutov 2015-08-03 1:19 ` Eric Ludlam @ 2015-08-03 13:49 ` David Engster 2015-08-03 14:09 ` Dmitry Gutov ` (2 more replies) 2015-08-05 1:29 ` Eric Ludlam 2 siblings, 3 replies; 101+ messages in thread From: David Engster @ 2015-08-03 13:49 UTC (permalink / raw) To: Dmitry Gutov; +Cc: Emacs Development, Eric Ludlam Dmitry Gutov writes: > On 08/02/2015 04:52 PM, Eric Ludlam wrote: >> Perhaps all the work you're doing could just hang off ede since it only >> takes 2 lines of code, and if it is missing some sort of pruning tricks, >> it could just be added to ede. > > project.el defines a generic API. EDE is one implementation. From what I see in project.el, this is an API for defining a set of directories. I'm not saying that Emacs does not need such an API, but I would not call this a "project API". What about things like setting up toolchains (compiler, linker, debugger), configurations (debug/release), support for external build systems, setting up environment variables and pre-processor symbols, and so on? I mean, this is the bread and butter of projects in pretty much any IDE. Will you add this to project,el eventually? Otherwise, calling EDE an implementation of your generic project API is pretty daring, IMHO. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 13:49 ` David Engster @ 2015-08-03 14:09 ` Dmitry Gutov 2015-08-03 14:27 ` David Engster 2015-08-03 15:09 ` Eli Zaretskii 2015-08-03 16:25 ` Stephen Leake 2015-08-03 21:33 ` Stefan Monnier 2 siblings, 2 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 14:09 UTC (permalink / raw) To: David Engster; +Cc: Emacs Development, Eric Ludlam On 08/03/2015 04:49 PM, David Engster wrote: > From what I see in project.el, this is an API for defining a set of > directories. I'm not saying that Emacs does not need such an API, but I > would not call this a "project API". These are the traits common to most projects that we've identified so far. Ones that a third-party package will almost always be able to use. > What about things like setting up > toolchains (compiler, linker, debugger), configurations (debug/release), > support for external build systems, setting up environment variables and > pre-processor symbols, and so on? What linker and pre-processor? Seriously, you're talking to a Ruby developer here. What aspects of "setting up a debugger" would you consider generalizable over different project systems? Same question about "support for external build systems". Do you need help setting up environment variables? If that comes down to editing a list, then I suppose "environment variables" could become a standard piece of project metadata. Especially if someone out there is interested in writing a env var editor. > I mean, this is the bread and butter > of projects in pretty much any IDE. Will you add this to project,el > eventually? If people such as yourself help identify more pieces of knowledge that apply to most projects out there, I'd be happy to add them. > Otherwise, calling EDE an implementation of your generic > project API is pretty daring, IMHO. Well, as you've put it, this is obviously false. EDE is *an* implementation. Like some class can implement some (often small) interface. This metaphor should be familiar to a C++ programmer. Aside from that, EDE is a project system obviously geared towards C/C++ development. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 14:09 ` Dmitry Gutov @ 2015-08-03 14:27 ` David Engster 2015-08-03 15:13 ` Dmitry Gutov 2015-08-03 16:35 ` Stephen Leake 2015-08-03 15:09 ` Eli Zaretskii 1 sibling, 2 replies; 101+ messages in thread From: David Engster @ 2015-08-03 14:27 UTC (permalink / raw) To: Dmitry Gutov; +Cc: Eric Ludlam, Emacs Development Dmitry Gutov writes: > On 08/03/2015 04:49 PM, David Engster wrote: > >> From what I see in project.el, this is an API for defining a set of >> directories. I'm not saying that Emacs does not need such an API, but I >> would not call this a "project API". > > These are the traits common to most projects that we've identified so > far. Ones that a third-party package will almost always be able to > use. As I've said: I'm not saying it's not useful. >> What about things like setting up >> toolchains (compiler, linker, debugger), configurations (debug/release), >> support for external build systems, setting up environment variables and >> pre-processor symbols, and so on? > > What linker and pre-processor? Seriously, you're talking to a Ruby > developer here. Just because Ruby does not have them, project.el should not have support for different toolchains? > What aspects of "setting up a debugger" would you consider > generalizable over different project systems? A debugger needs to know where the executable is and how it should be called. > Same question about "support for external build systems". If you say "compile this project", you will usually not call the compiler directly but run some external build systems. Depending on the current configuration (debug/release), it must be called differently. > Do you need help setting up environment variables? Yes. I might need to set things like CFLAGS dependend on the current configuration. Or I might want to cross-compile, in which case I need to set ARCH and CROSS_COMPILE. >> of projects in pretty much any IDE. Will you add this to project,el >> eventually? > > If people such as yourself help identify more pieces of knowledge that > apply to most projects out there, I'd be happy to add them. I've mentioned a few. If you add support for this, your simple API will grow, and I wouldn't be surprised if in the end it looks a bit like EDE. > Aside from that, EDE is a project system obviously geared towards > C/C++ development. It can (and does) support other languages. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 14:27 ` David Engster @ 2015-08-03 15:13 ` Dmitry Gutov 2015-08-03 21:35 ` David Engster 2015-08-03 16:35 ` Stephen Leake 1 sibling, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 15:13 UTC (permalink / raw) To: David Engster; +Cc: Eric Ludlam, Emacs Development On 08/03/2015 05:27 PM, David Engster wrote: >> What linker and pre-processor? Seriously, you're talking to a Ruby >> developer here. > > Just because Ruby does not have them, project.el should not have support > for different toolchains? I'm not sure yet how to fit the toolchain-specialized aspects best. Maybe we'll have a set of methods that are optional to implement. Or different "types" of projects the consumer will have to "cast" the project object to. In any case, you'll need to present a use case first. If that information will only be consumed by functions inside EDE itself, there's not much use in making it a part of the general API. >> What aspects of "setting up a debugger" would you consider >> generalizable over different project systems? > > A debugger needs to know where the executable is and how it should be > called. Just for your reference, the ways I tend to use the debugger in Ruby projects are quite different from calling it with a path to the executable. Or consider some Java web application project. To launch it in debug mode, you pass an argument to the application launcher, and then an application (maybe the IDE itself) connects to it via a socket. > If you say "compile this project", you will usually not call the > compiler directly but run some external build systems. Depending on the > current configuration (debug/release), it must be called differently. Is there something there to be automated in Emacs, that will save the user a lot of effort, compared to 'make env=DEBUG' in the console? > Yes. I might need to set things like CFLAGS dependend on the current > configuration. Or I might want to cross-compile, in which case I need to > set ARCH and CROSS_COMPILE. So, the API would not only list the variables, but also somehow describe the possible values. >> If people such as yourself help identify more pieces of knowledge that >> apply to most projects out there, I'd be happy to add them. > > I've mentioned a few. I hope you can see that there's more design work to be done that just mentioning them. > If you add support for this, your simple API will > grow, and I wouldn't be surprised if in the end it looks a bit like EDE. I guess we'll have to see. But for now, compare the 140-line long project.el and 12000 lines in ede.el and ede/*.el combined. >> Aside from that, EDE is a project system obviously geared towards >> C/C++ development. > > It can (and does) support other languages. It contains a lot of stuff that's useless from a Ruby developer's standpoint (or Python, or Elisp, or R), and it doesn't compensate that with a lot of features. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 15:13 ` Dmitry Gutov @ 2015-08-03 21:35 ` David Engster 2015-08-03 23:21 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: David Engster @ 2015-08-03 21:35 UTC (permalink / raw) To: Dmitry Gutov; +Cc: Eric Ludlam, Emacs Development Dmitry Gutov writes: > On 08/03/2015 05:27 PM, David Engster wrote: > >>> What linker and pre-processor? Seriously, you're talking to a Ruby >>> developer here. >> >> Just because Ruby does not have them, project.el should not have support >> for different toolchains? > > I'm not sure yet how to fit the toolchain-specialized aspects > best. Maybe we'll have a set of methods that are optional to > implement. You mean like ede-compile-project? > In any case, you'll need to present a use case first. I hit a key, the project gets compiled by calling some build command based on the currently selected configuration. > If that information will only be consumed by functions inside EDE > itself, there's not much use in making it a part of the general API. I don't understand this. >>> What aspects of "setting up a debugger" would you consider >>> generalizable over different project systems? >> >> A debugger needs to know where the executable is and how it should be >> called. > > Just for your reference, the ways I tend to use the debugger in Ruby > projects are quite different from calling it with a path to the > executable. > > Or consider some Java web application project. To launch it in debug > mode, you pass an argument to the application launcher, and then an > application (maybe the IDE itself) connects to it via a socket. Yes, the notion of "debugging the project" varies wildly; same goes for "running the executable". You can also add remote debugging on a target platform through a debug server. Why not try to support it all? >> If you say "compile this project", you will usually not call the >> compiler directly but run some external build systems. Depending on the >> current configuration (debug/release), it must be called differently. > > Is there something there to be automated in Emacs, that will save the > user a lot of effort, compared to 'make env=DEBUG' in the console? Every switch away from Emacs is a distraction. These are things I have to do *many* times a day. Besides, I need the Make output in a compilation buffer, so that I can jump to errors, etc. >> Yes. I might need to set things like CFLAGS dependend on the current >> configuration. Or I might want to cross-compile, in which case I need to >> set ARCH and CROSS_COMPILE. > > So, the API would not only list the variables, but also somehow > describe the possible values. The user can switch between different configurations, and dependend on the current configuration a certain set of environment variables is used when the build system is called. >>> If people such as yourself help identify more pieces of knowledge that >>> apply to most projects out there, I'd be happy to add them. >> >> I've mentioned a few. > > I hope you can see that there's more design work to be done that just > mentioning them. Of course, but since I'm quite happy with EDE, that's not an itch I need to scratch. You volunteered to create a project API, and I simply state what I would expect from such an API to be usable for me, as an EDE user. I don't want to sound so negative, and I surely don't want to discourage your efforts. A simple API that defines a certain set of directories based on the current buffer could cover a lot of what users need. While I still think that EDE could support this, it has the huge drawback that it cannot be automatically loaded at startup because of EIEIO. >> If you add support for this, your simple API will >> grow, and I wouldn't be surprised if in the end it looks a bit like EDE. > > I guess we'll have to see. But for now, compare the 140-line long > project.el and 12000 lines in ede.el and ede/*.el combined. You are comparing what is essentially an interface definition, covering a tiny amount of the EDE API, against the whole EDE suite. That factor is hardly surprising. I'm the first to say that EDE needs a serious overhaul. I also wouldn't object to simply require that the user already has a working build through whatever build system she chooses (make, ninja, ant, gradle, rake, you name it), which would mean we could drop a lot of stuff like the automatic Makefile generation. >>> Aside from that, EDE is a project system obviously geared towards >>> C/C++ development. >> >> It can (and does) support other languages. > > It contains a lot of stuff that's useless from a Ruby developer's > standpoint (or Python, or Elisp, or R), and it doesn't compensate that > with a lot of features. So if those features are only needed by compiled languages they don't count? This indeed will keep things simple... -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 21:35 ` David Engster @ 2015-08-03 23:21 ` Dmitry Gutov 2015-08-04 8:15 ` David Engster 2015-08-04 13:43 ` Eli Zaretskii 0 siblings, 2 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 23:21 UTC (permalink / raw) To: David Engster; +Cc: Eric Ludlam, Emacs Development On 08/04/2015 12:35 AM, David Engster wrote: > You mean like ede-compile-project? I suppose. And here we find out that EDE took over the project- namespace, a while ago. That's not nice. > I hit a key, the project gets compiled by calling some build command > based on the currently selected configuration. Sounds fine to me. The respective method should probably return the command to compile, not perform compilation itself. >> If that information will only be consumed by functions inside EDE >> itself, there's not much use in making it a part of the general API. > > I don't understand this. If the only functions that would use the information are included in CEDET, and there's no plans to extract them, there's no point in making them go through the project.el API, since they already know they're dealing with EDE. > Yes, the notion of "debugging the project" varies wildly; same goes for > "running the executable". You can also add remote debugging on a target > platform through a debug server. Why not try to support it all? I'm not particularly against it, just saying it's not easy to do in both generic and useful way. >> So, the API would not only list the variables, but also somehow >> describe the possible values. > > The user can switch between different configurations, and dependend on > the current configuration a certain set of environment variables is used > when the build system is called. Okay, the complications seem to outweigh the benefits here. > Of course, but since I'm quite happy with EDE, that's not an itch I need > to scratch. You volunteered to create a project API, and I simply state > what I would expect from such an API to be usable for me, as an EDE > user. The API is not for a user. It's for other packages. If you're happily using EDE, and CEDET, and not much else, then it's unlikely to make your life better. > You are comparing what is essentially an interface definition, covering > a tiny amount of the EDE API, against the whole EDE suite. That factor > is hardly surprising. Yes, and that's the point. The interface definition should be clear and as short as possible. > So if those features are only needed by compiled languages they don't > count? This indeed will keep things simple... How useful are those things for Haskell? Or ML? Initially yet, we'll probably avoid those (if only because they're not a priority for me personally), but generally useful compiled-language-related things can be added later on. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 23:21 ` Dmitry Gutov @ 2015-08-04 8:15 ` David Engster 2015-08-04 13:43 ` Eli Zaretskii 1 sibling, 0 replies; 101+ messages in thread From: David Engster @ 2015-08-04 8:15 UTC (permalink / raw) To: Dmitry Gutov; +Cc: Eric Ludlam, Emacs Development Dmitry Gutov writes: > On 08/04/2015 12:35 AM, David Engster wrote: > >> You mean like ede-compile-project? > > I suppose. > > And here we find out that EDE took over the project- namespace, a > while ago. That's not nice. That can be changed. > The API is not for a user. It's for other packages. If you're happily > using EDE, and CEDET, and not much else, then it's unlikely to make > your life better. OK. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 23:21 ` Dmitry Gutov 2015-08-04 8:15 ` David Engster @ 2015-08-04 13:43 ` Eli Zaretskii 2015-08-04 18:05 ` Dmitry Gutov 1 sibling, 1 reply; 101+ messages in thread From: Eli Zaretskii @ 2015-08-04 13:43 UTC (permalink / raw) To: Dmitry Gutov; +Cc: emacs-devel, deng, eric > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Tue, 4 Aug 2015 02:21:08 +0300 > Cc: Eric Ludlam <eric@siege-engine.com>, > Emacs Development <emacs-devel@gnu.org> > > >> So, the API would not only list the variables, but also somehow > >> describe the possible values. > > > > The user can switch between different configurations, and dependend on > > the current configuration a certain set of environment variables is used > > when the build system is called. > > Okay, the complications seem to outweigh the benefits here. I really think we could benefit from presenting some kind of goal or vision for the project.el development. AFAICS, it started as a way to standardize finding a "project root", then moved to finding "project directories", but now it seems we are discussing a much wider scope, something like infrastructure for project definition and possibly also IDEs? If it's the latter, then I won't expect to see arguments about "complications outweighing benefits" in reference to features that are basic for any IDE that is worth its menus. So maybe we should step back a notch and decide anew where is project.el going, and what is and isn't in its scope? ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 13:43 ` Eli Zaretskii @ 2015-08-04 18:05 ` Dmitry Gutov 2015-08-04 18:16 ` Eli Zaretskii 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-04 18:05 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, deng, eric On 08/04/2015 04:43 PM, Eli Zaretskii wrote: > I really think we could benefit from presenting some kind of goal or > vision for the project.el development. The vision is: identify general pieces of information that a interesting Lisp program can use without worrying too much about which kind of a project it is, and add them to the API in a way that's both easy to use and (hopefully) implement. Simply identifying pieces of information doesn't help until you have an application in mind. Which, at this point, will often imply "intend to implement that application yourself". There's no good way to verify usefulness of a piece of API aside from creating at least an quick-and-dirty, example consumer. > AFAICS, it started as a way to > standardize finding a "project root", then moved to finding "project > directories", but now it seems we are discussing a much wider scope, > something like infrastructure for project definition and possibly also > IDEs? Much of the recent discussion centered on the question of whether EDE is good enough. David also questioned whether the scope (of type-agnostic project operations) is a wide one. > If it's the latter, then I won't expect to see arguments about > "complications outweighing benefits" in reference to features that are > basic for any IDE that is worth its menus. It was a quick judgment: David doesn't seem interested on working on extracting some general behavior out of it. I'm not captivated either. In any case, that was either a weak proposal, or a very incomplete one. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 18:05 ` Dmitry Gutov @ 2015-08-04 18:16 ` Eli Zaretskii 2015-08-04 18:41 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Eli Zaretskii @ 2015-08-04 18:16 UTC (permalink / raw) To: Dmitry Gutov; +Cc: emacs-devel, deng, eric > Cc: deng@randomsample.de, eric@siege-engine.com, emacs-devel@gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Tue, 4 Aug 2015 21:05:31 +0300 > > The vision is: identify general pieces of information that a interesting > Lisp program can use without worrying too much about which kind of a > project it is So project.el is supposed to allow extracting information from projects, but is not supposed to help in defining a project? ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 18:16 ` Eli Zaretskii @ 2015-08-04 18:41 ` Dmitry Gutov 2015-08-04 19:23 ` Eli Zaretskii 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-04 18:41 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, deng, eric On 08/04/2015 09:16 PM, Eli Zaretskii wrote: > So project.el is supposed to allow extracting information from > projects, but is not supposed to help in defining a project? If I understand you right, it is not. Project implementations don't, in general, consume the project API. It's not really for them. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 18:41 ` Dmitry Gutov @ 2015-08-04 19:23 ` Eli Zaretskii 2015-08-04 19:40 ` João Távora 2015-08-04 20:15 ` Dmitry Gutov 0 siblings, 2 replies; 101+ messages in thread From: Eli Zaretskii @ 2015-08-04 19:23 UTC (permalink / raw) To: Dmitry Gutov; +Cc: emacs-devel, deng, eric > Cc: deng@randomsample.de, eric@siege-engine.com, emacs-devel@gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Tue, 4 Aug 2015 21:41:18 +0300 > > On 08/04/2015 09:16 PM, Eli Zaretskii wrote: > > > So project.el is supposed to allow extracting information from > > projects, but is not supposed to help in defining a project? > > If I understand you right, it is not. It is not what? There are 2 parts to that sentence, which one are you refuting (or confirming)? If you are saying that both parts of what I said are correct, then why did you and Stephen spend so many time discussing how to define those pieces of information for a project? It shouldn't be part of the job; instead, you should look at what information is provided as part of defining a project by different means out there, for example, in a Makefile. E.g., there are a couple of standard ways to specify in a Makefile the directories where the project's files live. Other significant parts of a project's information in a Makefile are recipes to build the project, to install/uninstall it, to clean the tree, etc. Also, various standard commands, like compilation command, link command, a command that creates a library and a manual, etc. These are all clear candidates for "project information an interested Lisp program would like to extract", no? ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 19:23 ` Eli Zaretskii @ 2015-08-04 19:40 ` João Távora 2015-08-05 2:52 ` Eli Zaretskii 2015-08-04 20:15 ` Dmitry Gutov 1 sibling, 1 reply; 101+ messages in thread From: João Távora @ 2015-08-04 19:40 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, Eric Ludlam, deng, Dmitry Gutov On Tue, Aug 4, 2015 at 8:23 PM, Eli Zaretskii <eliz@gnu.org> wrote: >> On 08/04/2015 09:16 PM, Eli Zaretskii wrote: > E.g., there are a couple of standard ways to specify in a Makefile the > directories where the project's files live. Other significant parts > of a project's information in a Makefile are recipes to build the > project, to install/uninstall it, to clean the tree, etc. Also, > various standard commands, like compilation command, link command, a > command that creates a library and a manual, etc. > > These are all clear candidates for "project information an interested > Lisp program would like to extract", no? If I may go out on possibly a very long limb here, and answering your specific question my understanding of project.el is that it helps in writing a "project implementation backend". It is that backend that understands Makefiles and supplies these bits of information, via project.el's API to interested Lisp programs, like xref, perhaps "M-x compile" or some other project-wide command. I'd welcome that, provided it becomes popular enough that some useful backends actually do get written. But I may be misreading the whole discussion, it's indeed become a bit confusing. -- João Távora ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 19:40 ` João Távora @ 2015-08-05 2:52 ` Eli Zaretskii 0 siblings, 0 replies; 101+ messages in thread From: Eli Zaretskii @ 2015-08-05 2:52 UTC (permalink / raw) To: João Távora; +Cc: emacs-devel, eric, deng, dgutov > From: João Távora <joaotavora@gmail.com> > Date: Tue, 4 Aug 2015 20:40:08 +0100 > Cc: Dmitry Gutov <dgutov@yandex.ru>, emacs-devel <emacs-devel@gnu.org>, deng@randomsample.de, > Eric Ludlam <eric@siege-engine.com> > > If I may go out on possibly a very long limb here, and answering > your specific question my understanding of project.el is that it helps in > writing a "project implementation backend". No, I don't think it does that. I think it allows other Lisp programs to extract information from a project system that is already built. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 19:23 ` Eli Zaretskii 2015-08-04 19:40 ` João Távora @ 2015-08-04 20:15 ` Dmitry Gutov 2015-08-05 2:49 ` Eli Zaretskii 1 sibling, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-04 20:15 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, deng, eric On 08/04/2015 10:23 PM, Eli Zaretskii wrote: > It is not what? There are 2 parts to that sentence, which one are you > refuting (or confirming)? Confirming both, more or less. > If you are saying that both parts of what I said are correct, then why > did you and Stephen spend so many time discussing how to define those > pieces of information for a project? It shouldn't be part of the job; > instead, you should look at what information is provided as part of > defining a project by different means out there, for example, in a > Makefile. Some of that discussion was misunderstandings, some of it was arguing about semantics, and some of it was arguing about what's the best way to present a certain information in the API. There are many ways to present the same information. We've certainly discussed different project files and information contained (or missing) in them. Not Makefiles, though. > E.g., there are a couple of standard ways to specify in a Makefile the > directories where the project's files live. Other significant parts > of a project's information in a Makefile are recipes to build the > project, to install/uninstall it, to clean the tree, etc. Also, > various standard commands, like compilation command, link command, a > command that creates a library and a manual, etc. > > These are all clear candidates for "project information an interested > Lisp program would like to extract", no? I can't easily answer that. What's the use for "command that creates a manual" (even setting aside the fact that not all projects have that)? Calling it in M-x project-create-manual, a command that only does that one thing? Not very interesting, IMHO. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 20:15 ` Dmitry Gutov @ 2015-08-05 2:49 ` Eli Zaretskii 2015-08-05 6:18 ` Stephen Leake 2015-08-05 9:42 ` Dmitry Gutov 0 siblings, 2 replies; 101+ messages in thread From: Eli Zaretskii @ 2015-08-05 2:49 UTC (permalink / raw) To: Dmitry Gutov; +Cc: emacs-devel, deng, eric > Cc: deng@randomsample.de, eric@siege-engine.com, emacs-devel@gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Tue, 4 Aug 2015 23:15:16 +0300 > > > E.g., there are a couple of standard ways to specify in a Makefile the > > directories where the project's files live. Other significant parts > > of a project's information in a Makefile are recipes to build the > > project, to install/uninstall it, to clean the tree, etc. Also, > > various standard commands, like compilation command, link command, a > > command that creates a library and a manual, etc. > > > > These are all clear candidates for "project information an interested > > Lisp program would like to extract", no? > > I can't easily answer that. What's the use for "command that creates a > manual" (even setting aside the fact that not all projects have that)? > Calling it in M-x project-create-manual, a command that only does that > one thing? Yes, for starters. Building the whole project from scratch, including the manuals, is another. Finding out whether the project _has_ a manual building instructions is yet another. > Not very interesting, IMHO. I don't understand this criterion. I think the criterion should be "is this useful". ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 2:49 ` Eli Zaretskii @ 2015-08-05 6:18 ` Stephen Leake 2015-08-05 15:08 ` Eli Zaretskii 2015-08-05 9:42 ` Dmitry Gutov 1 sibling, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-05 6:18 UTC (permalink / raw) To: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Cc: deng@randomsample.de, eric@siege-engine.com, emacs-devel@gnu.org >> From: Dmitry Gutov <dgutov@yandex.ru> >> Date: Tue, 4 Aug 2015 23:15:16 +0300 >> >> > E.g., there are a couple of standard ways to specify in a Makefile the >> > directories where the project's files live. Other significant parts >> > of a project's information in a Makefile are recipes to build the >> > project, to install/uninstall it, to clean the tree, etc. Also, >> > various standard commands, like compilation command, link command, a >> > command that creates a library and a manual, etc. >> > >> > These are all clear candidates for "project information an interested >> > Lisp program would like to extract", no? >> >> I can't easily answer that. What's the use for "command that creates a >> manual" (even setting aside the fact that not all projects have that)? >> Calling it in M-x project-create-manual, a command that only does that >> one thing? > > Yes, for starters. Building the whole project from scratch, including > the manuals, is another. Finding out whether the project _has_ a > manual building instructions is yet another. > >> Not very interesting, IMHO. > > I don't understand this criterion. I think the criterion should be > "is this useful". Apparently EDE has the notion of "targets" for this sort of thing. So the EDE users find it useful, for actually running the targets. The question for project.el is whether is would be useful for _other_ (non-EDE) elisp programs; we already have EDE to create and run the targets. I can imagine a full-blown Configuration Management application (think IBM ClearCase) that manages the created binaries as well as the sources; it would want a list of the targets of a project, to ensure they were tracked in the CM database. But such a thing would probably not be written on top of project.el; it would be more like a new EDE project type, or a complete replacement for EDE. Short of that, I can't think of a non-EDE elisp program that would want to know about targets. There might be one someday; we can extend project.el then. There are probably other features in EDE that some non-EDE programs would want to know about; I'm still learning about EDE. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 6:18 ` Stephen Leake @ 2015-08-05 15:08 ` Eli Zaretskii 2015-08-05 15:36 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Eli Zaretskii @ 2015-08-05 15:08 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel > From: Stephen Leake <stephen_leake@stephe-leake.org> > Date: Wed, 05 Aug 2015 01:18:54 -0500 > > >> What's the use for "command that creates a manual" (even setting > >> aside the fact that not all projects have that)? Calling it in > >> M-x project-create-manual, a command that only does that one > >> thing? > > > > Yes, for starters. Building the whole project from scratch, including > > the manuals, is another. Finding out whether the project _has_ a > > manual building instructions is yet another. > > > >> Not very interesting, IMHO. > > > > I don't understand this criterion. I think the criterion should be > > "is this useful". > > Apparently EDE has the notion of "targets" for this sort of thing. So > the EDE users find it useful, for actually running the targets. > > The question for project.el is whether is would be useful for _other_ > (non-EDE) elisp programs; we already have EDE to create and run the > targets. If EDE finds this useful, what doubt is there that other Lisp programs will? How many examples do we need before we decide that a specific trait of a project is "useful"? I say one is enough (sometimes more than enough). > Short of that, I can't think of a non-EDE elisp program that would want > to know about targets. There might be one someday; we can extend > project.el then. That's exactly my problem: I find such an approach surprising, to say the least, when building such basic infrastructure. When you build infrastructure, you don't wait for clients to come requesting basic features. You are supposed to study the use cases and the entities you want to support, and provide support for at least the standard/basic aspects and traits of those entities. I think there should be small doubt that access to project documentation, including its build command, is one of these basics, because every project has that. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 15:08 ` Eli Zaretskii @ 2015-08-05 15:36 ` Dmitry Gutov 2015-08-05 16:31 ` Eli Zaretskii ` (2 more replies) 0 siblings, 3 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-05 15:36 UTC (permalink / raw) To: Eli Zaretskii, Stephen Leake; +Cc: emacs-devel On 08/05/2015 06:08 PM, Eli Zaretskii wrote: > If EDE finds this useful, what doubt is there that other Lisp programs > will? How many examples do we need before we decide that a specific > trait of a project is "useful"? I say one is enough (sometimes more > than enough). A trait should be useful *and* fairly common. Do you see someone in a hurry to rewrite EDE on top of project.el? I don't. So we have zero potential consumers so far. >> Short of that, I can't think of a non-EDE elisp program that would want >> to know about targets. There might be one someday; we can extend >> project.el then. > > That's exactly my problem: I find such an approach surprising, to say > the least, when building such basic infrastructure. On the other hand, it wastes less time on speculative features, and the API is almost guaranteed to turn out better in the end. > I think there > should be small doubt that access to project documentation, including > its build command, is one of these basics, because every project has > that. I don't even know what "access to project documentation" means. Is it just the build command? Or the paths of resulting files? Or their formats? Or an index? Or a way to display them? ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 15:36 ` Dmitry Gutov @ 2015-08-05 16:31 ` Eli Zaretskii 2015-08-05 16:45 ` David Engster 2015-08-06 7:54 ` Stephen Leake 2 siblings, 0 replies; 101+ messages in thread From: Eli Zaretskii @ 2015-08-05 16:31 UTC (permalink / raw) To: Dmitry Gutov; +Cc: stephen_leake, emacs-devel > Cc: emacs-devel@gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Wed, 5 Aug 2015 18:36:08 +0300 > > On 08/05/2015 06:08 PM, Eli Zaretskii wrote: > > > If EDE finds this useful, what doubt is there that other Lisp programs > > will? How many examples do we need before we decide that a specific > > trait of a project is "useful"? I say one is enough (sometimes more > > than enough). > > A trait should be useful *and* fairly common. It's common, just look at any IDE out there. > Do you see someone in a hurry to rewrite EDE on top of project.el? I > don't. So we have zero potential consumers so far. Like I said, I consider this kind of arguments irrelevant for developing infrastructure. Either it's important to develop infrastructure for accessing project traits, or it isn't. If it is (and I gather you decided it is), then the infrastructure should IMO be reasonably complete. > >> Short of that, I can't think of a non-EDE elisp program that would want > >> to know about targets. There might be one someday; we can extend > >> project.el then. > > > > That's exactly my problem: I find such an approach surprising, to say > > the least, when building such basic infrastructure. > > On the other hand, it wastes less time on speculative features, and the > API is almost guaranteed to turn out better in the end. Not IME. What I saw more than once is unfinished infrastructure that no one wanted to continue developing, because the person(s) who started moved on without arriving at some reasonably complete component. > > I think there > > should be small doubt that access to project documentation, including > > its build command, is one of these basics, because every project has > > that. > > I don't even know what "access to project documentation" means. Is it > just the build command? Or the paths of resulting files? Or their > formats? Or an index? Or a way to display them? There are a lot of IDEs out there to learn from and get answers to all these questions very quickly. IME, doing that kind of research is always part of any infrastructure development. It sounds like you would prefer waiting till someone comes up and teaches you/us all these answers, or presents clear requirements that allow you to formulate those answers, but IME that seldom happens, because people who already went through that research, or gained the answers from personal experience, are a few and far in-between, and they are unlikely to line up to help us, of all the projects. I went through such an experience myself years ago, when I started to work on bidirectional display. Anyway, I think my views on this are clear, so I think it's time for me to stop arguing about this. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 15:36 ` Dmitry Gutov 2015-08-05 16:31 ` Eli Zaretskii @ 2015-08-05 16:45 ` David Engster 2015-08-05 22:17 ` Dmitry Gutov 2015-08-06 7:56 ` Stephen Leake 2015-08-06 7:54 ` Stephen Leake 2 siblings, 2 replies; 101+ messages in thread From: David Engster @ 2015-08-05 16:45 UTC (permalink / raw) To: Dmitry Gutov; +Cc: Eli Zaretskii, Stephen Leake, emacs-devel Dmitry Gutov writes: > On 08/05/2015 06:08 PM, Eli Zaretskii wrote: > >> If EDE finds this useful, what doubt is there that other Lisp programs >> will? How many examples do we need before we decide that a specific >> trait of a project is "useful"? I say one is enough (sometimes more >> than enough). > > A trait should be useful *and* fairly common. > > Do you see someone in a hurry to rewrite EDE on top of project.el? What does that mean? Isn't project.el always at the top, fed by various project backends? -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 16:45 ` David Engster @ 2015-08-05 22:17 ` Dmitry Gutov 2015-08-06 7:56 ` Stephen Leake 1 sibling, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-05 22:17 UTC (permalink / raw) To: David Engster; +Cc: Eli Zaretskii, Stephen Leake, emacs-devel On 08/05/2015 07:45 PM, David Engster wrote: > What does that mean? Isn't project.el always at the top, fed by various > project backends? Sorry for being unclear. Probably not EDE's core, but maybe the integrations that build on it (like ede/speedbar, for instance), or parts of CEDET that use EDE. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 16:45 ` David Engster 2015-08-05 22:17 ` Dmitry Gutov @ 2015-08-06 7:56 ` Stephen Leake 1 sibling, 0 replies; 101+ messages in thread From: Stephen Leake @ 2015-08-06 7:56 UTC (permalink / raw) To: emacs-devel David Engster <deng@randomsample.de> writes: > Dmitry Gutov writes: >> On 08/05/2015 06:08 PM, Eli Zaretskii wrote: >> >>> If EDE finds this useful, what doubt is there that other Lisp programs >>> will? How many examples do we need before we decide that a specific >>> trait of a project is "useful"? I say one is enough (sometimes more >>> than enough). >> >> A trait should be useful *and* fairly common. >> >> Do you see someone in a hurry to rewrite EDE on top of project.el? > > What does that mean? Isn't project.el always at the top, fed by various > project backends? As I understand it, EDE provides both project backends and build functions that use information from the project backends. A future refactoring of EDE could use the Emacs project.el API to define the interface between the two. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 15:36 ` Dmitry Gutov 2015-08-05 16:31 ` Eli Zaretskii 2015-08-05 16:45 ` David Engster @ 2015-08-06 7:54 ` Stephen Leake 2 siblings, 0 replies; 101+ messages in thread From: Stephen Leake @ 2015-08-06 7:54 UTC (permalink / raw) To: emacs-devel Dmitry Gutov <dgutov@yandex.ru> writes: > Do you see someone in a hurry to rewrite EDE on top of project.el? I > don't. So we have zero potential consumers so far. I think Eli's point is valid here. EDE has an internal "project info" data structure and API. It's possibly not very welll separated from the rest of the EDE code. A future replacement for EDE (or a rewrite/refactoring of EDE) could use the standard Emacs project API, and thus gain instant access to all new project implementations. So we should at least plan on supporting that level of functionality. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 2:49 ` Eli Zaretskii 2015-08-05 6:18 ` Stephen Leake @ 2015-08-05 9:42 ` Dmitry Gutov 2015-08-05 15:23 ` Eli Zaretskii 2015-08-06 7:43 ` Stephen Leake 1 sibling, 2 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-05 9:42 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, deng, eric On 08/05/2015 05:49 AM, Eli Zaretskii wrote: > Yes, for starters. Building the whole project from scratch, including > the manuals, is another. Finding out whether the project _has_ a > manual building instructions is yet another. Project could define a way to build it whole, for instance, without giving out specific information about every goal of build process. Or exposing it all, in a standardized structure, without, however, having the list of specific targets a project can support (manuals, etc) pre-defined. >> Not very interesting, IMHO. > > I don't understand this criterion. I think the criterion should be > "is this useful". Just a criterion for what I spend time on. Do you really consider this on the same of usefulness as, say, xref-find-references? ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 9:42 ` Dmitry Gutov @ 2015-08-05 15:23 ` Eli Zaretskii 2015-08-05 15:31 ` Dmitry Gutov 2015-08-06 7:43 ` Stephen Leake 1 sibling, 1 reply; 101+ messages in thread From: Eli Zaretskii @ 2015-08-05 15:23 UTC (permalink / raw) To: Dmitry Gutov; +Cc: emacs-devel, deng, eric > Cc: deng@randomsample.de, eric@siege-engine.com, emacs-devel@gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Wed, 5 Aug 2015 12:42:07 +0300 > > Yes, for starters. Building the whole project from scratch, including > the manuals, is another. Finding out whether the project _has_ a > manual building instructions is yet another. > > Project could define a way to build it whole, for instance, without giving out specific information about every goal of build process. It could, but why would it want to? It sounds silly to have to rebuild everything when you only need to rebuild some (small) part. > Or exposing it all, in a standardized structure, without, however, having the list of specific targets a project can support (manuals, etc) pre-defined. Again, sounds silly, given how most, if not all, modern build tools work. But (shrug) not a catastrophe: the corresponding attribute will simply be nil, or the command to build everything. IOW, I see no problem here, and no reasons to consider this unfit for the API. > Not very interesting, IMHO. > > I don't understand this criterion. I think the criterion should be > "is this useful". > > Just a criterion for what I spend time on. I don't think you have this luxury when you work on infrastructure. E.g., company.el is "not very interesting" for me, but I still try very hard to fix every issue in the display engine that you or your users report. > Do you really consider this on the same of usefulness as, say, xref-find-references? Yes. They are both no-brainers to have in the infrastructure that AFAIU you are trying to provide. It might even be much more useful to a Lisp program that only cares about the manual of a project. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 15:23 ` Eli Zaretskii @ 2015-08-05 15:31 ` Dmitry Gutov 2015-08-05 16:16 ` Eli Zaretskii 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-05 15:31 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, deng, eric On 08/05/2015 06:23 PM, Eli Zaretskii wrote: > Again, sounds silly, given how most, if not all, modern build tools > work. That is counter to my experience. The modern build tools work in a variety of different ways. > But (shrug) not a catastrophe: the corresponding attribute will simply > be nil, or the command to build everything. And I don't understand what you mean here. > IOW, I see no problem here, and no reasons to consider this unfit for > the API. It's fit. As soon as we're sure someone is going to write a Lisp program making use of this part of the API. And by "sure", I mean we have at least a proof-of-concept patch for a non-trivial piece of functionality. > I don't think you have this luxury when you work on infrastructure. I don't think anyone here has the authority to tell me what to spend time on. > E.g., company.el is "not very interesting" for me, but I still try > very hard to fix every issue in the display engine that you or your > users report. You don't work on adding new features to it, however. > Yes. They are both no-brainers to have in the infrastructure that > AFAIU you are trying to provide. It might even be much more useful to > a Lisp program that only cares about the manual of a project. It makes sense to go for features with bigger impact first. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 15:31 ` Dmitry Gutov @ 2015-08-05 16:16 ` Eli Zaretskii 2015-08-06 6:44 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Eli Zaretskii @ 2015-08-05 16:16 UTC (permalink / raw) To: Dmitry Gutov; +Cc: emacs-devel, deng, eric > Cc: deng@randomsample.de, eric@siege-engine.com, emacs-devel@gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Wed, 5 Aug 2015 18:31:35 +0300 > > On 08/05/2015 06:23 PM, Eli Zaretskii wrote: > > > Again, sounds silly, given how most, if not all, modern build tools > > work. > > That is counter to my experience. The modern build tools work in a > variety of different ways. You mean, some of them don't support specifying how to build documentation? Which ones? > > But (shrug) not a catastrophe: the corresponding attribute will simply > > be nil, or the command to build everything. > > And I don't understand what you mean here. An attribute can exist even if with some projects its value will be nil or a trivial "build-all" command. IOW, even if some projects won't have any non-trivial value for this attribute, it's not an argument against the attribute's existence. > > IOW, I see no problem here, and no reasons to consider this unfit for > > the API. > > It's fit. As soon as we're sure someone is going to write a Lisp program > making use of this part of the API. And by "sure", I mean we have at > least a proof-of-concept patch for a non-trivial piece of functionality. I don't think it's correct to wait for specific requests for every single sub-feature, when developing infrastructure like this one. Some basic traits should be supported just because they are basic, before someone asks. > > I don't think you have this luxury when you work on infrastructure. > > I don't think anyone here has the authority to tell me what to spend > time on. I didn't tell you anything except my views on this. > > E.g., company.el is "not very interesting" for me, but I still try > > very hard to fix every issue in the display engine that you or your > > users report. > > You don't work on adding new features to it, however. Yes, I do. But that's besides the point, isn't it? > > Yes. They are both no-brainers to have in the infrastructure that > > AFAIU you are trying to provide. It might even be much more useful to > > a Lisp program that only cares about the manual of a project. > > It makes sense to go for features with bigger impact first. If this is just a matter of priority, I'm okay with whatever the decision is. I was under the impression that the mere need is being questioned. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 16:16 ` Eli Zaretskii @ 2015-08-06 6:44 ` Dmitry Gutov 0 siblings, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-06 6:44 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, deng, eric On 08/05/2015 07:16 PM, Eli Zaretskii wrote: >> That is counter to my experience. The modern build tools work in a >> variety of different ways. > > You mean, some of them don't support specifying how to build > documentation? Which ones? I didn't mean building docs in particular, but yes, kinda. You can build docs with Rake (and in different formats, depending on which ones you want), but there's no standard task name for that. > I don't think it's correct to wait for specific requests for every > single sub-feature, when developing infrastructure like this one. > Some basic traits should be supported just because they are basic, > before someone asks. I guess so. When we get around to the "building" aspect of projects, we'll try to make it reasonably complete. Without any build-related commands in Emacs, though, we'll still fall short of providing the complete editing experience you seem to want. And those would be exactly the kind of consumers I'm looking in order to grow the project API. >>> E.g., company.el is "not very interesting" for me, but I still try >>> very hard to fix every issue in the display engine that you or your >>> users report. >> >> You don't work on adding new features to it, however. > > Yes, I do. But that's besides the point, isn't it? Adding new features to Company, I mean. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 9:42 ` Dmitry Gutov 2015-08-05 15:23 ` Eli Zaretskii @ 2015-08-06 7:43 ` Stephen Leake 2015-08-06 10:25 ` Dmitry Gutov 1 sibling, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-06 7:43 UTC (permalink / raw) To: emacs-devel Dmitry Gutov <dgutov@yandex.ru> writes: > On 08/05/2015 05:49 AM, Eli Zaretskii wrote: > >>> Not very interesting, IMHO. >> >> I don't understand this criterion. I think the criterion should be >> "is this useful". > > Just a criterion for what I spend time on. Since we are discussing core Emacs functionality, you should be clearer about this. There is a big difference between: - that would be a useful core feature, but I don't want to work on it and - that would not be a useful core feature. > Do you really consider this on the same of usefulness as, say, > xref-find-references? Personally, I use 'make' much more than I use cross reference tools, so yes, build is more useful. For Emacs core, they are both important, which is why we have 'compile' and 'xref-find-*'. For project.el, this fits into "provide build related project information to other elisp programs" in a couple of ways: - 'compile' tries to provide tab completion on Makefile targets; it doesn't do it well (it misses targets in #include <makefile>). Presumably EDE can do better, so project.el should provide project-build-targets, so compile can use that to provide the completion list. That would then automatically work for ant, gradle, etc projects as those are added to EDE or another project implementation. - 'compile' also tries to guess the build command; it could use project-build-command. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-06 7:43 ` Stephen Leake @ 2015-08-06 10:25 ` Dmitry Gutov 2015-08-06 14:27 ` Stephen Leake 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-06 10:25 UTC (permalink / raw) To: Stephen Leake, emacs-devel On 08/06/2015 10:43 AM, Stephen Leake wrote: > Since we are discussing core Emacs functionality, you should be clearer > about this. There is a big difference between: I doesn't seem particularly useful, and I don't want to work on it. But there's no reason to assume that I'm going to veto it (do I even have a veto power?), if someone's going to implement it in a clean way. > Personally, I use 'make' much more than I use cross reference tools, so > yes, build is more useful. You can still run 'make' in the console, but cross-referencing needs an editor to function adequately. > - 'compile' tries to provide tab completion on Makefile targets; it > doesn't do it well (it misses targets in #include <makefile>). > Presumably EDE can do better, so project.el should provide > project-build-targets, so compile can use that to provide the completion > list. That would then automatically work for ant, gradle, etc projects > as those are added to EDE or another project implementation. If project-build-targets just returns a list of strings, that would be insufficient to run the build. So we need a more informative structure. name, description, full command to run. Maybe also cram running build targets in different environments in there somehow. > - 'compile' also tries to guess the build command; it could use > project-build-command. Maybe all required info will be in project-build-targets. Or we'll just have project-default-target, which will return a simple string. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-06 10:25 ` Dmitry Gutov @ 2015-08-06 14:27 ` Stephen Leake 2015-08-06 23:16 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-06 14:27 UTC (permalink / raw) To: emacs-devel Dmitry Gutov <dgutov@yandex.ru> writes: > On 08/06/2015 10:43 AM, Stephen Leake wrote: > >> Since we are discussing core Emacs functionality, you should be clearer >> about this. There is a big difference between: > > I doesn't seem particularly useful, and I don't want to work on it. > But there's no reason to assume that I'm going to veto it (do I even > have a veto power?), if someone's going to implement it in a clean > way. > >> Personally, I use 'make' much more than I use cross reference tools, so >> yes, build is more useful. > > You can still run 'make' in the console, `next-error' is a very powerful Emacs feature; nothing equivalent is available in a console. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-06 14:27 ` Stephen Leake @ 2015-08-06 23:16 ` Dmitry Gutov 2015-08-07 14:10 ` Stephen Leake 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-06 23:16 UTC (permalink / raw) To: Stephen Leake, emacs-devel On 08/06/2015 05:27 PM, Stephen Leake wrote: >> You can still run 'make' in the console, > > `next-error' is a very powerful Emacs feature; nothing equivalent is > available in a console. Point taken. Still, I wouldn't want to settle on a version of project-build-tasks over-specialized for, say, for C/C++ Makefile-based projects. The best way to do that is by implementing a flexible "run task" command. M-x compile could use the API for its default value, but I don't think it should be the focus when adding the new feature. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-06 23:16 ` Dmitry Gutov @ 2015-08-07 14:10 ` Stephen Leake 2015-08-07 14:44 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-07 14:10 UTC (permalink / raw) To: emacs-devel Dmitry Gutov <dgutov@yandex.ru> writes: > On 08/06/2015 05:27 PM, Stephen Leake wrote: > >>> You can still run 'make' in the console, >> >> `next-error' is a very powerful Emacs feature; nothing equivalent is >> available in a console. > > Point taken. > > Still, I wouldn't want to settle on a version of project-build-tasks > over-specialized for, say, for C/C++ Makefile-based projects. The best > way to do that is by implementing a flexible "run task" command. That's what 'compile' _is_; it runs any command in a subprocess. How much more flexible can you get? -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-07 14:10 ` Stephen Leake @ 2015-08-07 14:44 ` Dmitry Gutov 0 siblings, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-07 14:44 UTC (permalink / raw) To: Stephen Leake, emacs-devel On 08/07/2015 05:10 PM, Stephen Leake wrote: > That's what 'compile' _is_; it runs any command in a subprocess. How > much more flexible can you get? Well, if you're sure about that. To continue this discussion, we'll need a proof-of-concept patch with new method(s) in project.el, a EDE-based implementation (I guess?) and the changes to the compile command that make use of the new information and at least provide completion for task names. Maybe also allows switching between different environments to compile in (test/development/debug/release/etc). ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 14:27 ` David Engster 2015-08-03 15:13 ` Dmitry Gutov @ 2015-08-03 16:35 ` Stephen Leake 2015-08-03 16:45 ` Eli Zaretskii 1 sibling, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-03 16:35 UTC (permalink / raw) To: emacs-devel David Engster <deng@randomsample.de> writes: > Dmitry Gutov writes: >> On 08/03/2015 04:49 PM, David Engster wrote: >> >>> From what I see in project.el, this is an API for defining a set of >>> directories. I'm not saying that Emacs does not need such an API, but I >>> would not call this a "project API". >> >> These are the traits common to most projects that we've identified so >> far. Ones that a third-party package will almost always be able to >> use. > > As I've said: I'm not saying it's not useful. > >>> What about things like setting up >>> toolchains (compiler, linker, debugger), configurations (debug/release), >>> support for external build systems, setting up environment variables and >>> pre-processor symbols, and so on? >> >> What linker and pre-processor? Seriously, you're talking to a Ruby >> developer here. > > Just because Ruby does not have them, project.el should not have support > for different toolchains? It depends on whether there are really common features across several toolchains. I guess the top-level API for these things in EDE is common across several toolchains, but I'm not clear; you've talked about some things are in EDE 'only for Java', or only for some other backend. I agree the core API should allow all possible backends, but there's a trade-off. If the core includes stuff that only one or a few backends need, then that imposes extra overhead (if only by overloading the implementer understanding of the API). >> What aspects of "setting up a debugger" would you consider >> generalizable over different project systems? > > A debugger needs to know where the executable is and how it should be > called. gud does that, but if a new debugger needs to be added, perhaps gud should consult the project system first, rather than having a separate way to add a debugger. > If you say "compile this project", you will usually not call the > compiler directly but run some external build systems. Depending on the > current configuration (debug/release), it must be called differently. Right. I do that by editing Makefiles directly; M-x compile is always "make -r <target>" for me. I've always felt that anything else just gets in the way. Clearly I'm in the minority on this one :(>. Although I've recently started working on JDEE, and it doesn't have a Makefile, just an ant build.xml. So I'm trying to force myself to not add a Makefile. We'll see. So far it's very frustrating trying to track down exactly what ant is doing; must easier with a makefile. >> Do you need help setting up environment variables? > > Yes. I might need to set things like CFLAGS dependend on the current > configuration. Or I might want to cross-compile, in which case I need to > set ARCH and CROSS_COMPILE. These seem very backend specific. How much of this is really in the EDE core, and how much in some "make/gcc backend"? -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 16:35 ` Stephen Leake @ 2015-08-03 16:45 ` Eli Zaretskii 2015-08-03 21:07 ` Stephen Leake 0 siblings, 1 reply; 101+ messages in thread From: Eli Zaretskii @ 2015-08-03 16:45 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel > From: Stephen Leake <stephen_leake@stephe-leake.org> > Date: Mon, 03 Aug 2015 11:35:11 -0500 > > > A debugger needs to know where the executable is and how it should be > > called. > > gud does that No, it doesn't. It leaves it to the user to type the exact file name every single time. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 16:45 ` Eli Zaretskii @ 2015-08-03 21:07 ` Stephen Leake 2015-08-03 21:33 ` David Engster 2015-08-04 2:35 ` Eli Zaretskii 0 siblings, 2 replies; 101+ messages in thread From: Stephen Leake @ 2015-08-03 21:07 UTC (permalink / raw) To: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Stephen Leake <stephen_leake@stephe-leake.org> >> Date: Mon, 03 Aug 2015 11:35:11 -0500 >> >> > A debugger needs to know where the executable is and how it should be >> > called. >> >> gud does that > > No, it doesn't. It leaves it to the user to type the exact file name > every single time. Ah; I was confusing "the debugger exectuable" and "the user executable being debugged". It does remember the last file debugged, so it's not "every single time" :). In addition, when I build things in makefiles, gud does a good job of guessing the correct user executable, and the others are available thru file completion (since they are all in the same directory). I don't see how a project can do better; there will still be a list of possible executables; the most recently compiled isn't necessarily the one I want to debug. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 21:07 ` Stephen Leake @ 2015-08-03 21:33 ` David Engster 2015-08-04 2:35 ` Eli Zaretskii 1 sibling, 0 replies; 101+ messages in thread From: David Engster @ 2015-08-03 21:33 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel Stephen Leake writes: > In addition, when I build things in makefiles, gud does a good job of > guessing the correct user executable, and the others are available thru > file completion (since they are all in the same directory). Not for me. Executables are all over the place and I always have to make sure I switch to the correct directory first. Gud frequently guesses wrong, probably because we have a very complicated parallel build. Also, I frequently have to switch between debugging locally and remotely on the target platform through gdbserver. > I don't see how a project can do better; there will still be a list of > possible executables; the most recently compiled isn't necessarily the > one I want to debug. Well, let's take Eclipse. It automatically scans the tree for binaries which you can quickly select for debugging. You can define different debug configurations where you can define the debugger to use, how it should be called and with which init file, if and where it should stop execution on startup, which executable it should debug and what arguments and environment it should provide, and if maybe this executable should be copied to a target system and debugged through gdbserver... And that's just what *I* regularly use. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 21:07 ` Stephen Leake 2015-08-03 21:33 ` David Engster @ 2015-08-04 2:35 ` Eli Zaretskii 1 sibling, 0 replies; 101+ messages in thread From: Eli Zaretskii @ 2015-08-04 2:35 UTC (permalink / raw) To: Stephen Leake; +Cc: emacs-devel > From: Stephen Leake <stephen_leake@stephe-leake.org> > Date: Mon, 03 Aug 2015 16:07:17 -0500 > > In addition, when I build things in makefiles, gud does a good job of > guessing the correct user executable It does? For me, it always suggests config.bat as the default when I want to debug Emacs. > I don't see how a project can do better; there will still be a list of > possible executables; the most recently compiled isn't necessarily the > one I want to debug. A list of executables I may wish to debug is much shorter than the list of all executable files. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 14:09 ` Dmitry Gutov 2015-08-03 14:27 ` David Engster @ 2015-08-03 15:09 ` Eli Zaretskii 2015-08-03 15:16 ` Dmitry Gutov 2015-08-04 11:48 ` Eric Ludlam 1 sibling, 2 replies; 101+ messages in thread From: Eli Zaretskii @ 2015-08-03 15:09 UTC (permalink / raw) To: Dmitry Gutov; +Cc: eric, deng, emacs-devel > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Mon, 3 Aug 2015 17:09:29 +0300 > Cc: Emacs Development <emacs-devel@gnu.org>, > Eric Ludlam <eric@siege-engine.com> > > > What about things like setting up > > toolchains (compiler, linker, debugger), configurations (debug/release), > > support for external build systems, setting up environment variables and > > pre-processor symbols, and so on? > > What linker and pre-processor? Seriously, you're talking to a Ruby > developer here. I sincerely hope the results of this development will support more than just Ruby. Most languages today, even those that are "normally" interpreted, have some kind of compiler. And even Ruby might grow a compiler some day. I think supporting that is a no-brainer. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 15:09 ` Eli Zaretskii @ 2015-08-03 15:16 ` Dmitry Gutov 2015-08-03 15:29 ` Eli Zaretskii 2015-08-04 11:48 ` Eric Ludlam 1 sibling, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 15:16 UTC (permalink / raw) To: Eli Zaretskii; +Cc: eric, deng, emacs-devel On 08/03/2015 06:09 PM, Eli Zaretskii wrote: > I sincerely hope the results of this development will support more > than just Ruby. Obviously. > Most languages today, even those that are "normally" > interpreted, have some kind of compiler. And even Ruby might grow a > compiler some day. I think supporting that is a no-brainer. For all I know, when talking about "supporting a compiler", you and David have entirely different use cases in mind. It'll need a concrete design proposal. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 15:16 ` Dmitry Gutov @ 2015-08-03 15:29 ` Eli Zaretskii 2015-08-03 19:01 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Eli Zaretskii @ 2015-08-03 15:29 UTC (permalink / raw) To: Dmitry Gutov; +Cc: eric, deng, emacs-devel > Cc: deng@randomsample.de, emacs-devel@gnu.org, eric@siege-engine.com > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Mon, 3 Aug 2015 18:16:44 +0300 > > > Most languages today, even those that are "normally" > > interpreted, have some kind of compiler. And even Ruby might grow a > > compiler some day. I think supporting that is a no-brainer. > > For all I know, when talking about "supporting a compiler", you and > David have entirely different use cases in mind. I don't think so, because everything David said made perfect sense to me. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 15:29 ` Eli Zaretskii @ 2015-08-03 19:01 ` Dmitry Gutov 2015-08-03 19:19 ` Eli Zaretskii 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 19:01 UTC (permalink / raw) To: Eli Zaretskii; +Cc: eric, deng, emacs-devel On 08/03/2015 06:29 PM, Eli Zaretskii wrote: > I don't think so, because everything David said made perfect sense to > me. IIUC, (and I'm mostly guessing), you want the project to report on the program a debugger should run. To do that, we can introduce a project-attributes dict, and it may, or may not, include a key `debugger-program'. This looks like a low-cost, low-benefit proposal. Anything else? ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 19:01 ` Dmitry Gutov @ 2015-08-03 19:19 ` Eli Zaretskii 2015-08-03 21:05 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Eli Zaretskii @ 2015-08-03 19:19 UTC (permalink / raw) To: Dmitry Gutov; +Cc: eric, deng, emacs-devel > Cc: deng@randomsample.de, emacs-devel@gnu.org, eric@siege-engine.com > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Mon, 3 Aug 2015 22:01:17 +0300 > > IIUC, (and I'm mostly guessing), you want the project to report on the > program a debugger should run. David mentioned more than just that. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 19:19 ` Eli Zaretskii @ 2015-08-03 21:05 ` Dmitry Gutov 0 siblings, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 21:05 UTC (permalink / raw) To: Eli Zaretskii; +Cc: eric, deng, emacs-devel On 08/03/2015 10:19 PM, Eli Zaretskii wrote: > David mentioned more than just that. Unfortunately, I couldn't make out any other specific proposals. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 15:09 ` Eli Zaretskii 2015-08-03 15:16 ` Dmitry Gutov @ 2015-08-04 11:48 ` Eric Ludlam 2015-08-04 16:20 ` Dmitry Gutov 1 sibling, 1 reply; 101+ messages in thread From: Eric Ludlam @ 2015-08-04 11:48 UTC (permalink / raw) To: Eli Zaretskii, Dmitry Gutov; +Cc: eric, deng, emacs-devel On 08/03/2015 11:09 AM, Eli Zaretskii wrote: >> >What linker and pre-processor? Seriously, you're talking to a Ruby >> >developer here. > I sincerely hope the results of this development will support more > than just Ruby. Most languages today, even those that are "normally" > interpreted, have some kind of compiler. And even Ruby might grow a > compiler some day. I think supporting that is a no-brainer. > . > To flip things around, what would it take to have CEDET support ruby? I don't know anything about it, but if you just need to identify the root of some vc identified directory tree, then it already does. Eric ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 11:48 ` Eric Ludlam @ 2015-08-04 16:20 ` Dmitry Gutov 0 siblings, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-04 16:20 UTC (permalink / raw) To: Eric Ludlam, Eli Zaretskii; +Cc: eric, deng, emacs-devel On 08/04/2015 02:48 PM, Eric Ludlam wrote: > To flip things around, what would it take to have CEDET support ruby? Semantic support seems paramount (semantic-symref depends on it). However, it seems to be a losing proposition: writing a grammar for Ruby is hard enough (there's no official grammar for the current version of Ruby). IIRC, there was some incomplete version of grammar for it in CEDET upstream, which doesn't work (at all? well? I don't remember exactly), and even if someone were to spend the effort on fixing it: - It won't be included in Emacs, unless that someone wrote the grammar from scratch. - It won't help with code completion, which is considered the primary goal of Semantic (AFAICT), because the latter lacks facilities to handle type inference, or types in dynamic languages in general. > I don't know anything about it, but if you just need to identify the > root of some vc identified directory tree, then it already does. Yes, thanks, I don't need to load a 12000-lines project to do that. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 13:49 ` David Engster 2015-08-03 14:09 ` Dmitry Gutov @ 2015-08-03 16:25 ` Stephen Leake 2015-08-03 21:33 ` Stefan Monnier 2 siblings, 0 replies; 101+ messages in thread From: Stephen Leake @ 2015-08-03 16:25 UTC (permalink / raw) To: emacs-devel David Engster <deng@randomsample.de> writes: > From what I see in project.el, this is an API for defining a set of > directories. I'm not saying that Emacs does not need such an API, but I > would not call this a "project API". What about things like setting up > toolchains (compiler, linker, debugger), Ada mode uses AdaCore gprbuild, which uses a project file that defines all that stuff. No help from Emacs needed. And so far, the only thing Emacs needs from the project file is the source path, for compilation-source-path. > configurations (debug/release), support for external build systems, To support a build system, you provide a project backend. I assume I could provide an EDE backend for gprbuild, and another for gradle (for Android Java projects). > setting up environment variables and pre-processor symbols, No environment variables for Ada, Java only needs CLASSPATH, which is set from the JDEE project file (so that could use an EDE feature?). No preprocessor in Ada or Java. > and so on? I mean, this is the bread and butter of projects in pretty > much any IDE. I suspect there may be self-selection going on here; people/project backends that need all of those features use EDE, and new project backends get written. But for people/projects that don't need all of those things, EDE seems overwhelming. So it seems like we are discussing refactoring EDE, and adding some features it doesn't currently have. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 13:49 ` David Engster 2015-08-03 14:09 ` Dmitry Gutov 2015-08-03 16:25 ` Stephen Leake @ 2015-08-03 21:33 ` Stefan Monnier 2015-08-03 22:15 ` David Engster 2 siblings, 1 reply; 101+ messages in thread From: Stefan Monnier @ 2015-08-03 21:33 UTC (permalink / raw) To: David Engster; +Cc: Emacs Development, Eric Ludlam, Dmitry Gutov > From what I see in project.el, this is an API for defining a set of > directories. I'm not saying that Emacs does not need such an API, but I > would not call this a "project API". What about things like setting up > toolchains (compiler, linker, debugger), configurations (debug/release), > support for external build systems, setting up environment variables and > pre-processor symbols, and so on? The point of view is very different: project.el is meant to provide to the rest of Emacs the information needed to perform operations that require knowledge about "the current project". So the API is designed as a kind of "intersection (rather than union) of all possible project systems". Currently the "rest of Emacs" part that uses project knowledge is: - text search within the project - xref thingies We could/should/will extend the API if there's a need for it in order to implement further generic functionality. I can imagine a few such generic functions: - "make" - "run" - flymake I don't think any of those requires exposing concepts like toolchains or compilers in the API (tho any particular implementation of that API may/will probably be tailored to some particular toolchain/compiler and/or will need to be taught bout the toolchain/compiler setup). Stefan ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 21:33 ` Stefan Monnier @ 2015-08-03 22:15 ` David Engster 2015-08-03 22:50 ` Dmitry Gutov 2015-08-04 7:13 ` Stefan Monnier 0 siblings, 2 replies; 101+ messages in thread From: David Engster @ 2015-08-03 22:15 UTC (permalink / raw) To: Stefan Monnier; +Cc: Emacs Development, Eric Ludlam, Dmitry Gutov Stefan Monnier writes: >> From what I see in project.el, this is an API for defining a set of >> directories. I'm not saying that Emacs does not need such an API, but I >> would not call this a "project API". What about things like setting up >> toolchains (compiler, linker, debugger), configurations (debug/release), >> support for external build systems, setting up environment variables and >> pre-processor symbols, and so on? > > The point of view is very different: project.el is meant to provide to > the rest of Emacs the information needed to perform operations that > require knowledge about "the current project". And what constitutes "the current project", exactly? Only the list of directories associated with it? > So the API is designed as a kind of "intersection (rather than union) of > all possible project systems". That intersection is tiny. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 22:15 ` David Engster @ 2015-08-03 22:50 ` Dmitry Gutov 2015-08-04 7:13 ` Stefan Monnier 1 sibling, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-03 22:50 UTC (permalink / raw) To: David Engster, Stefan Monnier; +Cc: Eric Ludlam, Emacs Development On 08/04/2015 01:15 AM, David Engster wrote: > And what constitutes "the current project", exactly? Only the list of > directories associated with it? That definition probably varies between different technologies. And we look for the common ground. > That intersection is tiny. That's a matter of opinion. I find it more interesting than a lot of the other stuff you've mentioned. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-03 22:15 ` David Engster 2015-08-03 22:50 ` Dmitry Gutov @ 2015-08-04 7:13 ` Stefan Monnier 2015-08-04 8:13 ` David Engster 2015-08-04 9:40 ` Stephen Leake 1 sibling, 2 replies; 101+ messages in thread From: Stefan Monnier @ 2015-08-04 7:13 UTC (permalink / raw) To: David Engster; +Cc: Eric Ludlam, Dmitry Gutov, Emacs Development > And what constitutes "the current project", exactly? That's for the backend(s) to decide. > Only the list of directories associated with it? Not necessarily, tho so far it's the only thing that a backend needs to expose. >> So the API is designed as a kind of "intersection (rather than union) of >> all possible project systems". > That intersection is tiny. Do you find this to be a problem? If so, why? This is not meant to replace/reimplement EDE, but only to provide a common API between different systems, EDE being hopefully one of them. Stefan ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 7:13 ` Stefan Monnier @ 2015-08-04 8:13 ` David Engster 2015-08-05 13:42 ` Stefan Monnier 2015-08-04 9:40 ` Stephen Leake 1 sibling, 1 reply; 101+ messages in thread From: David Engster @ 2015-08-04 8:13 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eric Ludlam, Dmitry Gutov, Emacs Development Stefan Monnier writes: >>> So the API is designed as a kind of "intersection (rather than union) of >>> all possible project systems". >> That intersection is tiny. > > Do you find this to be a problem? If so, why? It's a very limited view of a project, so other packages who want to work on a project cannot depend on much information unless they ask one backend directly. > This is not meant to replace/reimplement EDE, but only to provide > a common API between different systems, EDE being hopefully one of them. OK. Let's see how it evolves. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 8:13 ` David Engster @ 2015-08-05 13:42 ` Stefan Monnier 2015-08-06 11:27 ` {Spam?} " Eric Ludlam 0 siblings, 1 reply; 101+ messages in thread From: Stefan Monnier @ 2015-08-05 13:42 UTC (permalink / raw) To: David Engster; +Cc: Eric Ludlam, Dmitry Gutov, Emacs Development > It's a very limited view of a project, so other packages who want to > work on a project cannot depend on much information unless they ask one > backend directly. Yes, it's very limited, simply because there hasn't be any other such "other package" which required something else. As mentioned I can see things like "flymake" imposing extra requirements (mostly a new method to compile some file(s) and return the corresponding compilation messages). Stefan ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: {Spam?} Re: progmodes/project.el and search paths 2015-08-05 13:42 ` Stefan Monnier @ 2015-08-06 11:27 ` Eric Ludlam 2015-08-06 23:10 ` Stefan Monnier 0 siblings, 1 reply; 101+ messages in thread From: Eric Ludlam @ 2015-08-06 11:27 UTC (permalink / raw) To: Stefan Monnier, David Engster Cc: Eric Ludlam, Dmitry Gutov, Emacs Development On 08/05/2015 09:42 AM, Stefan Monnier wrote: >> It's a very limited view of a project, so other packages who want to >> work on a project cannot depend on much information unless they ask one >> backend directly. > Yes, it's very limited, simply because there hasn't be any other such > "other package" which required something else. As mentioned I can see > things like "flymake" imposing extra requirements (mostly a new method > to compile some file(s) and return the corresponding compilation > messages). > I would think that backend projects can also depend on services from a generic project system. Comint provides a generic way to interact with a shell command. Using comint is easier than starting from scratch and makes no assumptions by providing for a wide range of features that might be needed by sub shells. A side effect is that every subordinate shell buffer in Emacs is consistent, configuring options in comint works everywhere, and keybindings are the same. Another side effect is that a you can write services that work with comint that thus work in many different shells. Wouldn't it be cool if users could write things that feel like project management backend features that work with multiple back-ends. It would be nice if the generic base project were the same. Any project backend you use would have similar keybindings, menus, detection mechanisms, file location systems, etc because it will be easier to use the base support than invent something new. Eric ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: {Spam?} Re: progmodes/project.el and search paths 2015-08-06 11:27 ` {Spam?} " Eric Ludlam @ 2015-08-06 23:10 ` Stefan Monnier 2015-08-07 11:18 ` Eric Ludlam 0 siblings, 1 reply; 101+ messages in thread From: Stefan Monnier @ 2015-08-06 23:10 UTC (permalink / raw) To: Eric Ludlam; +Cc: Eric Ludlam, Dmitry Gutov, David Engster, Emacs Development > It would be nice if the generic base project were the same. Any project > backend you use would have similar keybindings, menus, detection mechanisms, > file location systems, etc because it will be easier to use the base support > than invent something new. Could be nice, but I think it's a different beast from project.el. I actually have the impression that what you describe already exists and is called EDE ;-) Stefan ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: {Spam?} Re: progmodes/project.el and search paths 2015-08-06 23:10 ` Stefan Monnier @ 2015-08-07 11:18 ` Eric Ludlam 2015-08-07 11:43 ` David Engster 2015-08-07 12:08 ` Alexis 0 siblings, 2 replies; 101+ messages in thread From: Eric Ludlam @ 2015-08-07 11:18 UTC (permalink / raw) To: Stefan Monnier Cc: Eric Ludlam, Dmitry Gutov, David Engster, Emacs Development On 08/06/2015 07:10 PM, Stefan Monnier wrote: >> It would be nice if the generic base project were the same. Any project >> backend you use would have similar keybindings, menus, detection mechanisms, >> file location systems, etc because it will be easier to use the base support >> than invent something new. > Could be nice, but I think it's a different beast from project.el. > I actually have the impression that what you describe already exists and > is called EDE ;-) > Hmmm. Now I'm back to the beginning of the thread. Perhaps project.el is really project-root.el. I think that would clarify a lot, and would continue to be helpful to the tools that only need to know about the root. A clear intent would prevent misconceptions from causing the base to grow into something unintended. I'm also interested in how to resolve some of the misconceptions about EDE. Perhaps looking at what prevents it from being 'on by default', which I'm sure is a long list, is a good way to move it in the right direction, even if it never is on by default. Eric ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: {Spam?} Re: progmodes/project.el and search paths 2015-08-07 11:18 ` Eric Ludlam @ 2015-08-07 11:43 ` David Engster 2015-08-07 12:17 ` Dmitry Gutov 2015-08-07 12:08 ` Alexis 1 sibling, 1 reply; 101+ messages in thread From: David Engster @ 2015-08-07 11:43 UTC (permalink / raw) To: Eric Ludlam; +Cc: Eric Ludlam, Dmitry Gutov, Stefan Monnier, Emacs Development Eric Ludlam writes: > On 08/06/2015 07:10 PM, Stefan Monnier wrote: >>> It would be nice if the generic base project were the same. Any project >>> backend you use would have similar keybindings, menus, detection mechanisms, >>> file location systems, etc because it will be easier to use the base support >>> than invent something new. >> Could be nice, but I think it's a different beast from project.el. >> I actually have the impression that what you describe already exists and >> is called EDE ;-) >> > > Hmmm. Now I'm back to the beginning of the thread. > > Perhaps project.el is really project-root.el. I think that would > clarify a lot, and would continue to be helpful to the tools that only > need to know about the root. A clear intent would prevent > misconceptions from causing the base to grow into something > unintended. Despite what may have been said, project.el is *not* meant to be some kind of "base class" for different project systems (that was at least what I initially thought, and which you seem to think as well). The way I see it, project.el is more like an adapter, although one that delegates the actual implementation, so something like an "abstract adapter". Dmitry and Stefan are saying that this adapter should only provide the information that is common to all possible project systems, which pretty much means "the list of files/directories that are part of this project". While I agree this will already be very useful, as it allows you to use xref, grep, ido-find-file, etc. on the current project. I also think this is a very small feature for claiming the whole project- namespace. > I'm also interested in how to resolve some of the misconceptions about > EDE. Perhaps looking at what prevents it from being 'on by default', Last time we discussed this it boiled down to "we cannot preload EIEIO". -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: {Spam?} Re: progmodes/project.el and search paths 2015-08-07 11:43 ` David Engster @ 2015-08-07 12:17 ` Dmitry Gutov 2015-08-07 12:40 ` David Engster 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-07 12:17 UTC (permalink / raw) To: David Engster, Eric Ludlam; +Cc: Eric Ludlam, Stefan Monnier, Emacs Development On 08/07/2015 02:43 PM, David Engster wrote: > Despite what may have been said, project.el is *not* meant to be some > kind of "base class" for different project systems (that was at least > what I initially thought, and which you seem to think as well). The way > I see it, project.el is more like an adapter, although one that > delegates the actual implementation, so something like an "abstract > adapter". In OOP terms, it most closely resembles an "interface". > Dmitry and Stefan are saying that this adapter should only > provide the information that is common to all possible project systems, > which pretty much means "the list of files/directories that are part of > this project". Not necessarily. As has been touched on in a related discussion, it should be possible, for example, to define a fairly agnostic project-build-targets (or "tasks", not sure which is the more common term). I'm sure the list can be continued. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: {Spam?} Re: progmodes/project.el and search paths 2015-08-07 12:17 ` Dmitry Gutov @ 2015-08-07 12:40 ` David Engster 2015-08-07 12:54 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: David Engster @ 2015-08-07 12:40 UTC (permalink / raw) To: Dmitry Gutov; +Cc: Eric Ludlam, Eric Ludlam, Stefan Monnier, Emacs Development Dmitry Gutov writes: > On 08/07/2015 02:43 PM, David Engster wrote: > >> Despite what may have been said, project.el is *not* meant to be some >> kind of "base class" for different project systems (that was at least >> what I initially thought, and which you seem to think as well). The way >> I see it, project.el is more like an adapter, although one that >> delegates the actual implementation, so something like an "abstract >> adapter". > > In OOP terms, it most closely resembles an "interface". EDE already has an interface. project.el allows to use (a small part of) that interface through another, hence 'adapter'. >> Dmitry and Stefan are saying that this adapter should only >> provide the information that is common to all possible project systems, >> which pretty much means "the list of files/directories that are part of >> this project". > > Not necessarily. As has been touched on in a related discussion, it > should be possible, for example, to define a fairly agnostic > project-build-targets (or "tasks", not sure which is the more common > term). > > I'm sure the list can be continued. Yes, but your usual answer so far was that you want a concrete package that needs it, so there's a chicken/egg problem here. -David ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: {Spam?} Re: progmodes/project.el and search paths 2015-08-07 12:40 ` David Engster @ 2015-08-07 12:54 ` Dmitry Gutov 0 siblings, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-07 12:54 UTC (permalink / raw) To: David Engster; +Cc: Eric Ludlam, Eric Ludlam, Stefan Monnier, Emacs Development On 08/07/2015 03:40 PM, David Engster wrote: > EDE already has an interface. project.el allows to use (a small part of) > that interface through another, hence 'adapter'. Well, it's a different interface. We can have several. An adapter, in this case, is the project.el backend for EDE, which does little more than delegation. > Yes, but your usual answer so far was that you want a concrete package > that needs it, so there's a chicken/egg problem here. I disagree with that assessment: we need "concrete" consumers in Emacs anyway (at least some basic implementations). Not too long ago, there was a discussion on emacs-devel with people complaining that Emacs lacks a lot of IDE-like features. If certain important features of project.el are only used by third-party code (or, even worse, are still waiting for someone to use them), I'm sure the complaints will continue. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: {Spam?} Re: progmodes/project.el and search paths 2015-08-07 11:18 ` Eric Ludlam 2015-08-07 11:43 ` David Engster @ 2015-08-07 12:08 ` Alexis 1 sibling, 0 replies; 101+ messages in thread From: Alexis @ 2015-08-07 12:08 UTC (permalink / raw) To: emacs-devel Eric Ludlam <ericludlam@gmail.com> writes: > I'm also interested in how to resolve some of the misconceptions > about EDE. Perhaps looking at what prevents it from being 'on > by default', which I'm sure is a long list, is a good way to > move it in the right direction, even if it never is on by > default. Well, say i'm developing a small Perl5 project - one main module, supported by two helper modules, all of which are pure Perl. When i look at the overview of EDE in the Emacs manual: https://www.gnu.org/software/emacs/manual/html_node/emacs/EDE.html and read: A project may contain one or more targets. A target can be an object file, executable program, or some other type of file, which is “built” from one or more of the files in the project. i think to myself: "Okay, so EDE might be able to help me build (say) a .tar.gz when i feel the project is ready for release and distribution, but it's not relevant prior to that stage, since the source .pm files /are/ the executable files. There's nothing to 'build' during development and testing, so it seems i have no use for EDE during those phases." i would also guess that, similarly, EDE might not be that useful when developing in languages such as ELisp, Python or Ruby. Am i wrong? What would using EDE get me in such contexts? Alexis. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 7:13 ` Stefan Monnier 2015-08-04 8:13 ` David Engster @ 2015-08-04 9:40 ` Stephen Leake 2015-08-04 17:43 ` Dmitry Gutov 1 sibling, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-04 9:40 UTC (permalink / raw) To: emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > This is not meant to replace/reimplement EDE, but only to provide > a common API between different systems, EDE being hopefully one of > them. It would help if the header comment in project.el said something along those lines. There is a big difference between: "provide an interface between projects and the rest of Emacs" and "provide a core API for implementing projects". The current project.el header is unclear as to which it is intended to be. I was assuming projects.el was the latter, and thus in competition with EDE, projectile, etc. If it's really the former, that puts a much different light on things; it should be providing wrappers/adaptors for EDE, projectile, etc, so grep, compile etc can use project information. In particular, there should be no "xref-find-regexp"; instead, "grep" and similar commands should be enhanced to optionally use project functions to get the search path. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 9:40 ` Stephen Leake @ 2015-08-04 17:43 ` Dmitry Gutov 2015-08-04 19:49 ` Stephen Leake 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-04 17:43 UTC (permalink / raw) To: Stephen Leake, emacs-devel On 08/04/2015 12:40 PM, Stephen Leake wrote: > It would help if the header comment in project.el said something along > those lines. > > There is a big difference between: > > "provide an interface between projects and the rest of Emacs" > > and > > "provide a core API for implementing projects". > > The current project.el header is unclear as to which it is intended to > be. Would adding this help? diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el index d849f93..16578f1 100644 --- a/lisp/progmodes/project.el +++ b/lisp/progmodes/project.el @@ -22,6 +22,10 @@ ;; This file contains generic infrastructure for dealing with ;; projects, and a number of public functions: finding the current ;; root, related project directories, search path, etc. +;; +;; The goal is to make it easy for Lisp programs to operate on the +;; current project, without having to know which package handles +;; detection of that project type, parsing its config files, etc. ;;; Code: > If it's really the former, that puts a much different light on things; > it should be providing wrappers/adaptors for EDE, projectile, etc, so > grep, compile etc can use project information. "Should" is a strong word. There's wrapper for EDE living in lisp/cedet/ede.el already. Projectile "should" include its own wrapper as well. > In particular, there should be no "xref-find-regexp"; instead, "grep" > and similar commands should be enhanced to optionally use project > functions to get the search path. xref-find-regexp can still be useful: it'll search a user-specified directory and present the results using the xref interface. At the moment, xref-find-regexp offers both types of searches. To search in an arbitrary directory, you preface it with C-u. ^ permalink raw reply related [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 17:43 ` Dmitry Gutov @ 2015-08-04 19:49 ` Stephen Leake 2015-08-04 20:03 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-04 19:49 UTC (permalink / raw) To: emacs-devel Dmitry Gutov <dgutov@yandex.ru> writes: > > Would adding this help? > > diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el > index d849f93..16578f1 100644 > --- a/lisp/progmodes/project.el > +++ b/lisp/progmodes/project.el > @@ -22,6 +22,10 @@ > ;; This file contains generic infrastructure for dealing with > ;; projects, and a number of public functions: finding the current > ;; root, related project directories, search path, etc. > +;; > +;; The goal is to make it easy for Lisp programs to operate on the > +;; current project, without having to know which package handles > +;; detection of that project type, parsing its config files, etc. Better. I suggest adding "and Emacs core functions", to emphasize that grep etc should use the project facilities, and also mention existing backends: ;; The goal is to make it easy for Lisp programs, and Emacs core ;; functions, to operate on the current project, without having to know ;; which package handles detection of that project type, parsing its ;; config files, etc. Existing project packages should provide thin ;; wrappers around their functions to implement this API. > There's wrapper for EDE living in > lisp/cedet/ede.el already. That's a start. > Projectile "should" include its own wrapper as well. Yes. >> In particular, there should be no "xref-find-regexp"; instead, "grep" >> and similar commands should be enhanced to optionally use project >> functions to get the search path. > > xref-find-regexp can still be useful: it'll search a user-specified > directory and present the results using the xref interface. > > At the moment, xref-find-regexp offers both types of searches. To > search in an arbitrary directory, you preface it with C-u. I thought you agreed that the compilation-mode output of grep was better for this than the xref display. grep currently searches a user-specified directory; it could offer the choice of project search path via C-u 1 or some such. And we could add a grep-project command so the choice shows up in command completion. grep-project could get filename patterns from the project, as ede almost provides. That would require an additional wrapper in ede, for project-file-pattern. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 19:49 ` Stephen Leake @ 2015-08-04 20:03 ` Dmitry Gutov 2015-08-05 6:02 ` Stephen Leake 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-04 20:03 UTC (permalink / raw) To: Stephen Leake, emacs-devel On 08/04/2015 10:49 PM, Stephen Leake wrote: > I suggest adding "and Emacs core functions", to emphasize that grep etc > should use the project facilities, and also mention existing backends: I don't particularly like that. We're obviously talking about only some of the "core functions" (and it's debatable whether Grep is one of them). At least "Lisp programs" is a superset of them. > I thought you agreed that the compilation-mode output of grep was better > for this than the xref display. You stated that. I never agreed. I just mentioned that it has an advantage (each result is displayed as soon as the process finds it), but we should be able to implement it for xref as well, using generators. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-04 20:03 ` Dmitry Gutov @ 2015-08-05 6:02 ` Stephen Leake 2015-08-05 9:59 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-05 6:02 UTC (permalink / raw) To: emacs-devel Dmitry Gutov <dgutov@yandex.ru> writes: > On 08/04/2015 10:49 PM, Stephen Leake wrote: > >> I suggest adding "and Emacs core functions", to emphasize that grep etc >> should use the project facilities, and also mention existing backends: > > I don't particularly like that. We're obviously talking about only > some of the "core functions" (and it's debatable whether Grep is one > of them). At least "Lisp programs" is a superset of them. > >> I thought you agreed that the compilation-mode output of grep was better >> for this than the xref display. > > You stated that. I never agreed. I just mentioned that it has an > advantage (each result is displayed as soon as the process finds it), > but we should be able to implement it for xref as well, using > generators. Ok, so xref-find-regexp uses the project search path by default, and the xref display; it uses a user-specified root dir with C-u. Do you aggree it would be good if rgrep supported an option to use the project search path? Possibly only via the command grep-project (or project-grep)? -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 6:02 ` Stephen Leake @ 2015-08-05 9:59 ` Dmitry Gutov 2015-08-06 7:25 ` Stephen Leake 0 siblings, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-05 9:59 UTC (permalink / raw) To: Stephen Leake, emacs-devel On 08/05/2015 09:02 AM, Stephen Leake wrote: > Ok, so xref-find-regexp uses the project search path by default, > and the xref display; it uses a user-specified root dir with C-u. That's a bit awkward, though. While it's not very surprising, there's still nothing in this command's name to indicate that it uses the current project by default. > Do you aggree it would be good if rgrep supported an option to use the > project search path? Possibly only via the command grep-project (or > project-grep)? I think we should have project-find-regexp (or project-rgrep) which uses the xref interface. grep-project will be functionally redundant. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 9:59 ` Dmitry Gutov @ 2015-08-06 7:25 ` Stephen Leake 2015-08-07 14:21 ` Dmitry Gutov 0 siblings, 1 reply; 101+ messages in thread From: Stephen Leake @ 2015-08-06 7:25 UTC (permalink / raw) To: emacs-devel Dmitry Gutov <dgutov@yandex.ru> writes: > On 08/05/2015 09:02 AM, Stephen Leake wrote: > >> Do you aggree it would be good if rgrep supported an option to use the >> project search path? Possibly only via the command grep-project (or >> project-grep)? > > I think we should have project-find-regexp (or project-rgrep) which > uses the xref interface. grep-project will be functionally redundant. Since they are the same function, why is the name project-find-regexp better than the name grep-project? The latter will be _much_ easier to discover for the current grep user; M-x gre<tab> will show it. The purpose of project.el is to provide access to project information to other elisp programs; it is _not_ to provide user level programs. 'grep' is a user level elisp program that could use some project information. If it is possible to provide user control over that via prefixes or prompts, then the name doesn't have to change. If that's too complicated, grep-project seems like the best choice; similar to grep-find. 'project-find-regexp' is a user level elisp program; project.el is not supposed to provide those. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-06 7:25 ` Stephen Leake @ 2015-08-07 14:21 ` Dmitry Gutov 0 siblings, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-07 14:21 UTC (permalink / raw) To: Stephen Leake, emacs-devel On 08/06/2015 10:25 AM, Stephen Leake wrote: >> I think we should have project-find-regexp (or project-rgrep) which >> uses the xref interface. grep-project will be functionally redundant. > > Since they are the same function, why is the name project-find-regexp > better than the name grep-project? project-rgrep sounds just as good to me (M-x grep is flat). Would grep-project, in your view, use the xref interface? If not, they're not the same either. > The latter will be _much_ easier to discover for the current grep user; > M-x gre<tab> will show it. On the other hand, M-x project-<tab> won't. And the user won't be able to discover all core commands that work on the current project. Both problems should be solved by making execute-extended-command use a new completion category that would perform laxer matching. > The purpose of project.el is to provide access to project information to > other elisp programs; it is _not_ to provide user level programs. It's the purpose of the API. Nothing says we can't provide some commands that use the API together with it. We could move them to a yet-another package is that's the consensus, but there is value in keeping them together. Not every project command is going to have a "normal" counterpart. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-02 17:17 ` Dmitry Gutov 2015-08-03 1:19 ` Eric Ludlam 2015-08-03 13:49 ` David Engster @ 2015-08-05 1:29 ` Eric Ludlam 2015-08-11 20:01 ` Dmitry Gutov 2015-12-29 2:00 ` Dmitry Gutov 2 siblings, 2 replies; 101+ messages in thread From: Eric Ludlam @ 2015-08-05 1:29 UTC (permalink / raw) To: Dmitry Gutov, Emacs Development [-- Attachment #1: Type: text/plain, Size: 2925 bytes --] On 08/02/2015 01:17 PM, Dmitry Gutov wrote: > If you can propose better ways to do the same things, or something > else worth adding, please do. AND On 08/04/2015 09:43 AM, Eli Zaretskii wrote: > I really think we could benefit from presenting some kind of goal or > vision for the project.el development. AFAICS, it started as a way to > standardize finding a "project root", then moved to finding "project > directories", but now it seems we are discussing a much wider scope, > something like infrastructure for project definition and possibly also > IDEs? If it's the latter, then I won't expect to see arguments about > "complications outweighing benefits" in reference to features that are > basic for any IDE that is worth its menus. > > So maybe we should step back a notch and decide anew where is > project.el going, and what is and isn't in its scope? Hi Dmitry, I was tempted to start replying to multiple sub-threads in multiple emails, but I suspect there is nuance I am not expressing well, so I've opted to go back to one of your original questions to engage in a more positive way. I scraped through some old threads, old code, and my memory to put together the attached org file. The intent is to answer your first question about things worth adding to a generic project. It also happens to be something that can answer Eli's question, so I quoted him too. When the generic project was first proposed, I thought it was a good idea where either EDE could be adapted to be a base, or EDE could adapt to a new base. I wasn't too picky, but I clearly had many assumptions about what it was that did not match. EDE's story is one where it started out specific, but became more generic because I needed to provide services to external tools I was writing, and there was just a lot of code donations and demand for more project definitions. Emacs has a lot of tools that could use a project service, and sometimes all the tool needs is a small push of convenience, and not full project management system. A generic project system will make it easy to add small conveniences in lots of places you may not expect because tool authors will feel safe using it. The attached is my experience of things I either needed in EDE's base classes to implement a project, or that some tool needed, and what I learned along the way. You might think it is too EDE specific, but I put in only the most basic stuff, and put in things I kind of wish I had time to add to EDE. I am sure there are entries you consider out of scope. In my mind projects need a centralizing force to bring consistency in the same way comint brought some very basic consistency to shell buffers. We should not skimp in approaching this goal. I consider this file incomplete, but I hope it proves useful as a starting point. I'd be happy to periodically update it for a little while if it is useful. Eric [-- Attachment #2: generic_project.txt --] [-- Type: text/plain, Size: 12119 bytes --] Eric's notes for Generic Project * Definitions ** Project Directory A top level directory in which source code for a project resides. ** Project A collection of Meta Data and Services that describe how code in a Project Directory is organized, versioned, compiled, etc. ** Project Meta Data A piece of information that describes some aspect of the project. Also: `slot' on a 'Project' in CLOS parlance ** Project Service Functionality provided by a generic project interface Also: 'method' on a 'Project' in CLOS parlance ** Project Client A feature that depends on either the Project Directory, Project, or a Project Service, usually developed independently of project implementations. ** Project Implementation An implementation of a concrete project that provides actual functionality, some of which is provided via the generic project service. ** Thing Something the generic project should handle that doesn't fit into one of the above categories... yet. * Service: Current Project Directory Provides external tools a way to identify the top-most directory of the current project. ** Features: *** Performance; For a buffer: As fast as a buffer local variable ref *** Performance; For a file or directory name: First time lookup should take advantage of any known open projects. *** Nested Projects There should be ways for there to be projects in projects. **** Example: Having a set of small projects for testing in a larger project ** Project Client Examples *** Identify a TAGS file to use for navigating a source tree *** Starting point for search tools *** Location of Manifests, AUTHOR, and other dist files * Service: Current Project Tools that need to know about some meta-data of the current project will need a way to refer to the current project. The current project needs to be some 'thing' that provides project features. ** Features *** Project Meta Data Projects can have meta data. There should be some standard meta-data, and meta data optionally provided by specific project implementations. *** Project Services Projects can have behavior and provide services. There should be some basic standard services, and optionally provided services specific to project implementations. *** Project Commands Services for users to execute when they are managing their project. There should be some basic standard commands, and optionally provided commands specific to project implementations. Project Commands may seem optional from a generic project point of view. They are primarily helpful as a homogenizing force between disparate project implementations which would be a good thing in the long run, the way comint made all shell buffers very similar and easy to move between. *** Menu/Toolbar/Other UI Services Project implementations should have a standard way to integrate into the Development menu where users expect project commands to exist. *** Extensible The thing used to represent a generic project should be extensible so project implementations can implement abstract services, or add new ones. *** Sensible Defaults The generic project thing should have sensible defaults for meta data and specific services so a project implementation doesn't need to implement them all. ** Example Features *** Version number of project Useful when distinguishing between different local repositories you might be working on. *** Compile Integration with the `compile' command, a standard action for many compiled programs. A more generic term that would work for running tests in an interpreted environment may be preferred. * Service: Project Detection All project management systems need a way to 'detect' a project the first time a user visits a file. Many of those systems are the same, and involve walking up and down directory trees looking for dominating files. ** Features: *** Dominating File Detection A core way to identify projects that use a dominating file, such as a top level build.xml, or a AndroidManifest.xml. *** Distributed dominating file detection A core way to identify projects that use the same dominating file in all directories in a project, such as Makefile, CVS. *** Nested Projects There should be ways for there to be projects in projects. *** Declarative detection features When an project implementation integrates with the generic project system, it should be able to 'declare' how to detect a project of its preferred type. That way the source code for the project implementation will only be loaded into Emacs when a project of that type is detect. *** Many project types detectable w/out user configuration Users should have all available project types automatically detected without having to fiddle with their .emacs file. ELPA project implementations should have a way to add themselves to the detection list without loading their code. ** Project Detection Declaration features Features that a declaration of how to detect should provide *** How to detect A declarative way to express the desired detection mechanism. *** How to load the project code to support the project A function symbol and/or a symbol to require that will enable a project implementation to be created to support the detected project. *** Project Loading safety Some project types might need to load elisp-code-like structures to bring in meta data. If this is potentially unsafe, the project loader should ask the user, or consult a configuration, before allowing it. *** Detection ordering Some project types are a subset of other project types. Priority of project detection/loading should be a part of the detection declaration. **** Example: Makefile based project could exist with a .git repository. The Makefile project is more specific than the .git repository, and should be selected first. ** Tool Example *** A project manager (JDEE, EDE, etc) These tools need to detect projects that belong to them, but EDE might identify an ANT project, and JDEE might identify its .prj file, and they would attempt to co-exist without a mediating generic project service. When a tool asks for the current project, there should be only one. * Service: Project Commands Emacs already has a wide range of useful IDE like tools, such as `compile' and `gud'. These are generic, and require the user to type in any specific options needed for their project. Generic projects should provide integration with those tools, and likely multiple variants regarding the project in question. The project service should allow project implementations to augment these core features with variants. ** Features *** Compile integration Projects implementations should be able to provide default compile arguments based on project meta data. Multiple variations should be available such as `whole project' or `just this thing in my buffer'. *** GUD integration Provide default debugger commands, and arguments to GUD supported debugger commands. *** Update meta-data Emacs is not the only project system out there, so users should have a way to force a project implementation to refresh itself from on-disk changes. *** Project browsing Start users preferred file browsing project client on the project. **** Examples: ECB, Speedbar, Dired, external executable * Service: Keybinding / Menu / Toolbar management Project implementations should have a standard way to integrate into the Development menu where users expect project commands to exist. User commands provided should be consistent between types of projects. The purpose of this is to provide a homogenizing force so that different project implementations will gain a similar look and feel so users can move between different projects, and enjoy consistent keybindings and UI elements. ** Features *** Organized project options Tools should be logically organized in whichever UI elements are used to express it. *** Default project commands There should be some default project commands, like 'compile' built in. *** Room for custom options There should be an easy way for a project implementation to add new items into the logical groupings provided. ** Examples *** Arduino: upload to dev board Arduino project implementation needs to add an entry to the menu system to upload a project onto the dev board, separate from compiling to a binary. *** JDEE vs EDE Compile commands are different between JDEE and EDE. If a standard way of invoking the disparate commands were available, users could work in C++ and Java and use the same keybindings or menus in each. * Service: Project Local Variables Tools like dir-locals provide some level of this feature. Generic projects could help enable project local variables. * Meta Data: Version Projects should provide a way to extract version numbers, and make them readily available so users can distinguish between multiple versions of the same code base. * Meta Data: Arbitrary Project implementations should have a way to provide their own meta data to external commands, and have a way to negotiate for commonality between project implementations that support the same concept. * Meta Data: Client Data Project clients that develop a user service based on generic project data should have a way to provide their own data to a project where the original project implementation author may not know what is needed. ** Examples *** Flymake Flymake should have a way to ask for meta data about compilation, and if the project implementation doesn't support it, there should be no errors, and Flymake shouldn't need to know anything specific about all implementations that might support it. *** User Option for client The project client 'projectile' can toggle between code and test, but the project implementation author didn't account for that. Projectile will need to associate user supplied meta-data about test locations with the project. Puzzle - persistence? * Service: Finding Files Finding files comes in a couple different flavors. 1) User wants to visit a file via a 'short name', and the project can find it. 2) Code often refers to other files using a short name, and refers to an 'include path' concept in the project to find it. ** Features: *** Enable a project client to provide user tools to 'jump' to a file. *** Enable language specific 'include path' concept Useful when multiple files with the same name exist, and are distinguished by location. ** Examples *** C++ include path The semantic smart completion engine depends on include paths to find headers which provide definitions to complete on. *** Java Classpath Semantic depends on a java classpath to find jar files from which it extracts symbol information used for smart completion. *** cscope or GNU Global integration as project service These tools track not just references, but files in a project, and have easy ways to convert a short file name into a full path name. *** project client to jump from 'include' statement in C++ to the file *** project client to jump between a src file (xml, etc) to generated output. * Command: Load a project; Service: Known unopened projects The generic project system should have a handy way to provide a completion list of known projects a user might choose to load. Project lists might be provided by recording all projects a user may have previously opened, or via scan of areas relevant to the user. ** Examples: *** desktop A project client might associated a working set of buffers to load upon opening a project. * Thing: Multiple activities on one project Projects usually have multiple activities in them. At the most basic there are 'get it ready to use' support files, such as compilation system, INSTALL doc, etc, and the source code. Full project documentation, and multi-feature projects have multiple build targets, and complex build systems. The generic project should consider supporting concepts for multiple activities in one project. ** Examples *** `compile' knows which thing to compile in a project *** Language specificity Operations in a particular language restricted to other files of that language. *** Code & Doc Enable a project client to support navigating between code and doc. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 1:29 ` Eric Ludlam @ 2015-08-11 20:01 ` Dmitry Gutov 2015-08-12 0:49 ` Eric Ludlam 2015-12-29 2:00 ` Dmitry Gutov 1 sibling, 1 reply; 101+ messages in thread From: Dmitry Gutov @ 2015-08-11 20:01 UTC (permalink / raw) To: Eric Ludlam, Emacs Development Hi Eric, Sorry for the late reply. I wasn't really sure what kind of response is expected. In short, I like your vision as a whole, but whether each particular element of it can apply to project.el, should be investigated separately. On some of them: * I wonder what's the main use of having hierarchical projects vs. simply tracking several currently open projects (or project roots). And how the consumers of the API would take advantage of that distinction. * Project services seem to be the proper way to handle optional traits of significant size that some projects have, and others don't. * "Project Loading safety" are probably out of scope. I'd expect the majority of implementations to be based around loading a project build (or settings) file of pre-existing format. The popular IDEs don't ask the user whether they want to load the project file. On the other hand, some project types might need to be enabled/visited explicitly. * WRT "Detection ordering", the current project-find-functions structure might be enough. Any Makefile project implementation will most likely go before the VC project anyway (unless the respective add-hook form is told to APPEND, for some reason). And it's hard for us to expect for third-party implementations to be able to declare priorities between themselves. * "Update meta-data" might be handled by each respective implementation, in conjunction with auto-revert-mode. * "Project browsing" shouldn't need much more than project.el exposes already. * "Project Local Variables" look very similar to project metadata. * The "'include path' concept" looks very similar to project-search-path. This might be the main purpose for the project-search-path entries that lie within project-roots. * WRT cscope, etc, integration, I think the main unsolved problem we currently have is that they aren't re-run automatically. Nor do we have a command that would do that for the user on demand. * "Load a project" seems incompatible with the project-find-functions approach. Which is unfortunate. * "navigating between code and doc", as stated, doesn't seem particularly useful: if you're reading the source code of a function, the docstring is usually right above. Jumping from the doc to the source is better, but this is closer to the domain of xref. * "Language specificity" might be useful, but in practical terms it sounds like adding an optional (?) "language" argument to project-search-path and any similar method that we add later. Which complicates implementations. Or just have an Emacs-global repository of file name patterns corresponding to languages? Not sure. * "commands specific to project implementations" could also be added by the minor modes supplied by the implementations' packages. On 08/05/2015 04:29 AM, Eric Ludlam wrote: > Emacs has a lot of tools that could use a project service, and sometimes > all the tool needs is a small push of convenience, and not full project > management system. A generic project system will make it easy to add > small conveniences in lots of places you may not expect because tool > authors will feel safe using it. That's the idea, yes. > The attached is my experience of things I either needed in EDE's base > classes to implement a project, or that some tool needed, and what I > learned along the way. You might think it is too EDE specific, I wouldn't say the majority of items in the list are EDE specific. > I consider this file incomplete, but I hope it proves useful as a > starting point. I'd be happy to periodically update it for a little > while if it is useful. Were we using a nice issue tracker, I'd have asked to create a tracking issue (or would've created one myself), but in the present situation, I guess you'll have to resend it with updates. But in any case, I'd say technical proposals on specific features, as well as patches, would provide more value here, rather than maintaining a high-level overview of things we might get around to someday. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-11 20:01 ` Dmitry Gutov @ 2015-08-12 0:49 ` Eric Ludlam 2015-08-12 7:25 ` project terminology Stephen Leake 2015-08-12 9:28 ` progmodes/project.el and search paths Dmitry Gutov 0 siblings, 2 replies; 101+ messages in thread From: Eric Ludlam @ 2015-08-12 0:49 UTC (permalink / raw) To: Dmitry Gutov, Emacs Development Hi Dmitry, Thanks for the thoughtful reply. On 08/11/2015 04:01 PM, Dmitry Gutov wrote: > Hi Eric, > > Sorry for the late reply. I wasn't really sure what kind of response > is expected. > > In short, I like your vision as a whole, but whether each particular > element of it can apply to project.el, should be investigated separately. > > On some of them: > > * I wonder what's the main use of having hierarchical projects vs. > simply tracking several currently open projects (or project roots). > And how the consumers of the API would take advantage of that > distinction. > I doubt consumers of the project API would care. Nested projects is simply something the detection system should enable. If there is a centralized way of scanning for projects, it would need to support this. If each project system detects projects its own way, then it wouldn't matter at the core level. My hypothesis is that project detection is tricky, and new custom projects would pop up more frequently if there were infrastructure for detecting them. If project.el were to provide that infrastructure, it would need to support nested projects. > * Project services seem to be the proper way to handle optional traits > of significant size that some projects have, and others don't. > * "Project Loading safety" are probably out of scope. I'd expect the > majority of implementations to be based around loading a project build > (or settings) file of pre-existing format. The popular IDEs don't ask > the user whether they want to load the project file. On the other > hand, some project types might need to be enabled/visited explicitly. > Every project is indeed different. A system that needs to load lisp code needs to protect against ill formatted lisp code. Putting this in a core system will help implementations from all creating the same latent bug. If project.el isn't going to provide detection services, then it won't need this feature. > * WRT "Detection ordering", the current project-find-functions > structure might be enough. Any Makefile project implementation will > most likely go before the VC project anyway (unless the respective > add-hook form is told to APPEND, for some reason). And it's hard for > us to expect for third-party implementations to be able to declare > priorities between themselves. Project.el is providing a list of ways for projects to present themselves. At the moment however, projectile v EDE v your vc detector could all claim ownership of the same project. At the moment it probably doesn't matter because since all three provide different services based on their personal consideration of 'root'. Any tool needing to know only the root would be happy, but if there were some extra service to be provided, a priority between the 3 would be needed. If project.el were to provide the detection service as well, then the 3 would need to prioritize. In EDE, a project adds itself to the list of possible projects. Each project classifies itself by a term that places itself in one of three bins, and hopefully those bins don't conflict. ;) Anyway, EDE's bins are: 'unique - something like the Emacs or Linux project - detect these first. no flag provide - One of the specialized build system projects like Automake support, or Maven. 'generic - Something that provides very little overall support, such as detection by VC, or existence of SCons. - Always at the end. This is pretty easy to self bin by. > * "Update meta-data" might be handled by each respective > implementation, in conjunction with auto-revert-mode. Right. Each project implementation needs to update their data from the file system their own way. If there are generic development menus, it would be helpful for the user to have one way to update project meta data regardless of which type of implementation is active. > * "Project browsing" shouldn't need much more than project.el exposes > already. > Yup. I was trying to be complete. > * "Project Local Variables" look very similar to project metadata. They are a type of meta data that let users provide extended behavior to the project. > * The "'include path' concept" looks very similar to > project-search-path. This might be the main purpose for the > project-search-path entries that lie within project-roots. > I will need to look at your search path concept more. Are there multiple paths for different purposes? > * WRT cscope, etc, integration, I think the main unsolved problem we > currently have is that they aren't re-run automatically. Nor do we > have a command that would do that for the user on demand. > Several of these tools have an 'update the current tags file' type mode you can get at. I don't recall if I wrapped that up with a handy function or not. > * "Load a project" seems incompatible with the project-find-functions > approach. Which is unfortunate. > > * "navigating between code and doc", as stated, doesn't seem > particularly useful: if you're reading the source code of a function, > the docstring is usually right above. Jumping from the doc to the > source is better, but this is closer to the domain of xref. Just pulling out examples, some were from threads on the ML. The key concept is that a project is rarely just one single activity. Usually there are doc activities, code activities, build system activities. Some projects have Java activities, and native C++ activities. All these different things are for a common goal, and often there is a common lingo between them. I represent that lingo as 'tags' in Semantic, and it enables you to 'copy' a tag which includes the meta data around the name and 'paste' it into a buffer of a different language. The only useful one there can turn a lisp tag into a texi @defun block, but you could imagine a way to update your java native methods bidirectionally. The project knows how to link these systems together, such as finding the right doc for your Lisp code. This is an advanced maneuver that would be great for a tool author to be able to do, and is more likely if that kind of association can be easily made. > * "Language specificity" might be useful, but in practical terms it > sounds like adding an optional (?) "language" argument to > project-search-path and any similar method that we add later. Which > complicates implementations. Or just have an Emacs-global repository > of file name patterns corresponding to languages? Not sure. > Language specificity was key for me because of the way mode-local works, and how each language had customized behavior for the parsing tools. I tended to focus on language specific accurate behaviors, and not generic potentially inaccurate features. For example, smart completion vs dabbrevs. As noted somewhere, symref had a way of converting modes to patterns. I had to ask around, but eventually got something pretty good. [...] >> I consider this file incomplete, but I hope it proves useful as a >> starting point. I'd be happy to periodically update it for a little >> while if it is useful. > > Were we using a nice issue tracker, I'd have asked to create a > tracking issue (or would've created one myself), but in the present > situation, I guess you'll have to resend it with updates. > > But in any case, I'd say technical proposals on specific features, as > well as patches, would provide more value here, rather than > maintaining a high-level overview of things we might get around to someday My goal is to be more helpful than adversarial, but at the time I didn't have a clear picture where project.el would fit, and if there were anything I had to deal with in the future. I'd be glad if I could stop maintaining parts of EDE because it became part of some core system. It is still unclear to me where the boundaries lie. I'm not sure where a patch would be helpful yet. My hope is that my document would help you define the bounds of project.el based in a set of terms we can both agree to. It would then be obvious how I could help by donating pieces of EDE as needed. My recommendation for you is to perhaps not reply to individual clarifications above, but perhaps define where you think project.el fits into the whole using whatever you like from my doc, and that would help us focus the conversation on the boundary, and not on random things I invented that aren't close to the boundary. Thanks Eric ^ permalink raw reply [flat|nested] 101+ messages in thread
* project terminology 2015-08-12 0:49 ` Eric Ludlam @ 2015-08-12 7:25 ` Stephen Leake 2015-08-12 9:28 ` progmodes/project.el and search paths Dmitry Gutov 1 sibling, 0 replies; 101+ messages in thread From: Stephen Leake @ 2015-08-12 7:25 UTC (permalink / raw) To: emacs-devel Eric Ludlam <ericludlam@gmail.com> writes: > On 08/11/2015 04:01 PM, Dmitry Gutov wrote: >> >> * I wonder what's the main use of having hierarchical projects vs. >> simply tracking several currently open projects (or project roots). >> And how the consumers of the API would take advantage of that >> distinction. >> > > I doubt consumers of the project API would care. Nested projects is > simply something the detection system should enable. I'm puzzled by this terminology; I don't think "nested" means the same thing as "hierarchical"; nested is a subset of hierarchical. For example, here are two possible project layouts on the disk: layout 1: /core # math and IO utils /my_proj # project file includes dependency on /core layout 2: /my_proj # no dependency in project file /core # math and IO utils Other than the layout change, the code is the same (the build scripts that specify paths etc are slightly different). I would call both of these "hierarchical", but only layout 2 is "nested". Apparently, the term "subproject" is also appropriate for core in layout 2. In any case, it would be good to have the option to treat "my_proj" as a single project, that "includes" "core" in any operations that involve searching for things in "the current project". As for distinguishing hiearchical from a simple set of open projects, that's what the project dependency declarations are for. They don't necessarily need to be visible to project.el; some backends may handle that entirely on their own (the Ada mode backend does, for example). > A system that needs to load lisp > code needs to protect against ill formatted lisp code. Putting this > in a core system will help implementations from all creating the same > latent bug. If project.el isn't going to provide detection services, > then it won't need this feature. There is a difference between detecting the presence of a project, and using that project. An extremely simple way to detect projects is to ask the user to specify a project location. Another way is to look for a special file name. Once the project file is found, then it must be loaded. But the detection step (whether ask user or some other method), does not require loading the elisp file (or parsing the project file in general). In the above, I've assumed "project detection" meant "deciding whether there is a project in a disk directory". In other circustances, "project detection" means "pick which of the available projects is appropriate for the current action". It's not always clear from context which meaning is meant. Perhaps we could use the term "project selection" for the latter. In project.el, project-find is used for project selection, but the current implementations of it do project detection as the selection algorithm. Other choices are possible; the "project-explicit" implementation I proposed separates detection from selection. I think EDE is similar, in that it tends to use project detection as part of the selection algorithm. -- -- Stephe ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-12 0:49 ` Eric Ludlam 2015-08-12 7:25 ` project terminology Stephen Leake @ 2015-08-12 9:28 ` Dmitry Gutov 1 sibling, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-08-12 9:28 UTC (permalink / raw) To: Eric Ludlam, Emacs Development On 08/12/2015 03:49 AM, Eric Ludlam wrote: > Every project is indeed different. A system that needs to load lisp > code needs to protect against ill formatted lisp code. Putting this in > a core system will help implementations from all creating the same > latent bug. If project.el isn't going to provide detection services, > then it won't need this feature. Probably not, at least not at first. > In EDE, a project adds itself to the list of possible projects. Each > project classifies itself by a term that places itself in one of three > bins, and hopefully those bins don't conflict. ;) Not to dismiss the detailed approach, but the user could reorder the elements in project-find-functions as well, if the default order is not to their liking. What you're describing would be an improvement, but not of the revolutionary kind. And I'm not really sure how to fit the classification together with project-find-functions. > Right. Each project implementation needs to update their data from the > file system their own way. If there are generic development menus, it > would be helpful for the user to have one way to update project meta > data regardless of which type of implementation is active. I'd rather user never had to do that at all. >> * "Project Local Variables" look very similar to project metadata. > > They are a type of meta data that let users provide extended behavior to > the project. That sounds fine in theory, but we need a more concrete application. Your Projectile example, for instance, would work just as well if it didn't set the "project local variables" via the project API, and just stored the information in its internal storage. > I will need to look at your search path concept more. Are there > multiple paths for different purposes? No, just this one. Adding other paths will need more justification. > Several of these tools have an 'update the current tags file' type mode > you can get at. I don't recall if I wrapped that up with a handy > function or not. That would be nice. But then you don't need any extra information about the current project to do that, just knowing how to find the file and/or call the tool. In that case, it shouldn't need anything new in the project API. Not even how to find the root, maybe (just use locate-dominating-file, especially when there are several tags files in the project). > The key concept is that a project is rarely just one single activity. > Usually there are doc activities, code activities, build system > activities. Some projects have Java activities, and native C++ > activities. All these different things are for a common goal, and often > there is a common lingo between them. I represent that lingo as 'tags' > in Semantic, and it enables you to 'copy' a tag which includes the meta > data around the name and 'paste' it into a buffer of a different > language. That sounds like it needs a parser to work well. xref tries to handle it as "identifier at point", but we'll have to see whether the different implementations will include context information in it (via text properties). > The only useful one there can turn a lisp tag into a texi > @defun block, but you could imagine a way to update your java native > methods bidirectionally. The project knows how to link these systems > together, such as finding the right doc for your Lisp code. For one thing, writing documentation separately from sources is an alien idea to me (and for other Ruby, Java or JS programmers, I think). So this kind of information may need to be exposed as an optional service. > Language specificity was key for me because of the way mode-local works, > and how each language had customized behavior for the parsing tools. I > tended to focus on language specific accurate behaviors, and not generic > potentially inaccurate features. For example, smart completion vs > dabbrevs. I like smart completion, but by tying it to projects we'd discount the less accurate implementations that are still useful. On the other hand, a package that implements project handling can add its own, accurate completion functions as well. > As noted somewhere, symref had a way of converting modes to patterns. I > had to ask around, but eventually got something pretty good. It didn't work well for Ruby files initially, so I had to add the respective entries to semantic-symref-filepattern-alist. I'm sure there are languages out there that are still not covered. > I'd be glad if I could stop maintaining parts of EDE because it became > part of some core system. It is still unclear to me where the > boundaries lie. I'm not sure where a patch would be helpful yet. My > hope is that my document would help you define the bounds of project.el > based in a set of terms we can both agree to. It would then be obvious > how I could help by donating pieces of EDE as needed. My understanding of EDE is if there are tools that would be worth porting, they would need to be rewritten on top of more general methods, and would probably need rethinking. > My recommendation for you is to perhaps not reply to individual > clarifications above, but perhaps define where you think project.el fits > into the whole using whatever you like from my doc, and that would help > us focus the conversation on the boundary, and not on random things I > invented that aren't close to the boundary. I'll try to do something like that in the foreseeable future. But really, from where I'm standing, the question is not how to decide the list of features, but how to implement each particular feature in a way that would make sense for as many project types as possible. ^ permalink raw reply [flat|nested] 101+ messages in thread
* Re: progmodes/project.el and search paths 2015-08-05 1:29 ` Eric Ludlam 2015-08-11 20:01 ` Dmitry Gutov @ 2015-12-29 2:00 ` Dmitry Gutov 1 sibling, 0 replies; 101+ messages in thread From: Dmitry Gutov @ 2015-12-29 2:00 UTC (permalink / raw) To: Eric Ludlam, Emacs Development Hi Eric, On 08/05/2015 04:29 AM, Eric Ludlam wrote: > The attached is my experience of things I either needed in EDE's base > classes to implement a project, or that some tool needed, and what I > learned along the way. You might think it is too EDE specific, but I > put in only the most basic stuff, and put in things I kind of wish I had > time to add to EDE. I've put a list of TODO items into project.el. It's less ambitious than your list, but hopefully it covers the essentials that we (together with the community) will be able to consistently implement across the board. I have omitted certain useful things like "jump from 'include' statement in C++ to the file" and "navigating between code and doc" because they seem to require separate (though related) facilities that we're yet to design. ^ permalink raw reply [flat|nested] 101+ messages in thread
end of thread, other threads:[~2015-12-29 2:00 UTC | newest] Thread overview: 101+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-08-02 13:52 progmodes/project.el and search paths Eric Ludlam 2015-08-02 17:17 ` Dmitry Gutov 2015-08-03 1:19 ` Eric Ludlam 2015-08-03 16:16 ` Stephen Leake 2015-08-03 22:56 ` Dmitry Gutov 2015-08-08 13:07 ` Nix 2015-08-09 5:18 ` Stephen Leake 2015-08-09 12:17 ` David Engster 2015-08-09 15:55 ` Stephen Leake 2015-08-10 11:29 ` David Engster 2015-08-10 16:43 ` Stephen Leake 2015-08-12 10:10 ` David Engster 2015-08-12 13:49 ` Stephen Leake 2015-08-12 15:36 ` David Engster 2015-08-13 11:53 ` Nix 2015-08-13 12:05 ` Dmitry Gutov 2015-08-14 11:52 ` Eric Ludlam 2015-08-14 22:30 ` Dmitry Gutov 2015-08-15 0:48 ` Eric Ludlam 2015-08-15 7:05 ` Eli Zaretskii 2015-08-10 17:12 ` Nix 2015-08-03 22:47 ` Dmitry Gutov 2015-08-04 11:52 ` Eric Ludlam 2015-08-04 16:09 ` Dmitry Gutov 2015-08-03 13:49 ` David Engster 2015-08-03 14:09 ` Dmitry Gutov 2015-08-03 14:27 ` David Engster 2015-08-03 15:13 ` Dmitry Gutov 2015-08-03 21:35 ` David Engster 2015-08-03 23:21 ` Dmitry Gutov 2015-08-04 8:15 ` David Engster 2015-08-04 13:43 ` Eli Zaretskii 2015-08-04 18:05 ` Dmitry Gutov 2015-08-04 18:16 ` Eli Zaretskii 2015-08-04 18:41 ` Dmitry Gutov 2015-08-04 19:23 ` Eli Zaretskii 2015-08-04 19:40 ` João Távora 2015-08-05 2:52 ` Eli Zaretskii 2015-08-04 20:15 ` Dmitry Gutov 2015-08-05 2:49 ` Eli Zaretskii 2015-08-05 6:18 ` Stephen Leake 2015-08-05 15:08 ` Eli Zaretskii 2015-08-05 15:36 ` Dmitry Gutov 2015-08-05 16:31 ` Eli Zaretskii 2015-08-05 16:45 ` David Engster 2015-08-05 22:17 ` Dmitry Gutov 2015-08-06 7:56 ` Stephen Leake 2015-08-06 7:54 ` Stephen Leake 2015-08-05 9:42 ` Dmitry Gutov 2015-08-05 15:23 ` Eli Zaretskii 2015-08-05 15:31 ` Dmitry Gutov 2015-08-05 16:16 ` Eli Zaretskii 2015-08-06 6:44 ` Dmitry Gutov 2015-08-06 7:43 ` Stephen Leake 2015-08-06 10:25 ` Dmitry Gutov 2015-08-06 14:27 ` Stephen Leake 2015-08-06 23:16 ` Dmitry Gutov 2015-08-07 14:10 ` Stephen Leake 2015-08-07 14:44 ` Dmitry Gutov 2015-08-03 16:35 ` Stephen Leake 2015-08-03 16:45 ` Eli Zaretskii 2015-08-03 21:07 ` Stephen Leake 2015-08-03 21:33 ` David Engster 2015-08-04 2:35 ` Eli Zaretskii 2015-08-03 15:09 ` Eli Zaretskii 2015-08-03 15:16 ` Dmitry Gutov 2015-08-03 15:29 ` Eli Zaretskii 2015-08-03 19:01 ` Dmitry Gutov 2015-08-03 19:19 ` Eli Zaretskii 2015-08-03 21:05 ` Dmitry Gutov 2015-08-04 11:48 ` Eric Ludlam 2015-08-04 16:20 ` Dmitry Gutov 2015-08-03 16:25 ` Stephen Leake 2015-08-03 21:33 ` Stefan Monnier 2015-08-03 22:15 ` David Engster 2015-08-03 22:50 ` Dmitry Gutov 2015-08-04 7:13 ` Stefan Monnier 2015-08-04 8:13 ` David Engster 2015-08-05 13:42 ` Stefan Monnier 2015-08-06 11:27 ` {Spam?} " Eric Ludlam 2015-08-06 23:10 ` Stefan Monnier 2015-08-07 11:18 ` Eric Ludlam 2015-08-07 11:43 ` David Engster 2015-08-07 12:17 ` Dmitry Gutov 2015-08-07 12:40 ` David Engster 2015-08-07 12:54 ` Dmitry Gutov 2015-08-07 12:08 ` Alexis 2015-08-04 9:40 ` Stephen Leake 2015-08-04 17:43 ` Dmitry Gutov 2015-08-04 19:49 ` Stephen Leake 2015-08-04 20:03 ` Dmitry Gutov 2015-08-05 6:02 ` Stephen Leake 2015-08-05 9:59 ` Dmitry Gutov 2015-08-06 7:25 ` Stephen Leake 2015-08-07 14:21 ` Dmitry Gutov 2015-08-05 1:29 ` Eric Ludlam 2015-08-11 20:01 ` Dmitry Gutov 2015-08-12 0:49 ` Eric Ludlam 2015-08-12 7:25 ` project terminology Stephen Leake 2015-08-12 9:28 ` progmodes/project.el and search paths Dmitry Gutov 2015-12-29 2:00 ` Dmitry Gutov
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.