unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* cl-defgeneric vs random funcall in project.el
@ 2015-07-28 11:31 Stephen Leake
  2015-07-28 15:09 ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-07-28 11:31 UTC (permalink / raw)
  To: emacs-devel

What is the rationale for having both cl-defgeneric and random funcall
in project.el?

For example, 'project-search-path' is a cl-defgeneric, which means the
various backends (elisp, etags, eventually Ada mode) can override it
with cl-defmethod.

But the default method uses 'project-search-path-function', which
elisp-mode sets to `elisp-search-function'; elisp.el does not use
cl-defmethod to override project-search-path.

Why do we need both dispatching methods?

I suggest this rationale be documented in the Commentary section of the
project.el header comment.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-28 11:31 cl-defgeneric vs random funcall in project.el Stephen Leake
@ 2015-07-28 15:09 ` Dmitry Gutov
  2015-07-28 23:57   ` Stefan Monnier
  2015-07-29  1:00   ` Stephen Leake
  0 siblings, 2 replies; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-28 15:09 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 07/28/2015 02:31 PM, Stephen Leake wrote:

> But the default method uses 'project-search-path-function', which
> elisp-mode sets to `elisp-search-function'; elisp.el does not use
> cl-defmethod to override project-search-path.

It's largely because, currently, project-search-path is supposed to 
return the source directories inside the project, as well as outside. 
And if a project implementation has to provide its own implementation, 
it's much cleaner, I think, if it only has to deal with 
project-search-path-function, instead of being forced to 
cl-call-next-method.

Further, this part seems like it'd be impossible to implement: "A 
specialized implementation should use the value
`project-search-path-function', or, better yet, call and combine
the results from the functions that this value is set to by all
major modes used in the project".

However, now that we've reached the consensus that project-search-path 
doesn't need to include the directories inside the current project, an 
implementation will only need to override it if it actually knows 
search-path better than the major-mode.

> Why do we need both dispatching methods?

The alternative (which is more feasible now) is to use the context 
specializer, like:

(cl-defmethod project-search-path (_project &context (major-mode (eql 
emacs-lisp-mode))))

It's also undocumented yet, and the method dispatch priority between it 
and method implementation dispatching on the first argument is also 
unclear now.

And it'll need a separate declaration for each deriving mode (such as 
lisp-interaction-mode).

Opinions welcome.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-28 15:09 ` Dmitry Gutov
@ 2015-07-28 23:57   ` Stefan Monnier
  2015-07-29  0:16     ` Dmitry Gutov
  2015-07-29  1:00   ` Stephen Leake
  1 sibling, 1 reply; 45+ messages in thread
From: Stefan Monnier @ 2015-07-28 23:57 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Stephen Leake, emacs-devel

> The alternative (which is more feasible now) is to use the context
> specializer, like:
>
> (cl-defmethod project-search-path (_project &context (major-mode (eql
> emacs-lisp-mode))))

FWIW, this sucks because this method won't be used in modes derived from
emacs-lisp-mode.

> It's also undocumented yet, and the method dispatch priority between it and
> method implementation dispatching on the first argument is also unclear now.

The priority of &context is lower than the first argument.
The priority between different &context can't be relied upon.

> And it'll need a separate declaration for each deriving mode (such as
> lisp-interaction-mode).

Exactly.


        Stefan



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-28 23:57   ` Stefan Monnier
@ 2015-07-29  0:16     ` Dmitry Gutov
  2015-07-30 21:26       ` Stefan Monnier
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-29  0:16 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Stephen Leake, emacs-devel

On 07/29/2015 02:57 AM, Stefan Monnier wrote:

> FWIW, this sucks because this method won't be used in modes derived from
> emacs-lisp-mode.

If we *really* wanted to use the same mechanism, though, I think 
implementing &context (derived-p emacs-lisp-mode) is not impossible.

> The priority of &context is lower than the first argument.
> The priority between different &context can't be relied upon.

Thanks.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-28 15:09 ` Dmitry Gutov
  2015-07-28 23:57   ` Stefan Monnier
@ 2015-07-29  1:00   ` Stephen Leake
  2015-07-29  1:19     ` Dmitry Gutov
  2015-07-30 21:31     ` Stefan Monnier
  1 sibling, 2 replies; 45+ messages in thread
From: Stephen Leake @ 2015-07-29  1:00 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/28/2015 02:31 PM, Stephen Leake wrote:
>
>> But the default method uses 'project-search-path-function', which
>> elisp-mode sets to `elisp-search-function'; elisp.el does not use
>> cl-defmethod to override project-search-path.
>
> And if a project implementation has to provide its own implementation,
> it's much cleaner, I think, if it only has to deal with
> project-search-path-function, instead of being forced to
> cl-call-next-method.

Well, that's my question. I don't see that as "cleaner"; I much prefer
object dispatching to random funcalls.

Why would it use 'cl-call-next-method'? That would call the
default method, which is wrong; that's why we're overriding it.

> Further, this part seems like it'd be impossible to implement: "A
> specialized implementation should use the value
> `project-search-path-function', or, better yet, call and combine
> the results from the functions that this value is set to by all
> major modes used in the project".

Right. I don't see that as a useful design. Just get rid of it.

Nothing else in project.el currently suggests that
'project-search-path-function' should be a list; the default method for
project-search-path doesn't treat it as a list. What is the use case
for this? I remember seeing something in the email chain; did that get
captured somewhere?

> However, now that we've reached the consensus that project-search-path
> doesn't need to include the directories inside the current project, 

I never agreed to that. In fact, I thought the only thing we _did_ agree
on was that project-search-path included the main project. Sigh.

> an implementation will only need to override it if it actually knows
> search-path better than the major-mode.

The "implementation" we are talking about here is presumably something
that knows how to read project files defined by some third-party tool
(Ada .gpr files, gradle files, etc). That would certainly know exactly
what the project search path should be, and it should override all
relevant project-* functions.

As far as I can see, the _primary_ purpose of a project-* implementation
is to provide the correct search-path and/or project-root, so the
functions that use those do the right thing.

>> Why do we need both dispatching methods?
>
> The alternative (which is more feasible now) is to use the context
> specializer, like:
>
> (cl-defmethod project-search-path (_project &context (major-mode (eql
> emacs-lisp-mode))))

That's one alternative.

Another is to have the project backends provide a minor mode, that
specifies the project implementation, by returning the proper value from
(current-project).

Or just let the major mode specify the project backend; that's what
happens in elisp and Ada mode now.

One implementation is to define a buffer-local current-project variable;
that specifies the backend as well as any relevant information.

In any case, we only need cl-defmethod in one form or another.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-29  1:00   ` Stephen Leake
@ 2015-07-29  1:19     ` Dmitry Gutov
  2015-07-29 14:24       ` Stephen Leake
  2015-07-30 21:31     ` Stefan Monnier
  1 sibling, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-29  1:19 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 07/29/2015 04:00 AM, Stephen Leake wrote:

> Why would it use 'cl-call-next-method'? That would call the
> default method, which is wrong; that's why we're overriding it.

A project-specific implementation could call cl-call-next-method, to 
find out what the major mode thinks the search-path should be.

> Nothing else in project.el currently suggests that
> 'project-search-path-function' should be a list; the default method for
> project-search-path doesn't treat it as a list. What is the use case
> for this? I remember seeing something in the email chain; did that get
> captured somewhere?

The value is not a list. It's a function, which could take different 
values in different modes. If a project implementation is concerned with 
different languages, it could obtain the value of this variable 
corresponding to all major modes (using temporary buffers, for instance; 
or the author could just hardcode the list), then call those functions, 
and append them together.

>> However, now that we've reached the consensus that project-search-path
>> doesn't need to include the directories inside the current project,
>
> I never agreed to that. In fact, I thought the only thing we _did_ agree
> on was that project-search-path included the main project. Sigh.

Seriously? Here's a quote from one of your messages:

"""
So some specific change proposals:

- Rename 'project-directories' to 'project-root-directories' or
   'project-roots'. The current project root should always be first in
   the list.

- 'project-search-path' should not include 'project-root-directories'.
"""



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-29  1:19     ` Dmitry Gutov
@ 2015-07-29 14:24       ` Stephen Leake
  2015-07-29 19:54         ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-07-29 14:24 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/29/2015 04:00 AM, Stephen Leake wrote:
>
>>> However, now that we've reached the consensus that project-search-path
>>> doesn't need to include the directories inside the current project,
>>
>> I never agreed to that. In fact, I thought the only thing we _did_ agree
>> on was that project-search-path included the main project. Sigh.
>
> Seriously? Here's a quote from one of your messages:
>
> """
> So some specific change proposals:
>
> - Rename 'project-directories' to 'project-root-directories' or
>   'project-roots'. The current project root should always be first in
>   the list.
>
> - 'project-search-path' should not include 'project-root-directories'.
> """

That means the _code_ for default method for the function
project-search-path should not append the value of the result of the
function project-root-directories.

It says nothing about the _meaning_ of the concept
"project-search-path", nor the result of the function pointed to by
project-search-path-function; it is free to include whatever directories
it wants.

The point was I thought that project-search-path and project-directories
were distinct concepts, not related to each other. The contents may
overlap, or either one may not be defined.

Now that I see that you intended them to mean project-path-read-only and
project-path-read-write, things have changed.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-29 14:24       ` Stephen Leake
@ 2015-07-29 19:54         ` Dmitry Gutov
  2015-07-30  7:04           ` Stephen Leake
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-29 19:54 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 07/29/2015 05:24 PM, Stephen Leake wrote:

>> """
>> So some specific change proposals:
>>
>> - Rename 'project-directories' to 'project-root-directories' or
>>    'project-roots'. The current project root should always be first in
>>    the list.
>>
>> - 'project-search-path' should not include 'project-root-directories'.
>> """
>
> That means the _code_ for default method for the function
> project-search-path should not append the value of the result of the
> function project-root-directories.

It was my mistake, then. I hope, though, after rereading that message, 
you can agree that it was an easy one to make.

> The point was I thought that project-search-path and project-directories
> were distinct concepts, not related to each other. The contents may
> overlap, or either one may not be defined.

Try to imagine the consequences of project-search-path having to include 
project-roots, or even some directories within them.

Take emacs-lisp-mode, it makes project-search-path return load-path, 
through project-search-path-function.

Now, take an actual project implementation, like the one returned by 
project-try-vc. It doesn't know anything particular about Elisp, but 
there might be some directories with Elisp code inside the current 
project tree. There's no guarantee that they're added to load-path, at 
any given time. Should the VC project provide its own implementation of 
project-search-path?

> Now that I see that you intended them to mean project-path-read-only and
> project-path-read-write, things have changed.

These are pretty bad names, though.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-29 19:54         ` Dmitry Gutov
@ 2015-07-30  7:04           ` Stephen Leake
  2015-07-30 11:30             ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-07-30  7:04 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/29/2015 05:24 PM, Stephen Leake wrote:
>
>>> """
>>> So some specific change proposals:
>>>
>>> - Rename 'project-directories' to 'project-root-directories' or
>>>    'project-roots'. The current project root should always be first in
>>>    the list.
>>>
>>> - 'project-search-path' should not include 'project-root-directories'.
>>> """
>>
>> That means the _code_ for default method for the function
>> project-search-path should not append the value of the result of the
>> function project-root-directories.
>
> It was my mistake, then. I hope, though, after rereading that message,
> you can agree that it was an easy one to make.
>
>> The point was I thought that project-search-path and project-directories
>> were distinct concepts, not related to each other. The contents may
>> overlap, or either one may not be defined.
>
> Try to imagine the consequences of project-search-path having to
> include project-roots, or even some directories within them.
>
> Take emacs-lisp-mode, it makes project-search-path return load-path,
> through project-search-path-function.

Yes; that's exactly what it should do.

> Now, take an actual project implementation, like the one returned by
> project-try-vc. It doesn't know anything particular about Elisp, but
> there might be some directories with Elisp code inside the current
> project tree. There's no guarantee that they're added to load-path, at
> any given time. Should the VC project provide its own implementation
> of project-search-path?

If the "vc project backend" wants to be a "real project backend", then
yes, it needs to define project-search-path.

That would not be useful for editing elisp files; the elisp project
backend is useful for that.

Which is why elisp-mode sets the elisp project backend to be active.

I don't see the problem.

>> Now that I see that you intended them to mean project-path-read-only and
>> project-path-read-write, things have changed.
>
> These are pretty bad names, though.

Why? they give the precise meaning. What names would you propose?

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-30  7:04           ` Stephen Leake
@ 2015-07-30 11:30             ` Dmitry Gutov
  2015-07-30 15:29               ` Stephen Leake
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-30 11:30 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 07/30/2015 10:04 AM, Stephen Leake wrote:

> If the "vc project backend" wants to be a "real project backend", then
> yes, it needs to define project-search-path.

Would its definition add the VC root to the list returned by 
project-search-path-function?

> That would not be useful for editing elisp files; the elisp project
> backend is useful for that.

There is no elisp project backend. Just an elisp-specific value of 
project-search-path-function.

Which obviously omits project-roots and project-ignores.

>>> Now that I see that you intended them to mean project-path-read-only and
>>> project-path-read-write, things have changed.
>>
>> These are pretty bad names, though.
>
> Why? they give the precise meaning. What names would you propose?

project-roots sounds better: it reflects each element being a "proper" 
project, hints at the presence of project files inside, and also implies 
"read-write".

Any elements of search-path outside of project-root can be considered 
read-only.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-30 11:30             ` Dmitry Gutov
@ 2015-07-30 15:29               ` Stephen Leake
  2015-07-30 15:37                 ` Stephen Leake
  2015-07-30 17:16                 ` Dmitry Gutov
  0 siblings, 2 replies; 45+ messages in thread
From: Stephen Leake @ 2015-07-30 15:29 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/30/2015 10:04 AM, Stephen Leake wrote:
>
>> If the "vc project backend" wants to be a "real project backend", then
>> yes, it needs to define project-search-path.
>
> Would its definition add the VC root to the list returned by
> project-search-path-function?

Not if I wrote it; it would not use project-search-path-function.

The point of a "project backend" is that it knows _everything_ about the
project. No stray hooks for users to pervert things.

I would write the vc implementation of project-search-path to return a
flat list of all the directories under the vc root.

>> That would not be useful for editing elisp files; the elisp project
>> backend is useful for that.
>
> There is no elisp project backend. Just an elisp-specific value of
> project-search-path-function.
>
> Which obviously omits project-roots and project-ignores.

Right. The elisp project backend is incomplete.

That's one problem with using two dispatching mechanisms; it's too easy
to get mixtures of incomplete implementations that "seem to work".

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-30 15:29               ` Stephen Leake
@ 2015-07-30 15:37                 ` Stephen Leake
  2015-07-30 16:10                   ` Dmitry Gutov
  2015-07-30 17:16                 ` Dmitry Gutov
  1 sibling, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-07-30 15:37 UTC (permalink / raw)
  To: emacs-devel

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> I would write the vc implementation of project-search-path to return a
> flat list of all the directories under the vc root.

But leaving out directories that the vc says to ignore.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-30 15:37                 ` Stephen Leake
@ 2015-07-30 16:10                   ` Dmitry Gutov
  2015-07-30 23:25                     ` Stephen Leake
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-30 16:10 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 07/30/2015 06:37 PM, Stephen Leake wrote:

> But leaving out directories that the vc says to ignore.

You haven't addressed the point that .gitignore can also say to ignore 
some files, as well as only certain files inside certain directories 
(either anchored to the repository root, or not).



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-30 15:29               ` Stephen Leake
  2015-07-30 15:37                 ` Stephen Leake
@ 2015-07-30 17:16                 ` Dmitry Gutov
  2015-07-30 23:33                   ` Stephen Leake
  1 sibling, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-30 17:16 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 07/30/2015 06:29 PM, Stephen Leake wrote:

> The point of a "project backend" is that it knows _everything_ about the
> project. No stray hooks for users to pervert things.

The problem is I (and others) have a lot of Elisp projects which don't 
adhere to some particular structure. No particular project files, aside 
from a .git repo, and maybe Makefile, or maybe some other -file.

It's not even easy to identify it: maybe there are some .el files in the 
root directory, but they might all be in a subdirectory.

I want to support this use case.

> I would write the vc implementation of project-search-path to return a
> flat list of all the directories under the vc root.

The flat list is really out of the question. It really goes against my 
intuition and every project backend will have to implement the logic of 
producing this flat list based on some roots and a list of ignores.

We won't be able to use rgrep on the resulting directories, but we'd 
still have to handle ignored files.

I'd rather project-ignores was powerful enough to describe the majority 
of cases, and xref-find-regexp knew how to interpret that list.

> Right. The elisp project backend is incomplete.

It's not a backend. Like described above, it would be hard to even write 
a reasonably efficient project-find-functions element for such a 
backend. What will it do? Recursively scan the VC repo for .el files?

> That's one problem with using two dispatching mechanisms; it's too easy
> to get mixtures of incomplete implementations that "seem to work".

I'd call it "composability". :)



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-29  0:16     ` Dmitry Gutov
@ 2015-07-30 21:26       ` Stefan Monnier
  0 siblings, 0 replies; 45+ messages in thread
From: Stefan Monnier @ 2015-07-30 21:26 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Stephen Leake, emacs-devel

>> FWIW, this sucks because this method won't be used in modes derived from
>> emacs-lisp-mode.
> If we *really* wanted to use the same mechanism, though, I think
> implementing &context (derived-p emacs-lisp-mode) is not impossible.

I think the syntax "&context (derived-p emacs-lisp-mode)" is impossible,
but indeed, something that behaves correctly is possible, maybe with
a syntax like "&context ((parent-modes) (includes emacs-lisp-mode))"
or "&context (major-mode (derived-mode-of emacs-lisp-mode))".


        Stefan



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-29  1:00   ` Stephen Leake
  2015-07-29  1:19     ` Dmitry Gutov
@ 2015-07-30 21:31     ` Stefan Monnier
  1 sibling, 0 replies; 45+ messages in thread
From: Stefan Monnier @ 2015-07-30 21:31 UTC (permalink / raw)
  To: Stephen Leake; +Cc: emacs-devel

>> And if a project implementation has to provide its own implementation,
>> it's much cleaner, I think, if it only has to deal with
>> project-search-path-function, instead of being forced to
>> cl-call-next-method.
> Well, that's my question. I don't see that as "cleaner"; I much prefer
> object dispatching to random funcalls.

Currently, project-search-path-function takes no argument, so there's
not much on which to dispatch.


        Stefan



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-30 16:10                   ` Dmitry Gutov
@ 2015-07-30 23:25                     ` Stephen Leake
  0 siblings, 0 replies; 45+ messages in thread
From: Stephen Leake @ 2015-07-30 23:25 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/30/2015 06:37 PM, Stephen Leake wrote:
>
>> But leaving out directories that the vc says to ignore.
>
> You haven't addressed the point that .gitignore can also say to ignore
> some files, as well as only certain files inside certain directories
> (either anchored to the repository root, or not).

Yes. Obviously that _could_ be implemented in elisp, but I doubt it
would be worth it.

I don't ever anticipate use a "vc project"; a project backend based on a
project manager (gprbuild, gradle, etc) would be much more useful.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-30 17:16                 ` Dmitry Gutov
@ 2015-07-30 23:33                   ` Stephen Leake
  2015-07-31  0:37                     ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-07-30 23:33 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/30/2015 06:29 PM, Stephen Leake wrote:
>
>> The point of a "project backend" is that it knows _everything_ about the
>> project. No stray hooks for users to pervert things.
>
> The problem is I (and others) have a lot of Elisp projects which don't
> adhere to some particular structure. No particular project files,
> aside from a .git repo, and maybe Makefile, or maybe some other -file.
>
> It's not even easy to identify it: maybe there are some .el files in
> the root directory, but they might all be in a subdirectory.
>
> I want to support this use case.

Ok.

What does "support" mean here?

If the elisp project backend just used load-path as project-search-path,
what desired functionality would you lose?

You have talked about limiting xref-find-regexp to some subset of the
files that are visible thru load-path; can you give a concrete
use case for that?

>> I would write the vc implementation of project-search-path to return a
>> flat list of all the directories under the vc root.
>
> The flat list is really out of the question. It really goes against my
> intuition and every project backend will have to implement the logic
> of producing this flat list based on some roots and a list of ignores.

Well, we obviously disagree; every project manager _I've_ worked with
already produces a flat list.

I don't understand why you are so dead set against supporting those
project managers.

As long as I have the ability to override sufficient project and xref
features so I can write backends for the project managers I use, I'm ok.
It would be nice if more of the utilities were able to handle flat
paths, but I can always re-implement them.

> We won't be able to use rgrep on the resulting directories, 

You can use grep; what's wrong with that?

> but we'd still have to handle ignored files.

That's no harder with a flat path than with a recursive path.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-30 23:33                   ` Stephen Leake
@ 2015-07-31  0:37                     ` Dmitry Gutov
  2015-07-31 14:36                       ` Stephen Leake
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-31  0:37 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 07/31/2015 02:33 AM, Stephen Leake wrote:

> If the elisp project backend just used load-path as project-search-path,
> what desired functionality would you lose?

xref-find-references won't search inside the current project root.

That's about all project-search-path is currently used for, but 
hopefully other uses will crop up in the future. They would fail similarly.

> You have talked about limiting xref-find-regexp to some subset of the
> files that are visible thru load-path; can you give a concrete
> use case for that?

Maybe if we have project-find-regexp-in-roots, it will only search 
inside project-roots. And project-find-regexp will search inside roots + 
search-path. It's unrelated to the issue above.

> Well, we obviously disagree; every project manager _I've_ worked with
> already produces a flat list.

I've addressed that in the other message, I think.

> I don't understand why you are so dead set against supporting those
> project managers.

One might call "always recursing" the "modern" mindset. For instance, 
when calling Ag (a popular recent re-implementation of Grep), you have 
to make extra effort *not* to recurse. There no corresponding option, even.

> As long as I have the ability to override sufficient project and xref
> features so I can write backends for the project managers I use, I'm ok.
> It would be nice if more of the utilities were able to handle flat
> paths, but I can always re-implement them.

I hope you'll consider whether recursing is too harmful, in each case.

>> We won't be able to use rgrep on the resulting directories,
>
> You can use grep; what's wrong with that?

Not "rgrep". Instead of calling find constructed command-line arguments 
once and collecting results, you'll have to call grep for each directory 
you're interested in. And instead of find+grep, one could use Ag.

>> but we'd still have to handle ignored files.
>
> That's no harder with a flat path than with a recursive path.

I'm sure it'll be more involved than the current implementation of 
xref--rgrep-command.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-31  0:37                     ` Dmitry Gutov
@ 2015-07-31 14:36                       ` Stephen Leake
  2015-07-31 22:52                         ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-07-31 14:36 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/31/2015 02:33 AM, Stephen Leake wrote:
>
>> If the elisp project backend just used load-path as project-search-path,
>> what desired functionality would you lose?
>
> xref-find-references won't search inside the current project root.

It would help to be more specific. Since we disagree, you can't assume
I'll guess what you have in mind; my first response was "so why would
you want to?". To be truly persuasive, you need to anticipate that
response, and address it before it happens.

A concrete example directory structure:

myproj/
    # contains README, some docs

    .git/
        # contains git repo

    build/
        contains Makefile, test outputs, intermediate files
        
    lisp/
        # contains *.el files

    lisp-emacs-24.3/
        # contains *.el files only needed with emacs 24.3
        
    test/
        # contains test drivers, known good results

The user has set load-path to (".../myproj/lisp" "~/.emacs.d/elpa/..."
".../emacs-25.0/share/..." ...)

One search the user might want to do: find the name of an elisp function
in a test driver script file, to see how it is tested, while
simultaneously finding all places that function is used in the *.el
files, to review the test cases to ensure they are sufficient.

xref-find-references should do that search.

So I agree this is a case for the user adding things to
project-search-path that are not on load-path.

It would be wrong to add the entire directory tree that contains .git;
the user is using emacs 25, so they don't want to see the emacs 24.3
files. Obviously, there is a separate case where they do want to see
those files; that's a different project configuration.

We could provide:

(defun project-create (root)
 "Return a project object that has root directory ROOT.
Also enters the project in `project-root-alist'."
 ...)

`project-root-alist' associates root directories with project objects;
`project-current' walks up the current directory tree until it finds a
matching entry in the alist. If that fails, it tries the other methods
it currenty uses.

(defun project-add-search-path (project path)
 "Add PATH (a list of directories) to `project-search-path' for PROJECT.
If the directories are relative, they are first expanded relative to
`default-directory'."
 ...
 )

(defun project-add-search-ignore (project ignores)
 "Add IGNORES (a list of shell glob patterns) to `project-ignores'
for PROJECT."
 ...)

Then, in the same place they add myproj/lisp to load-path, the user
adds:

(defvar my-proj (project-create ".../myproj")) 
(project-add-search-path my-proj
  '(".../myproj/build" ".../myproj/test"))

# ignore intermediate test files
(project-add-search-ignore my-proj '(".../myproj/build/*.temp"))


xref-find-references then searches all files in all directories on
`project-search-path' and respects `project-ignores'.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-31 14:36                       ` Stephen Leake
@ 2015-07-31 22:52                         ` Dmitry Gutov
  2015-08-01 10:21                           ` Stephen Leake
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-07-31 22:52 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 07/31/2015 05:36 PM, Stephen Leake wrote:

> It would help to be more specific. Since we disagree, you can't assume
> I'll guess what you have in mind; my first response was "so why would
> you want to?". To be truly persuasive, you need to anticipate that
> response, and address it before it happens.

I'm sorry, I think I've addressed that question in one (or several) 
previous messages. I'm not a native speaker, and simply writing that the 
first time takes a sizable amount of effort. Repeating, and rephrasing, 
and reiterating, adds to that.

Further, I wasn't sure which aspect of the question you wanted 
addressed: the question was "which functionality", not "why". The 
assumption was that you're already clear on the latter.

To sum up: false negative results (not finding a valid occurrence of a 
given symbol) are much worse than false positives (finding a false 
occurrence of the given symbol, for instance, finding words in README 
when looking for a Java method). So it's better when our default 
behavior, with zero configuration performed by the user (maybe yet, or 
maybe they're not going to configure anything further at all), leans 
toward false positives rather than false negatives.

For those reasons, I've left the current implementation of 
project-search-path largely intact in the recent commit. If the 
backend-specialized implementation has no extra information (the user 
hasn't performed any relevant configuration), then project-search-path 
includes entire project-roots.

> A concrete example directory structure:
> ...
> The user has set load-path to (".../myproj/lisp" "~/.emacs.d/elpa/..."
> ".../emacs-25.0/share/..." ...)

Maybe they didn't, but even if they did...

> One search the user might want to do: find the name of an elisp function
> in a test driver script file, to see how it is tested, while
> simultaneously finding all places that function is used in the *.el
> files, to review the test cases to ensure they are sufficient.

Indeed. It's a common operation.

> It would be wrong to add the entire directory tree that contains .git;
> the user is using emacs 25, so they don't want to see the emacs 24.3
> files. Obviously, there is a separate case where they do want to see
> those files; that's a different project configuration.

It's really impractical, IMO, to have different project configurations 
for different use cases. A fancy project backend could have those, 
though, and the user would be able to switch to their heart's content. 
We are not discussing a fancy project backend here, though.

To be clear, here are the two main "separate cases": navigating to all 
definitions, and aliases for different Emacs versions; navigating to all 
references to a given function, again, in code for different Emacs 
versions (to be able to rename them, for instance). As a project 
maintainer, I want all of those visible and on the tips of my fingers, 
because I need to maintain all code. Not just the code currently loaded 
in my Emacs.

> We could provide:
>
> (defun project-create (root)
> ...
> `project-root-alist' associates root directories with project objects;
> ...
> (defun project-add-search-path (project path)
> ...
> (defun project-add-search-ignore (project ignores)

Sorry, this is too much a centralized, manual configuration for my 
taste. The VC project backend is simple, and requires minimal 
intervention from the user. We could add some variables (to be set via 
.dir-locals.el), but I don't want to have to add anything to my init.el 
as soon as I start working on a new project. For one thing, my .emacs.d 
is published on GitHub, for all to see.

What you've described here, could be a separate project backend. One 
you're welcome to write.

> Then, in the same place they add myproj/lisp to load-path, the user
> adds:

Again, in all likelihood they don't do that anywhere. As a real example, 
I have a few small Elisp one-file packages, the Git checkouts of which 
are never in load-path.

During the (infrequent) bouts of their development, I evaluate the whole 
buffer, write the changes, test them interactively, then commit and 
push. Then wait a while while MELPA builds the new version, and upgrade 
the installed one via M-x list-packages U RET.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-07-31 22:52                         ` Dmitry Gutov
@ 2015-08-01 10:21                           ` Stephen Leake
  2015-08-01 12:09                             ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-08-01 10:21 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/31/2015 05:36 PM, Stephen Leake wrote:
>
>> It would be wrong to add the entire directory tree that contains .git;
>> the user is using emacs 25, so they don't want to see the emacs 24.3
>> files. Obviously, there is a separate case where they do want to see
>> those files; that's a different project configuration.
>
> It's really impractical, IMO, to have different project configurations
> for different use cases. 

Not for me.

> A fancy project backend could have those,
> though, and the user would be able to switch to their heart's content.

Switching is just selecting a different project file.

> We are not discussing a fancy project backend here, though.

I am.

I want to ensure the core project API does not prevent me from writing
the backends I want. It must also provide sufficient reason for me to
use it; the fewer features I need that it has, the less reason I have to
use it.

> To be clear, here are the two main "separate cases": navigating to all
> definitions, and aliases for different Emacs versions; navigating to
> all references to a given function, again, in code for different Emacs
> versions (to be able to rename them, for instance). As a project
> maintainer, I want all of those visible and on the tips of my fingers,
> because I need to maintain all code. Not just the code currently
> loaded in my Emacs.

Yes, that is a valid use case.

And if the default project behavior, without any user add-search-path or
add-ignores, gives you that, that's fine.

>> We could provide:
>>
>> (defun project-create (root)
>> ...
>> `project-root-alist' associates root directories with project objects;
>> ...
>> (defun project-add-search-path (project path)
>> ...
>> (defun project-add-search-ignore (project ignores)
>
> Sorry, this is too much a centralized, manual configuration for my
> taste. 

Is it a problem if they are present for others to use?

We are discussing a core Emacs feature that will be used by many people;
it cannot cater to only your personal opinions.

> What you've described here, could be a separate project backend. One
> you're welcome to write.

Any backend that supports project files will need functions like these;
project files are precisely centralized manual configuration.

So they belong in the root class.

If I end up writing everything in the backend, there's no point in using
the "unified project interface".

> As a real
> example, I have a few small Elisp one-file packages, the Git checkouts
> of which are never in load-path.

Ok, I agree it would be good if the default project implementation
supports this case.


-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-01 10:21                           ` Stephen Leake
@ 2015-08-01 12:09                             ` Dmitry Gutov
  2015-08-01 14:20                               ` Stephen Leake
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-01 12:09 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 08/01/2015 01:21 PM, Stephen Leake wrote:

> I want to ensure the core project API does not prevent me from writing
> the backends I want. It must also provide sufficient reason for me to
> use it; the fewer features I need that it has, the less reason I have to
> use it.

I think the API is sufficiently powerful for that already (flat vs 
recursive is the main unresolved issue). As for features, hopefully the 
list will grow, but many of them can come, eventually, from third-party 
packages.

> Is it a problem if they are present for others to use?
>
> We are discussing a core Emacs feature that will be used by many people;
> it cannot cater to only your personal opinions.

So far it's mostly you and me in this discussion. But there are existing 
packages that implement project support for Emacs, and the ones I've 
used (Projectile and Eproject) don't work like that either.

>> What you've described here, could be a separate project backend. One
>> you're welcome to write.
>
> Any backend that supports project files will need functions like these;
> project files are precisely centralized manual configuration.

Not at all, they need variables. You can easily set the variables from 
.dir-locals.el, and any Emacs-wielding team member on the same project 
will use them automatically.

So far I'm fuzzy on whether the variables should be global, or 
project-vc specific. Probably the latter.

> So they belong in the root class.

I'm pretty sure you expend a lot more effort on parsing any given kind 
of project file, than storing ignores or search-path would take.

> If I end up writing everything in the backend, there's no point in using
> the "unified project interface".

It's an interface, like the "I" in API. Backends implement a set of 
methods that other functions can use.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-01 12:09                             ` Dmitry Gutov
@ 2015-08-01 14:20                               ` Stephen Leake
  2015-08-01 16:49                                 ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-08-01 14:20 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/01/2015 01:21 PM, Stephen Leake wrote:
>
>> Any backend that supports project files will need functions like these;
>> project files are precisely centralized manual configuration.
>
> Not at all, they need variables. You can easily set the variables from
> .dir-locals.el, and any Emacs-wielding team member on the same project
> will use them automatically.

Ok, I agree they could be variables.

The point is the name should be declared in project.el, so all backends
use the same name.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-01 14:20                               ` Stephen Leake
@ 2015-08-01 16:49                                 ` Dmitry Gutov
  2015-08-01 19:08                                   ` Stephen Leake
  2015-08-04 19:59                                   ` João Távora
  0 siblings, 2 replies; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-01 16:49 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 08/01/2015 05:20 PM, Stephen Leake wrote:

>> Not at all, they need variables. You can easily set the variables from
>> .dir-locals.el, and any Emacs-wielding team member on the same project
>> will use them automatically.
>
> Ok, I agree they could be variables.

Thinking about this more, buffer-local variables have their own 
complications:

- You need an buffer open, inside the given directory tree, to get their 
values. The project-find-functions API just passes in DIR to get a 
project instance. There might not even be a buffer open in that directory.

- If a user tries to set their values in init.el, that won't end well.

> The point is the name should be declared in project.el, so all backends
> use the same name.

You can't reuse the variables. There might be several project instances 
open at the same time.

The best bet seems to be to store them in the project file, which a 
project backend reads and saves to the project instance's slots.

Not sure what this means for the VC project backend. Git allows storing 
arbitrary key-value pairs in .git/config via 'git config', but that 
isn't true for every VCS. Bazaar doesn't, AFAICT.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-01 16:49                                 ` Dmitry Gutov
@ 2015-08-01 19:08                                   ` Stephen Leake
  2015-08-04 19:59                                   ` João Távora
  1 sibling, 0 replies; 45+ messages in thread
From: Stephen Leake @ 2015-08-01 19:08 UTC (permalink / raw)
  To: emacs-devel

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/01/2015 05:20 PM, Stephen Leake wrote:
>
>>> Not at all, they need variables. You can easily set the variables from
>>> .dir-locals.el, and any Emacs-wielding team member on the same project
>>> will use them automatically.
>>
>> Ok, I agree they could be variables.
>
> Thinking about this more, buffer-local variables have their own
> complications:
>
> - You need an buffer open, inside the given directory tree, to get
> their values. The project-find-functions API just passes in DIR to get
> a project instance. There might not even be a buffer open in that
> directory.
>
> - If a user tries to set their values in init.el, that won't end well.
>
>> The point is the name should be declared in project.el, so all backends
>> use the same name.
>
> You can't reuse the variables. There might be several project
> instances open at the same time.

Yes; I assumed the variables would be read once by some project-create
function.

But that's one reason why I proposed functions in the first place.

> The best bet seems to be to store them in the project file, which a
> project backend reads and saves to the project instance's slots.

Yes.

> Not sure what this means for the VC project backend. Git allows
> storing arbitrary key-value pairs in .git/config via 'git config', but
> that isn't true for every VCS. Bazaar doesn't, AFAICT.

I would not mess with .git/config; that's not an Emacs project file, nor
is it intended to be one. It might be useful to read some info from it,
but there must be a separate Emacs project file.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-01 16:49                                 ` Dmitry Gutov
  2015-08-01 19:08                                   ` Stephen Leake
@ 2015-08-04 19:59                                   ` João Távora
  2015-08-04 20:56                                     ` Dmitry Gutov
  2015-08-05  7:02                                     ` Stephen Leake
  1 sibling, 2 replies; 45+ messages in thread
From: João Távora @ 2015-08-04 19:59 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Stephen Leake, emacs-devel

On Sat, Aug 1, 2015 at 5:49 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 08/01/2015 05:20 PM, Stephen Leake wrote:
>
>>> Not at all, they need variables. You can easily set the variables from
>>> .dir-locals.el, and any Emacs-wielding team member on the same project
>>> will use them automatically.
> Thinking about this more, buffer-local variables have their own
> complications:
>
> - You need an buffer open, inside the given directory tree, to get their
> values. The project-find-functions API just passes in DIR to get a project
> instance. There might not even be a buffer open in that directory.

You can probably

(with-temp-buffer (setq default-directory DIR) (hack-dir-local-variables))
   ...)

Won't that work? I use this ocasionally for my personal horrible ad-hoc
project  managing code (that I hope to replace with project.el eventually).

> - If a user tries to set their values in init.el, that won't end well.

What happens by default when a user sets other dir-local variables in
her init file? I'd be happy with that ending.

> The best bet seems to be to store them in the project file, which a project
> backend reads and saves to the project instance's slots.

I'd prefer that .dir-locals is used if you can manage it. I don't want to
have to debug other sources of variable settings other than the ones
I have to know now. It also makes backend writing much easier.

-- 
João Távora



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-04 19:59                                   ` João Távora
@ 2015-08-04 20:56                                     ` Dmitry Gutov
  2015-08-04 22:43                                       ` João Távora
  2015-08-07 15:02                                       ` Stefan Monnier
  2015-08-05  7:02                                     ` Stephen Leake
  1 sibling, 2 replies; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-04 20:56 UTC (permalink / raw)
  To: João Távora; +Cc: Stephen Leake, emacs-devel

On 08/04/2015 10:59 PM, João Távora wrote:

> You can probably
>
> (with-temp-buffer (setq default-directory DIR) (hack-dir-local-variables))
>     ...)
>
> Won't that work? I use this ocasionally for my personal horrible ad-hoc
> project  managing code (that I hope to replace with project.el eventually).

I suppose. Originally, I discarded this kind of idea as too hacky, but 
indeed, there's value in limiting ourselves to .dir-locals.el here.

To continue this line of thought, would you say the variables should 
have global meaning (meaning all project backends should honor them), or 
should they be only used in the VC project backend?

I've been leaning towards the latter, but project-vc-ignores is a pretty 
terrible name, in this context.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-04 20:56                                     ` Dmitry Gutov
@ 2015-08-04 22:43                                       ` João Távora
  2015-08-04 23:35                                         ` Dmitry Gutov
                                                           ` (2 more replies)
  2015-08-07 15:02                                       ` Stefan Monnier
  1 sibling, 3 replies; 45+ messages in thread
From: João Távora @ 2015-08-04 22:43 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Stephen Leake, emacs-devel

On Tue, Aug 4, 2015 at 9:56 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 08/04/2015 10:59 PM, João Távora wrote:
. Originally, I discarded this kind of idea as too hacky, but
> indeed, there's value in limiting ourselves to .dir-locals.el here.
>
> To continue this line of thought, would you say the variables should have
> global meaning (meaning all project backends should honor them), or should
> they be only used in the VC project backend?

Hmmm, you've kind of stumped me. I don't have enough context to answer.
I just cringed a bit at the idea of having to edit a different kind of
file for setting
emacs variables. I suppose you would provide something like
add-dir-local-variable, but still...

> I've been leaning towards the latter, but project-vc-ignores is a pretty
> terrible name, in this context.

Assuming my half-baked understanding of all of this is minimally
correct, I'd say "global" though. I might want to change my project
from make to cmake to something else and keep my list of ignored dirs,
right?

I wrote somewhere else that I think some kind of vision is needed
for the whole thing (a simple ASCII diagram would do for
me). Personally, I tend to like your approach and it seems to be modular
and simple, but I got lost in the details of your discussion with Stephen...

Is the general picture something like this?


                     uses              auto-picks one
                   fns from         and gets info from

  +---------------+        +----------+        +----------------+
  |    xref       |        |          |        |   project-vc   |
  |  M-x compile  |------->|project.el|------->|project-makefile|
  |grep-my-project|        |          |        |  project-xcode |
  +---------------+        +----------+        +----------------+


--
João Távora



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-04 22:43                                       ` João Távora
@ 2015-08-04 23:35                                         ` Dmitry Gutov
  2015-08-10  1:09                                         ` Dmitry Gutov
  2015-08-10  3:07                                         ` Stephen Leake
  2 siblings, 0 replies; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-04 23:35 UTC (permalink / raw)
  To: João Távora; +Cc: Stephen Leake, emacs-devel

On 08/05/2015 01:43 AM, João Távora wrote:

> Assuming my half-baked understanding of all of this is minimally
> correct, I'd say "global" though. I might want to change my project
> from make to cmake to something else and keep my list of ignored dirs,
> right?

I suppose that depends on whether the Makefile format the "make project" 
already includes a means to specify the search-path and the ignore 
patterns. Same for "cmake project" and CMakeLists.txt.

But if you're going to the VC project backend for all those projects 
anyway, the answer to the question won't make a difference.

So I suppose we could introduce those variables globally, but then say 
that the respective methods must use them, unless the project file 
includes syntax allowing the user to specify those values there.

> I wrote somewhere else that I think some kind of vision is needed
> for the whole thing (a simple ASCII diagram would do for
> me). Personally, I tend to like your approach and it seems to be modular
> and simple, but I got lost in the details of your discussion with Stephen...
>
> Is the general picture something like this?

There no solid plan on this, but I'd expect the functions on the left 
all start with project-. So M-x rgrep uses the current directory, and 
M-x project-rgrep uses the project-roots combined with 
project-search-path (maybe we'll have two commands for that: one won't 
include search-path, and another will).



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-04 19:59                                   ` João Távora
  2015-08-04 20:56                                     ` Dmitry Gutov
@ 2015-08-05  7:02                                     ` Stephen Leake
  1 sibling, 0 replies; 45+ messages in thread
From: Stephen Leake @ 2015-08-05  7:02 UTC (permalink / raw)
  To: emacs-devel

João Távora <joaotavora@gmail.com> writes:

> On Sat, Aug 1, 2015 at 5:49 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
>> On 08/01/2015 05:20 PM, Stephen Leake wrote:
>>
>> The best bet seems to be to store them in the project file, which a project
>> backend reads and saves to the project instance's slots.
>
> I'd prefer that .dir-locals is used if you can manage it. I don't want to
> have to debug other sources of variable settings other than the ones
> I have to know now. It also makes backend writing much easier.

Well, you learned about .dir-locals at some point. If a standard Emacs
project file had been available then, you might have learned that
instead :).

But since .dir-locals exists, it would make sense to have a project
implementation that uses .dir-local information as much as possible.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-04 20:56                                     ` Dmitry Gutov
  2015-08-04 22:43                                       ` João Távora
@ 2015-08-07 15:02                                       ` Stefan Monnier
  2015-08-07 15:32                                         ` Dmitry Gutov
  1 sibling, 1 reply; 45+ messages in thread
From: Stefan Monnier @ 2015-08-07 15:02 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Stephen Leake, João Távora, emacs-devel

> I suppose. Originally, I discarded this kind of idea as too hacky, but
> indeed, there's value in limiting ourselves to .dir-locals.el here.
> To continue this line of thought, would you say the variables should have
> global meaning (meaning all project backends should honor them), or should
> they be only used in the VC project backend?

Why choose: add a project method to get the variables (and make it use
hack-dir-local-variables by default).


        Stefan



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-07 15:02                                       ` Stefan Monnier
@ 2015-08-07 15:32                                         ` Dmitry Gutov
  2015-08-07 17:36                                           ` Stefan Monnier
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-07 15:32 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Stephen Leake, João Távora, emacs-devel

On 08/07/2015 06:02 PM, Stefan Monnier wrote:

> Why choose: add a project method to get the variables (and make it use
> hack-dir-local-variables by default).

That seems redundant: we already have project methods that return the 
respective resulting values.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-07 15:32                                         ` Dmitry Gutov
@ 2015-08-07 17:36                                           ` Stefan Monnier
  0 siblings, 0 replies; 45+ messages in thread
From: Stefan Monnier @ 2015-08-07 17:36 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Stephen Leake, João Távora, emacs-devel

>> Why choose: add a project method to get the variables (and make it use
>> hack-dir-local-variables by default).
> That seems redundant: we already have project methods that return the
> respective resulting values.

Then don't mind me,


        Stefan



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-04 22:43                                       ` João Távora
  2015-08-04 23:35                                         ` Dmitry Gutov
@ 2015-08-10  1:09                                         ` Dmitry Gutov
  2015-08-10  3:07                                         ` Stephen Leake
  2 siblings, 0 replies; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-10  1:09 UTC (permalink / raw)
  To: João Távora; +Cc: Stephen Leake, emacs-devel

On 08/05/2015 01:43 AM, João Távora wrote:

>> To continue this line of thought, would you say the variables should have
>> global meaning (meaning all project backends should honor them), or should
>> they be only used in the VC project backend?

They're local to the VC project backend now.

To make them kinda-global, I haven't found good enough names, and the 
docstrings were going to have too many caveats (like how different 
implementations can, but don't have to, use them).



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-04 22:43                                       ` João Távora
  2015-08-04 23:35                                         ` Dmitry Gutov
  2015-08-10  1:09                                         ` Dmitry Gutov
@ 2015-08-10  3:07                                         ` Stephen Leake
  2015-08-10  8:45                                           ` João Távora
  2015-08-10 18:02                                           ` Dmitry Gutov
  2 siblings, 2 replies; 45+ messages in thread
From: Stephen Leake @ 2015-08-10  3:07 UTC (permalink / raw)
  To: emacs-devel

João Távora <joaotavora@gmail.com> writes:

> I wrote somewhere else that I think some kind of vision is needed
> for the whole thing (a simple ASCII diagram would do for
> me). Personally, I tend to like your approach and it seems to be modular
> and simple, but I got lost in the details of your discussion with Stephen...
>
> Is the general picture something like this?
>
>
>                      uses              auto-picks one
>                    fns from         and gets info from
>
>   +---------------+        +----------+        +----------------+
>   |    xref       |        |          |        |   project-vc   |
>   |  M-x compile  |------->|project.el|------->|project-makefile|
>   |grep-my-project|        |          |        |  project-xcode |
>   +---------------+        +----------+        +----------------+

That's the design I'm trying to follow, but I'm not clear what Dmitry
thinks; he doesn't like grep-project, for example.

On the other hand, you can in principle use 'ede.el' instead of
'project.el'. I'm persuing that idea for a while; if that works well
enough, then there's no need for project.el.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10  3:07                                         ` Stephen Leake
@ 2015-08-10  8:45                                           ` João Távora
  2015-08-10 16:50                                             ` Stephen Leake
  2015-08-10 18:07                                             ` Dmitry Gutov
  2015-08-10 18:02                                           ` Dmitry Gutov
  1 sibling, 2 replies; 45+ messages in thread
From: João Távora @ 2015-08-10  8:45 UTC (permalink / raw)
  To: Stephen Leake; +Cc: emacs-devel

On Mon, Aug 10, 2015 at 4:07 AM, Stephen Leake
<stephen_leake@stephe-leake.org> wrote:
> João Távora <joaotavora@gmail.com> writes:
>
>> I wrote somewhere else that I think some kind of vision is needed
>> for the whole thing (a simple ASCII diagram would do for
>> me). Personally, I tend to like your approach and it seems to be modular
>> and simple, but I got lost in the details of your discussion with Stephen...
>>
>> Is the general picture something like this?
>>
>>
>>                      uses              auto-picks one
>>                    fns from         and gets info from
>>
>>   +---------------+        +----------+        +----------------+
>>   |    xref       |        |          |        |   project-vc   |
>>   |  M-x compile  |------->|project.el|------->|project-makefile|
>>   |grep-my-project|        |          |        |  project-xcode |
>>   +---------------+        +----------+        +----------------+
>
> That's the design I'm trying to follow, but I'm not clear what Dmitry
> thinks; he doesn't like grep-project, for example.

Perhaps he doesn't because xref kind of supersedes it right, it?  But
I agree it should be possible to implement grep-project and to get the
dirs it needs from a single interface in project.el

> On the other hand, you can in principle use 'ede.el' instead of
> 'project.el'. I'm persuing that idea for a while; if that works well
> enough, then there's no need for project.el.

I suppose. I tried, but not very hard, ede.el in the past. The reason
I didn't try very hard is that it scared me. If I recall pcorrectly,
there a lot to setup, new concepts to learn... and a funnyly named
file to edit:

I'm really hoping project.el vc-aware interface can:

* automatically understand 90% of my projects automatically.

* For the other 5% I'll try to make the projects themselves conform to
  some other standard representation that some other project backend
  automatically understands, i.e. I would rename targets of Makefiles
  and relocate dependencies if that helped some hypothetocal backend
  (that I assume is well designed)

* For the other 5% I'll bite the bullet and edit a file at the root
  of the project. I just hope that file is .dir-locals.el.

-- 
João Távora



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10  8:45                                           ` João Távora
@ 2015-08-10 16:50                                             ` Stephen Leake
  2015-08-10 19:38                                               ` João Távora
  2015-08-10 18:07                                             ` Dmitry Gutov
  1 sibling, 1 reply; 45+ messages in thread
From: Stephen Leake @ 2015-08-10 16:50 UTC (permalink / raw)
  To: emacs-devel

João Távora <joaotavora@gmail.com> writes:

> On Mon, Aug 10, 2015 at 4:07 AM, Stephen Leake
> <stephen_leake@stephe-leake.org> wrote:
>> João Távora <joaotavora@gmail.com> writes:
>>
>
>> On the other hand, you can in principle use 'ede.el' instead of
>> 'project.el'. I'm persuing that idea for a while; if that works well
>> enough, then there's no need for project.el.
>
> I suppose. I tried, but not very hard, ede.el in the past. The reason
> I didn't try very hard is that it scared me. If I recall pcorrectly,
> there a lot to setup, new concepts to learn... and a funnyly named
> file to edit:

EDE has gotten better, and the developers are still actively improving it.

> I'm really hoping project.el vc-aware interface can:
>
> * automatically understand 90% of my projects automatically.
>
> * For the other 5% I'll try to make the projects themselves conform to
>   some other standard representation that some other project backend
>   automatically understands, i.e. I would rename targets of Makefiles
>   and relocate dependencies if that helped some hypothetocal backend
>   (that I assume is well designed)
>
> * For the other 5% I'll bite the bullet and edit a file at the root
>   of the project. I just hope that file is .dir-locals.el.

I believe the current development version of EDE meets these goals, or
at least desires to (except for the name of the project description
file).

In particular, the user guide has been much improved.

Both the Emacs bundled version and the SourceForge development version
of EDE are a little broken right now, because of the eieio and
cl-generic changes in Emacs master. So you should wait a while before
trying it again.

-- 
-- Stephe



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10  3:07                                         ` Stephen Leake
  2015-08-10  8:45                                           ` João Távora
@ 2015-08-10 18:02                                           ` Dmitry Gutov
  1 sibling, 0 replies; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-10 18:02 UTC (permalink / raw)
  To: Stephen Leake, emacs-devel

On 08/10/2015 06:07 AM, Stephen Leake wrote:

> That's the design I'm trying to follow, but I'm not clear what Dmitry
> thinks; he doesn't like grep-project, for example.

I replied to that here: 
http://lists.gnu.org/archive/html/emacs-devel/2015-08/msg00287.html

grep-project is okay, but I like using xref, as well as project- 
namespace, better.

> there's no need for project.el.

That sounds rather offensive.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10  8:45                                           ` João Távora
  2015-08-10 16:50                                             ` Stephen Leake
@ 2015-08-10 18:07                                             ` Dmitry Gutov
  2015-08-10 19:21                                               ` João Távora
  1 sibling, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-10 18:07 UTC (permalink / raw)
  To: João Távora, Stephen Leake; +Cc: emacs-devel

On 08/10/2015 11:45 AM, João Távora wrote:

> But
> I agree it should be possible to implement grep-project and to get the
> dirs it needs from a single interface in project.el

I see no technical obstacles there.

> * For the other 5% I'll try to make the projects themselves conform to
>    some other standard representation that some other project backend
>    automatically understands, i.e. I would rename targets of Makefiles
>    and relocate dependencies if that helped some hypothetocal backend
>    (that I assume is well designed)

To be honest, I'm not sure how to better serve this use case.

Either someone will have to write another backend (or several) that 
target different build systems, or the VC backend will have to be 
equipped with a pluggable facility for reading project files.

Some combination of the two will probably turn out to be optimal.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10 18:07                                             ` Dmitry Gutov
@ 2015-08-10 19:21                                               ` João Távora
  2015-08-10 19:37                                                 ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: João Távora @ 2015-08-10 19:21 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Stephen Leake, emacs-devel

On Mon, Aug 10, 2015 at 7:07 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 08/10/2015 11:45 AM, João Távora wrote:
>
>> But
>> I agree it should be possible to implement grep-project and to get the
>> dirs it needs from a single interface in project.el
>
> I see no technical obstacles there.

That's what I suspected, but Stephen said that you don't like it. I mistakenly
took that as a suggestion that you wouldn't add the functionality to support
it.

>> * For the other 5% I'll try to make the projects themselves conform to
>>    some other standard representation that some other project backend
>>    automatically understands, i.e. I would rename targets of Makefiles
>>    and relocate dependencies if that helped some hypothetocal backend
>>    (that I assume is well designed)
>
> To be honest, I'm not sure how to better serve this use case.
>
> Either someone will have to write another backend (or several) that target
> different build systems, or the VC backend will have to be equipped with a
> pluggable facility for reading project files.
>
> Some combination of the two will probably turn out to be optimal.

I agree. But I think you misunderstood me, these particular 5% are the ones
where you're "off-the-hook", meaning I'll adapt my project to to fit
some existing
backend. For example, if I have a project in some obscure VCS that the
vc backend
doesn't support, I might consider switching VCS systems.

-- 
João Távora



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10 19:21                                               ` João Távora
@ 2015-08-10 19:37                                                 ` Dmitry Gutov
  2015-08-10 19:47                                                   ` João Távora
  0 siblings, 1 reply; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-10 19:37 UTC (permalink / raw)
  To: João Távora; +Cc: Stephen Leake, emacs-devel

On 08/10/2015 10:21 PM, João Távora wrote:

> That's what I suspected, but Stephen said that you don't like it. I mistakenly
> took that as a suggestion that you wouldn't add the functionality to support
> it.

There's no way I even could do that: grep-project would use the same 
project- methods as xref-find-regexp does now.

>> Either someone will have to write another backend (or several) that target
>> different build systems, or the VC backend will have to be equipped with a
>> pluggable facility for reading project files.
>>
>> Some combination of the two will probably turn out to be optimal.
>
> I agree. But I think you misunderstood me, these particular 5% are the ones
> where you're "off-the-hook",

Well, you mentioned Makefile targets, and we don't do anything with 
those, so far. There's no "default" project structure which you could 
adapt your projects for, except for "a root directory with files in it", 
and depending on the language, etc, we might have to several of those.



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10 16:50                                             ` Stephen Leake
@ 2015-08-10 19:38                                               ` João Távora
  0 siblings, 0 replies; 45+ messages in thread
From: João Távora @ 2015-08-10 19:38 UTC (permalink / raw)
  To: Stephen Leake; +Cc: emacs-devel

On Mon, Aug 10, 2015 at 5:50 PM, Stephen Leake
<stephen_leake@stephe-leake.org> wrote:
> João Távora <joaotavora@gmail.com> writes:
>> I suppose. I tried, but not very hard, ede.el in the past. The reason
>> I didn't try very hard is that it scared me. If I recall pcorrectly,
>> there a lot to setup, new concepts to learn... and a funnyly named
>> file to edit:
>
> EDE has gotten better, and the developers are still actively improving it.

I (require 'ede) in a recentish trunk and tried (ede-project-root) only to
find out that it requires an argument, and I couldn't make out how to
pass it something like `ede-guess-current-project'`. How can I do that?

In project.el, which I just tried for the first time, `project-root'
also requires
and argument, but it was rather trivial to find that `project-current` returns
one, and it works for 90% of my projects, which are git-based.

It's enough for me to throw away my

(defun joaot/vc-get-vc-dir-or-die (&optional dont-die)
  (or (locate-dominating-file default-directory ".git")
      (and (not dont-die)
           (error "Could not find git project dir from here (~a)"
                    default-directory))))

and I'm sure it'll work on many more situations. And Dmitry seems to be
working on functionality (xref) that makes my

(defun joaot/vc-rgrep (regexp) ...)

the most useful of my commands, not only obsolete, but ridiculously
limited in comparison.

> I believe the current development version of EDE meets these goals, or
> at least desires to (except for the name of the project description
> file).
>
> In particular, the user guide has been much improved.

I believe you and I've only very briefly skimmed it. It seems, though, to
be aimed at "creating projects". I'm interested in a tool that understands
existing projects. Or in tools that help me develop those tools.

> Both the Emacs bundled version and the SourceForge development version
> of EDE are a little broken right now, because of the eieio and
> cl-generic changes in Emacs master. So you should wait a while before
> trying it again.

That doesn't help EDE's case :-). But OK.

-- 
João Távora



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10 19:37                                                 ` Dmitry Gutov
@ 2015-08-10 19:47                                                   ` João Távora
  2015-08-10 19:55                                                     ` Dmitry Gutov
  0 siblings, 1 reply; 45+ messages in thread
From: João Távora @ 2015-08-10 19:47 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Stephen Leake, emacs-devel

On Mon, Aug 10, 2015 at 8:37 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:

> Well, you mentioned Makefile targets, and we don't do anything with those,
> so far. There's no "default" project structure which you could adapt your
> projects for, except for "a root directory with files in it", and depending
> on the language, etc, we might have to several of those.

We might be miscommunicating again. This is what I meant:

I'm not expecting you to develop it but perhaps in the future there will be a
project.el-based backend that, instead if looking up the tree for the
.git dir, looks for Makefiles. And perhaps in the future it might even parse
them and collect, say, targets. If "project target" is a concept ever added
to  project.el (not saying it should, but it could), then

   (project-targets (current-project))

could return some interesting info for that backend. And I would consider
restructuring my project if that could enable some tools that use
`project-targets`.

-- 
João Távora



^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: cl-defgeneric vs random funcall in project.el
  2015-08-10 19:47                                                   ` João Távora
@ 2015-08-10 19:55                                                     ` Dmitry Gutov
  0 siblings, 0 replies; 45+ messages in thread
From: Dmitry Gutov @ 2015-08-10 19:55 UTC (permalink / raw)
  To: João Távora; +Cc: Stephen Leake, emacs-devel

On 08/10/2015 10:47 PM, João Távora wrote:

> I'm not expecting you to develop it but perhaps in the future there will be a
> project.el-based backend that, instead if looking up the tree for the
> .git dir, looks for Makefiles.

Sounds good. That's close to what I meant as well, hence "someone will 
have to write another backend" in the original message.

> If "project target" is a concept ever added
> to  project.el (not saying it should, but it could),

I think it should, together with a tool or two that use it.



^ permalink raw reply	[flat|nested] 45+ messages in thread

end of thread, other threads:[~2015-08-10 19:55 UTC | newest]

Thread overview: 45+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-07-28 11:31 cl-defgeneric vs random funcall in project.el Stephen Leake
2015-07-28 15:09 ` Dmitry Gutov
2015-07-28 23:57   ` Stefan Monnier
2015-07-29  0:16     ` Dmitry Gutov
2015-07-30 21:26       ` Stefan Monnier
2015-07-29  1:00   ` Stephen Leake
2015-07-29  1:19     ` Dmitry Gutov
2015-07-29 14:24       ` Stephen Leake
2015-07-29 19:54         ` Dmitry Gutov
2015-07-30  7:04           ` Stephen Leake
2015-07-30 11:30             ` Dmitry Gutov
2015-07-30 15:29               ` Stephen Leake
2015-07-30 15:37                 ` Stephen Leake
2015-07-30 16:10                   ` Dmitry Gutov
2015-07-30 23:25                     ` Stephen Leake
2015-07-30 17:16                 ` Dmitry Gutov
2015-07-30 23:33                   ` Stephen Leake
2015-07-31  0:37                     ` Dmitry Gutov
2015-07-31 14:36                       ` Stephen Leake
2015-07-31 22:52                         ` Dmitry Gutov
2015-08-01 10:21                           ` Stephen Leake
2015-08-01 12:09                             ` Dmitry Gutov
2015-08-01 14:20                               ` Stephen Leake
2015-08-01 16:49                                 ` Dmitry Gutov
2015-08-01 19:08                                   ` Stephen Leake
2015-08-04 19:59                                   ` João Távora
2015-08-04 20:56                                     ` Dmitry Gutov
2015-08-04 22:43                                       ` João Távora
2015-08-04 23:35                                         ` Dmitry Gutov
2015-08-10  1:09                                         ` Dmitry Gutov
2015-08-10  3:07                                         ` Stephen Leake
2015-08-10  8:45                                           ` João Távora
2015-08-10 16:50                                             ` Stephen Leake
2015-08-10 19:38                                               ` João Távora
2015-08-10 18:07                                             ` Dmitry Gutov
2015-08-10 19:21                                               ` João Távora
2015-08-10 19:37                                                 ` Dmitry Gutov
2015-08-10 19:47                                                   ` João Távora
2015-08-10 19:55                                                     ` Dmitry Gutov
2015-08-10 18:02                                           ` Dmitry Gutov
2015-08-07 15:02                                       ` Stefan Monnier
2015-08-07 15:32                                         ` Dmitry Gutov
2015-08-07 17:36                                           ` Stefan Monnier
2015-08-05  7:02                                     ` Stephen Leake
2015-07-30 21:31     ` Stefan Monnier

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).