Re: project.el semantics

[Stephen Leake <stephen_leake@stephe-leake.org>]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/12/2015 12:14 AM, Stephen Leake wrote:
>
>> Ok, I can see the need for some sort of metadata. But that suggests:
>>
>> (cl-defgeneric project-metadata (project dir)
>> "Return interesting metadata for DIR in PROJECT."
>> ...)
>
> Yes, more or less. Instead of directories, we could have complex
> objects like John suggests, with their own properties, but the above
> seems be simpler.
>
>> We need more examples of useful metadata before we can begin to design a
>> format for the return result.
>
> Probably. I just have in mind one tag so far, e.g. let's call it
> "category". There might be none (in which case the directory is a part
> of the project), 

If I pass in a DIR that is _not_ related to the project, the category
should be something like "unknown". So we have at least four values for
category:

unknown
top-level (ie "the project")
library
other-dependency

But then there is also, orthogonal to category:

editable (or read-write)
read-only

>>> Then project-roots will contain dscovr, sal and common. And
>>> library-roots will have what's left in the search path (Ada runtime
>>> library).
>>
>> You did not answer the most important part of the question: "why?".
>
> Because, from dscovr depends on sal and common, and according to your
> description, the person working on dscovr can make changes to sal and
> common as well, if needed. 

So you are focusing on the "read-only" vs "read/write" aspect.

> So a search across "current project, as
> well as code I edit together with it" 

This is project-roots.

> would include dscovr, sal and common.
>
> Which sounds useful to me, but I don't really know your workflows. If
> the dscovr developer doesn't need to edit sal and common 98% of their
> time, maybe they should indeed go into project-library-roots.

Yes, the developer will want to search different subsets of the full
project at different times.

Which is why this needs to be more flexible.

The user should be able to specify a subset of the search path for each
search, at run-time. Having only project-roots and project-library-roots
available is too limiting.

> If they only need to edit sal or common for like 5-20% of their work
> on dscovr, maybe a finer distinction is needed, like three kinds of
> searches:
>
> - Across project-roots and project-library-roots.
>
> - project-roots only.
>
> - The one project roots the current buffer is in.
>
> Which complicates things a bit, but it's still simpler than having
> user predicates and arbitrary metadata.

The UI for a predicate is more complicated, yes. But the project.el API
is simpler; a single predicate arg is clearly simpler than the confusion
between project-roots and project-library-roots.

Which is why I keep emphasizing that project.el is _not_ about the UI.

>>> Personally, I expect that to be useful (e.g. limiting searches to the
>>> projects that you control). But if not, you an also disregard that
>>> distinction. Do you see any downsides to it?
>>
>> Yes; it's not flexible enough. There were times when I wanted to search
>> only in map/, or only in sal/. I just used grep-find.
>
> You can also use C-u M-x project-find-regexp, and point it to map/ or
> sal/. Specifying a single directory is relatively easy anyway.

Not with the current API:

(defun project-find-regexp (regexp)
...)

How would I specify a single root or directory?

>> The predicate can query the metadata, via project-metadata.
>
> I can display all available metadata to the user, and ask: directories
> with which pieces of metadata set do you want to search? The user
> enters the category names, separated by spaces, and the search
> proceeds.

That would be a dialog-box style UI, yes.

> I can't do the same with predicates. I can't even list them all,
> really.

The dialog box uses the user input to construct a predicate, which it
passes to the project.el UI. 

That's an example of what I had in mind; higher level code implementing
a nice UI to drive project.el functions.

The alternative to an arbitrary predicate function is to hard-code some
set of possible choices in the search code. Any such set is limiting.

It may be that we discover some commonly used predicates; they could be
standardized in project.el.

On the GDS project, one search we repeated often was for FIXME:
comments; we had some structure in them, identifying which release it
had to be fixed for, who was assigned to fix it, etc. So I could have
written a few predicates that take advantage of the structure, and used
a menu to select among them for a search.

>> We have to agree that the UI is a separate discussion, or we will get nowhere.
>
> The API should take possible UIs into account. Like described above,

Yes. Metadata query functions and search predicates work for a very
large variety of UIs (menus, dialog boxes, Siri), as well as a very
large variety of searches. "project-roots vs project-library-roots"
works for a similar set of UIs but a much smaller set of searches.

> if we standardize on metadata, I can imagine at least one UI. If we
> standardize on arbitrary predicates, it's much harder.

Metadata is provided by the backend to the client code. Predicates
are provided by the client code to the backend. They are complementary,
not mutually exclusive.

> This kind of change to the API should even come with a corresponding
> change proposal for xref-find-references, to demonstrate that it's
> viable.

Yes, I'll put it on my list.

-- 
-- Stephe