all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Project out of sources compilation
       [not found] <4wwljrdnra3bsloehioa46y24ozxajajmvf2elvskxxq3mhtg2.ref@pyv2z5snot6h>
@ 2024-03-16 13:12 ` Ergus
  2024-03-16 16:50   ` Konstantin Kharlamov
                     ` (2 more replies)
  0 siblings, 3 replies; 53+ messages in thread
From: Ergus @ 2024-03-16 13:12 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

Hi:

These days I have been working with cmake projects here and there and in
spite of I could handle most of the work from emacs, it required a lot
of extra time to make it more or less comfortable.

I wrote a couple of simple functions to manage my needs, but at the end
I think that there are some small piece missing to handle common
workflows and glue everything (with the users in mind of course):

1. Out of sources compilation.

Most of projects now prefer to do out-of sources compilation. Either to
keep source code clean or to keep multiple compilations at the same time
(i.e Debug/Release/win32)

The project.el package has already some compilation commands, but they
assume that the compilation will be executed in the project's
root... which is not true most of the time.

Maybe we may add an extra custom variable that could be specified in the
dir-locals.el in order specify where the compilation command must be
executed.

Some more heuristics here is possible, but I would settle for at least
something simpler.

2. Eglot compilations database place.

When compilation is out of sources the cmake generated
compile_commands.json also goes in that directory by default.

This issue can be managed with a line in dir-locals, or just manually
coping the database.

((eglot-workspace-configuration
	  . (:clangd (:initializationOptions (:compilationDatabasePath
	  "build")))))

Probably some simple slight integration of Eglot with Project may help.

   2.1 This mixes with the previous one because if we change the
   compilation directory the line with initializationOptions is not
   updated and requires manual intervention

3. Projects multiple backends

This one is tricky because at the moment I have gtags-mode, that
includes a backend for project.el, there is the default backend, but
also we could add something called cmake-backend (i.e looks for
CMakeLists.txt that includes a 'project' entry)

In that case emacs cannot use all the information form the three even if
it is not contradictory, so the user ends up opening the terminal and
doing things manually.

4. Flymake integration

Even without eglot, flymake should be capable to work very easily with
cmake projects.

This step is also a stage before doing a proper plugin integration of
tools like cppcheck for flymake.

5. Project local variables (a man can dream, a man can dream)

There are some situations where we want to have variables shared among a
project. (i.e some output directory, logging option when executing,
flags, environment variables).

At the moment these options work partially by using directory
variables. If we have the concept of a "project", maybe it is logical to
have some sort of project scope concept, specially for projects sharing
a common root.

For example vs-code adds a subdirectory with project variables that the
user (but also any plugin) can refer to in the project's scope.

--

I could try to implement some of this with your help; but I need some
feedback on which of them are desired and which are not. Or which ones
are maybe better to put as feature requests for a more skilled lisper or
package maintainer.

WDYT?
Ergus



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-16 13:12 ` Project out of sources compilation Ergus
@ 2024-03-16 16:50   ` Konstantin Kharlamov
  2024-03-16 19:00     ` Ergus
  2024-03-17  2:53   ` Dmitry Gutov
  2024-03-17 11:36   ` Augusto Stoffel
  2 siblings, 1 reply; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-03-16 16:50 UTC (permalink / raw)
  To: Ergus, Dmitry Gutov; +Cc: emacs-devel

On Sat, 2024-03-16 at 14:12 +0100, Ergus wrote:
> 2. Eglot compilations database place.
> 
> When compilation is out of sources the cmake generated
> compile_commands.json also goes in that directory by default.
> 
> This issue can be managed with a line in dir-locals, or just manually
> coping the database.
> 
> ((eglot-workspace-configuration
> 	  . (:clangd (:initializationOptions
> (:compilationDatabasePath
> 	  "build")))))

Just wanted to clarify two things:

1. You can't just copy clangd database (i.e. `compile_commands.json`)
because the field "directory" has absolute path and clangd doesn't
support relative one. So you'll also have to regexp-replace the paths
after it's been copied.
2. If your out-of-source dir is called `build/`, then you don't need to
configure clangd to support it, because it handles that special case.

> Probably some simple slight integration of Eglot with Project may
> help.
> 
>    2.1 This mixes with the previous one because if we change the
>    compilation directory the line with initializationOptions is not
>    updated and requires manual intervention

Some generic support for pointing out the compilation dir sounds indeed
useful if it's possible.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-16 16:50   ` Konstantin Kharlamov
@ 2024-03-16 19:00     ` Ergus
  2024-03-16 20:56       ` Konstantin Kharlamov
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-03-16 19:00 UTC (permalink / raw)
  To: Konstantin Kharlamov; +Cc: Dmitry Gutov, emacs-devel

Hi:

On Sat, Mar 16, 2024 at 07:50:49PM +0300, Konstantin Kharlamov wrote:
>On Sat, 2024-03-16 at 14:12 +0100, Ergus wrote:
>> 2. Eglot compilations database place.
>>
>> When compilation is out of sources the cmake generated
>> compile_commands.json also goes in that directory by default.
>>
>> This issue can be managed with a line in dir-locals, or just manually
>> coping the database.
>>
>> ((eglot-workspace-configuration
>> 	� . (:clangd (:initializationOptions
>> (:compilationDatabasePath
>> 	� "build")))))
>
>Just wanted to clarify two things:
>
>1. You can't just copy clangd database (i.e. `compile_commands.json`)
>because the field "directory" has absolute path and clangd doesn't
>support relative one. So you'll also have to regexp-replace the paths
>after it's been copied.

Yes, I know that (now). But that is actually the most popular solution in:

https://emacs.stackexchange.com/questions/73922/how-do-you-tell-eglot-and-clangd-about-the-compilation-database-compile-command

>2. If your out-of-source dir is called `build/`, then you don't need to
>configure clangd to support it, because it handles that special case.
>
Good to know, I didn't know because I always use names like build_debug,
build_x64, build_clang and so on.

>> Probably some simple slight integration of Eglot with Project may
>> help.
>>
>> �� 2.1 This mixes with the previous one because if we change the
>> �� compilation directory the line with initializationOptions is not
>> �� updated and requires manual intervention
>
>Some generic support for pointing out the compilation dir sounds indeed
>useful if it's possible.
>
Lets wait for Dmitry's suggestion.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-16 19:00     ` Ergus
@ 2024-03-16 20:56       ` Konstantin Kharlamov
  0 siblings, 0 replies; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-03-16 20:56 UTC (permalink / raw)
  To: Ergus; +Cc: Dmitry Gutov, emacs-devel

On Sat, 2024-03-16 at 20:00 +0100, Ergus wrote:
> Hi:
> 
> On Sat, Mar 16, 2024 at 07:50:49PM +0300, Konstantin Kharlamov wrote:
> > On Sat, 2024-03-16 at 14:12 +0100, Ergus wrote:
> > > 2. Eglot compilations database place.
> > > 
> > > When compilation is out of sources the cmake generated
> > > compile_commands.json also goes in that directory by default.
> > > 
> > > This issue can be managed with a line in dir-locals, or just
> > > manually
> > > coping the database.
> > > 
> > > ((eglot-workspace-configuration
> > > 	� . (:clangd (:initializationOptions
> > > (:compilationDatabasePath
> > > 	� "build")))))
> > 
> > Just wanted to clarify two things:
> > 
> > 1. You can't just copy clangd database (i.e.
> > `compile_commands.json`)
> > because the field "directory" has absolute path and clangd doesn't
> > support relative one. So you'll also have to regexp-replace the
> > paths
> > after it's been copied.
> 
> Yes, I know that (now). But that is actually the most popular
> solution in:
> 
> https://emacs.stackexchange.com/questions/73922/how-do-you-tell-eglot-and-clangd-about-the-compilation-database-compile-command

Actually, I stand corrected. I certainly remember that way not working
≈2 years ago (last time I tried and learned all that information);
however right now creating a symlink to `compile_commands.json` as well
as just copying does work.

I just tested that on a C project at work by symlinking/copying the
json file, then starting lsp and trying to "goto definition" to a
generated `.h` file that resides in the build dir. It works. I tested
that both with dir called `build` and `build-docs` (in the latter case
`build` dir was absent to make sure).

clangd version 16.0.6



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-16 13:12 ` Project out of sources compilation Ergus
  2024-03-16 16:50   ` Konstantin Kharlamov
@ 2024-03-17  2:53   ` Dmitry Gutov
  2024-03-17  7:22     ` Ergus
  2024-03-17 11:36   ` Augusto Stoffel
  2 siblings, 1 reply; 53+ messages in thread
From: Dmitry Gutov @ 2024-03-17  2:53 UTC (permalink / raw)
  To: Ergus; +Cc: emacs-devel

Hi!

On 16/03/2024 15:12, Ergus wrote:

> These days I have been working with cmake projects here and there and in
> spite of I could handle most of the work from emacs, it required a lot
> of extra time to make it more or less comfortable.
> 
> I wrote a couple of simple functions to manage my needs, but at the end
> I think that there are some small piece missing to handle common
> workflows and glue everything (with the users in mind of course):
> 
> 1. Out of sources compilation.
> 
> Most of projects now prefer to do out-of sources compilation. Either to
> keep source code clean or to keep multiple compilations at the same time
> (i.e Debug/Release/win32)
> 
> The project.el package has already some compilation commands, but they
> assume that the compilation will be executed in the project's
> root... which is not true most of the time.
> 
> Maybe we may add an extra custom variable that could be specified in the
> dir-locals.el in order specify where the compilation command must be
> executed.
> 
> Some more heuristics here is possible, but I would settle for at least
> something simpler.

project.el has just one, very simple command, where the only thing that 
it does it switch to the root first. How will you customize it? With a 
hook, where the user would write a function "determine a directory for 
compilation"? They might as well define a new command - or redefine this 
one. Or just an option with relative directory name?

> 2. Eglot compilations database place.
> 
> When compilation is out of sources the cmake generated
> compile_commands.json also goes in that directory by default.
> 
> This issue can be managed with a line in dir-locals, or just manually
> coping the database.
> 
> ((eglot-workspace-configuration
>        . (:clangd (:initializationOptions (:compilationDatabasePath
>        "build")))))
> 
> Probably some simple slight integration of Eglot with Project may help.

That seems like it can be a Eglot user option. Again, just storing a 
file name relative to the project root?

>    2.1 This mixes with the previous one because if we change the
>    compilation directory the line with initializationOptions is not
>    updated and requires manual intervention
> 
> 3. Projects multiple backends
> 
> This one is tricky because at the moment I have gtags-mode, that
> includes a backend for project.el, there is the default backend, but
> also we could add something called cmake-backend (i.e looks for
> CMakeLists.txt that includes a 'project' entry)
> 
> In that case emacs cannot use all the information form the three even if
> it is not contradictory, so the user ends up opening the terminal and
> doing things manually.

Indeed, project.el doesn't "merge" backends. Is there an obvious 
algorithm for merging backends which would be understandable to an 
average user?

If the only thing you wanted combined is the root detection, I suggest 
reusing the variable project-vc-extra-root-markers, automatically or 
manually (just have the user set it). If its data structure is now 
powerful enough, it can be updated. For example, if the logic both looks 
for CMakeLists.txt and checks its contents, 
project-vc-extra-root-markers could support cons entries like (MARKER . 
CHECK-FN).

> 4. Flymake integration
> 
> Even without eglot, flymake should be capable to work very easily with
> cmake projects.
> 
> This step is also a stage before doing a proper plugin integration of
> tools like cppcheck for flymake.

Is there something particular that is required?

> 5. Project local variables (a man can dream, a man can dream)
> 
> There are some situations where we want to have variables shared among a
> project. (i.e some output directory, logging option when executing,
> flags, environment variables).
> 
> At the moment these options work partially by using directory
> variables. If we have the concept of a "project", maybe it is logical to
> have some sort of project scope concept, specially for projects sharing
> a common root.
> 
> For example vs-code adds a subdirectory with project variables that the
> user (but also any plugin) can refer to in the project's scope.

Does such subdirectory in VS Code affect the project residing its its 
parent directory, or some other projects as well? Your description above 
sounds more like our .dir-locals.el.

And the directory locals facility in Emacs also has the less well-known 
capability called "directory class", see the bottom of 
https://www.gnu.org/software/emacs/manual/html_node/emacs/Directory-Variables.html. 
I think it should work okay for sharing variables between projects.

Perhaps it's less user-friendly than one had hoped, I don't know. Some 
improvements for related UIs would be welcome.

Just creating a parallel built-in way to set variables inside 
directories but for projects sounds a bit much. Also, we don't always 
detect a buffer's project before the user asks for it. So I'm not sure 
what hooks could be added, even if we wanted.

> I could try to implement some of this with your help; but I need some
> feedback on which of them are desired and which are not. Or which ones
> are maybe better to put as feature requests for a more skilled lisper or
> package maintainer.

Patches are welcome, but see above.

Cheers,
Dmitry.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-17  2:53   ` Dmitry Gutov
@ 2024-03-17  7:22     ` Ergus
  2024-03-17  8:45       ` Eli Zaretskii
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-03-17  7:22 UTC (permalink / raw)
  To: emacs-devel, Dmitry Gutov

Hi:

On March 17, 2024 3:53:42 AM GMT+01:00, Dmitry Gutov <dmitry@gutov.dev> wrote:
>Hi!
>
>On 16/03/2024 15:12, Ergus wrote:
>
>> These days I have been working with cmake projects here and there and in
>> spite of I could handle most of the work from emacs, it required a lot
>> of extra time to make it more or less comfortable.
>> 
>> I wrote a couple of simple functions to manage my needs, but at the end
>> I think that there are some small piece missing to handle common
>> workflows and glue everything (with the users in mind of course):
>> 
>> 1. Out of sources compilation.
>> 
>> Most of projects now prefer to do out-of sources compilation. Either to
>> keep source code clean or to keep multiple compilations at the same time
>> (i.e Debug/Release/win32)
>> 
>> The project.el package has already some compilation commands, but they
>> assume that the compilation will be executed in the project's
>> root... which is not true most of the time.
>> 
>> Maybe we may add an extra custom variable that could be specified in the
>> dir-locals.el in order specify where the compilation command must be
>> executed.
>> 
>> Some more heuristics here is possible, but I would settle for at least
>> something simpler.
>
>project.el has just one, very simple command, where the only thing that it does it switch to the root first. How will you customize it? With a hook, where the user would write a function "determine a directory for compilation"? They might as well define a new command - or redefine this one. Or just an option with relative directory name?
>

IMO the only thing we need is probably a variable/custom like project-build-dir. The user can define it in the dir-locals and the project command will use it if defined/ else use project-root. Maybe the backend could initialize it.

Alternatively (and not totally exclusive) project.el could define a project-build-dir function that project backends could optionally redefine (i.e I use a plist as project id in gags-mode and getting any stored property from there is very easy with a command). By default it will be an alias for project-root in the VC backend.

There is also some need for a 'bin' dir, that is, where the final executable will reside, useful to execute and debug with tools like gud and independent from 'build'...  For example in a python project this may be the project root OR where the file with __main__ resides, but a python project usually won't specify a build dir. But let's go for one thing at a time.


>> 2. Eglot compilations database place.
>> 
>> When compilation is out of sources the cmake generated
>> compile_commands.json also goes in that directory by default.
>> 
>> This issue can be managed with a line in dir-locals, or just manually
>> coping the database.
>> 
>> ((eglot-workspace-configuration
>>        . (:clangd (:initializationOptions (:compilationDatabasePath
>>        "build")))))
>> 
>> Probably some simple slight integration of Eglot with Project may help.
>
>That seems like it can be a Eglot user option. Again, just storing a file name relative to the project root?
>
Actually it doesn't need to be relative. For example QtCreator creates the build dir outside the sources directory in the same level. So, the only requirement is to be a valid path. Relative to root or absolute one

>>    2.1 This mixes with the previous one because if we change the
>>    compilation directory the line with initializationOptions is not
>>    updated and requires manual intervention
>> 
>> 3. Projects multiple backends
>> 
>> This one is tricky because at the moment I have gtags-mode, that
>> includes a backend for project.el, there is the default backend, but
>> also we could add something called cmake-backend (i.e looks for
>> CMakeLists.txt that includes a 'project' entry)
>> 
>> In that case emacs cannot use all the information form the three even if
>> it is not contradictory, so the user ends up opening the terminal and
>> doing things manually.
>
>Indeed, project.el doesn't "merge" backends. Is there an obvious algorithm for merging backends which would be understandable to an average user?
>
>If the only thing you wanted combined is the root detection, I suggest reusing the variable project-vc-extra-root-markers, automatically or manually (just have the user set it). If its data structure is now powerful enough, it can be updated. For example, if the logic both looks for CMakeLists.txt and checks its contents, project-vc-extra-root-markers could support cons entries like (MARKER . CHECK-FN).

Actually the idea is as simple as: if any of the backends explicitly specifies some property that the others don't, then use that one.

Applied in this case. VC backend does not have any "build" Information in general. But a cmake backend could detect a "build" directory and propose it (looking for a CMake cache inside it for example). An hipothetical  autotools backend may do the same.

>
>> 4. Flymake integration
>> 
>> Even without eglot, flymake should be capable to work very easily with
>> cmake projects.
>> 
>> This step is also a stage before doing a proper plugin integration of
>> tools like cppcheck for flymake.
>
>Is there something particular that is required?
>
>> 5. Project local variables (a man can dream, a man can dream)
>> 
>> There are some situations where we want to have variables shared among a
>> project. (i.e some output directory, logging option when executing,
>> flags, environment variables).
>> 
>> At the moment these options work partially by using directory
>> variables. If we have the concept of a "project", maybe it is logical to
>> have some sort of project scope concept, specially for projects sharing
>> a common root.
>> 
>> For example vs-code adds a subdirectory with project variables that the
>> user (but also any plugin) can refer to in the project's scope.
>
>Does such subdirectory in VS Code affect the project residing its its parent directory, or some other projects as well? Your description above sounds more like our .dir-locals.el.
>
>And the directory locals facility in Emacs also has the less well-known capability called "directory class", see the bottom of https://www.gnu.org/software/emacs/manual/html_node/emacs/Directory-Variables.html. I think it should work okay for sharing variables between projects.
>
>Perhaps it's less user-friendly than one had hoped, I don't know. Some improvements for related UIs would be welcome.
>
>Just creating a parallel built-in way to set variables inside directories but for projects sounds a bit much. Also, we don't always detect a buffer's project before the user asks for it. So I'm not sure what hooks could be added, even if we wanted.
>
Ok, I buy this one because there are the dir-locals, which are not ideal for project purposes, but it is already there and it isn't worth adding another abstraction layer

>> I could try to implement some of this with your help; but I need some
>> feedback on which of them are desired and which are not. Or which ones
>> are maybe better to put as feature requests for a more skilled lisper or
>> package maintainer.
>
>Patches are welcome, but see above.
>
>Cheers,
>Dmitry.
>

Best,
Ergus

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-17  7:22     ` Ergus
@ 2024-03-17  8:45       ` Eli Zaretskii
  2024-03-17 17:33         ` Ergus
  0 siblings, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-03-17  8:45 UTC (permalink / raw)
  To: Ergus, dmitry; +Cc: emacs-devel

> Date: Sun, 17 Mar 2024 08:22:56 +0100
> From: Ergus <spacibba@aol.com>
> 
> >> 1. Out of sources compilation.
> >> 
> >> Most of projects now prefer to do out-of sources compilation. Either to
> >> keep source code clean or to keep multiple compilations at the same time
> >> (i.e Debug/Release/win32)
> >> 
> >> The project.el package has already some compilation commands, but they
> >> assume that the compilation will be executed in the project's
> >> root... which is not true most of the time.
> >> 
> >> Maybe we may add an extra custom variable that could be specified in the
> >> dir-locals.el in order specify where the compilation command must be
> >> executed.
> >> 
> >> Some more heuristics here is possible, but I would settle for at least
> >> something simpler.
> >
> >project.el has just one, very simple command, where the only thing that it does it switch to the root first. How will you customize it? With a hook, where the user would write a function "determine a directory for compilation"? They might as well define a new command - or redefine this one. Or just an option with relative directory name?
> >
> 
> IMO the only thing we need is probably a variable/custom like project-build-dir. The user can define it in the dir-locals and the project command will use it if defined/ else use project-root. Maybe the backend could initialize it.
> 
> Alternatively (and not totally exclusive) project.el could define a project-build-dir function that project backends could optionally redefine (i.e I use a plist as project id in gags-mode and getting any stored property from there is very easy with a command). By default it will be an alias for project-root in the VC backend.

Maybe I'm missing something, but isn't the build tree just one more
tree that is part of the project?  If so, can't you use
project-external-roots to add this tree to the project?  I thought
this was the mechanism to add trees to a project as included in the
original design of project.el and its support in Emacs?

> There is also some need for a 'bin' dir, that is, where the final executable will reside, useful to execute and debug with tools like gud and independent from 'build'...  For example in a python project this may be the project root OR where the file with __main__ resides, but a python project usually won't specify a build dir. But let's go for one thing at a time.

Likewise here.

Or what am I missing?



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-16 13:12 ` Project out of sources compilation Ergus
  2024-03-16 16:50   ` Konstantin Kharlamov
  2024-03-17  2:53   ` Dmitry Gutov
@ 2024-03-17 11:36   ` Augusto Stoffel
  2024-03-17 17:47     ` Ergus
  2 siblings, 1 reply; 53+ messages in thread
From: Augusto Stoffel @ 2024-03-17 11:36 UTC (permalink / raw)
  To: Ergus; +Cc: Dmitry Gutov, emacs-devel

On Sat, 16 Mar 2024 at 14:12, Ergus wrote:

> 5. Project local variables (a man can dream, a man can dream)
>
> There are some situations where we want to have variables shared among a
> project. (i.e some output directory, logging option when executing,
> flags, environment variables).
>
> At the moment these options work partially by using directory
> variables. If we have the concept of a "project", maybe it is logical to
> have some sort of project scope concept, specially for projects sharing
> a common root.

For Emacs variables, why you think dir-local variables only works
“partially”?

As to environment variables, they are generally assumed to be global in
Emacs and it goes a bit against the grain to make them buffer-local /
project dependent.  Nonetheless, there are packages for that (one is
buffer-env, which I wrote) and they work fine (at least for me).

> For example vs-code adds a subdirectory with project variables that the
> user (but also any plugin) can refer to in the project's scope.

Is this meant to store editor configuration or environment variables?  I
guess turning VSCode configuration variables into Emacs alternatives
would be tricky.  But if this is about env vars and the directory
organization is sensible, then one could configure buffer-env to
interoperate.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-17  8:45       ` Eli Zaretskii
@ 2024-03-17 17:33         ` Ergus
  2024-03-17 17:38           ` Eli Zaretskii
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-03-17 17:33 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: dmitry, emacs-devel

On Sun, Mar 17, 2024 at 10:45:29AM +0200, Eli Zaretskii wrote:
>> Date: Sun, 17 Mar 2024 08:22:56 +0100
>> From: Ergus <spacibba@aol.com>
>>
>> >
>> >project.el has just one, very simple command, where the only thing that it does it switch to the root first. How will you customize it? With a hook, where the user would write a function "determine a directory for compilation"? They might as well define a new command - or redefine this one. Or just an option with relative directory name?
>> >
>>
>> IMO the only thing we need is probably a variable/custom like project-build-dir. The user can define it in the dir-locals and the project command will use it if defined/ else use project-root. Maybe the backend could initialize it.
>>
>> Alternatively (and not totally exclusive) project.el could define a project-build-dir function that project backends could optionally redefine (i.e I use a plist as project id in gags-mode and getting any stored property from there is very easy with a command). By default it will be an alias for project-root in the VC backend.
>
>Maybe I'm missing something, but isn't the build tree just one more
>tree that is part of the project?  If so, can't you use
>project-external-roots to add this tree to the project?  I thought
>this was the mechanism to add trees to a project as included in the
>original design of project.el and its support in Emacs?
>

>> There is also some need for a 'bin' dir, that is, where the final executable will reside, useful to execute and debug with tools like gud and independent from 'build'...  For example in a python project this may be the project root OR where the file with __main__ resides, but a python project usually won't specify a build dir. But let's go for one thing at a time.
>
>Likewise here.
>
>Or what am I missing?

Hi Eli:

More or less we have it, that's why I said that there was just some
(small) missing pieces.

We have the `project-external-roots`, but project.el uses them only to
find files and regexps. So it looks like they are intended to be source
places somehow.

The compile or debug programs doesn't know that they are intended to
execute there.

However, a `build` or `bin` are different because they are where the
`compile` or `gud` are intended to run; generally not a place to search
for sources (unless the source is generated like config.h).

Two examples:

```
git clone emacs
cd emacs
mkdir build # or mybuild, build_debug.. any name
./autogen.sh # (this is intendedto happen in the root)

cd build
../configure options  // This creates a Makefile
make

make install
```

On cmake

```
git clone project
cd project

   // Creates a CMakeCache.txt
   Alternative 1
     cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=true -DCMAKE_BUILD_TYPE=Debug -B build/
     cd build

   Alternative 2
     mkdir build
     cd build
     cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=true -DCMAKE_BUILD_TYPE=Debug ..

make
make install
```

The pattern is pretty much the same, so it is pretty simple to allow
projects.el to simplify the generation-compilation-execution-debug
workflow. And with autotools+cmake we are probably covering a huge range
of projects around.

Best,
Ergus



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-17 17:33         ` Ergus
@ 2024-03-17 17:38           ` Eli Zaretskii
  2024-03-17 17:58             ` Ergus
  0 siblings, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-03-17 17:38 UTC (permalink / raw)
  To: Ergus; +Cc: dmitry, emacs-devel

> Date: Sun, 17 Mar 2024 18:33:21 +0100
> From: Ergus <spacibba@aol.com>
> Cc: dmitry@gutov.dev, emacs-devel@gnu.org
> 
> >Maybe I'm missing something, but isn't the build tree just one more
> >tree that is part of the project?  If so, can't you use
> >project-external-roots to add this tree to the project?  I thought
> >this was the mechanism to add trees to a project as included in the
> >original design of project.el and its support in Emacs?
> >
> 
> >> There is also some need for a 'bin' dir, that is, where the final executable will reside, useful to execute and debug with tools like gud and independent from 'build'...  For example in a python project this may be the project root OR where the file with __main__ resides, but a python project usually won't specify a build dir. But let's go for one thing at a time.
> >
> >Likewise here.
> >
> >Or what am I missing?
> 
> Hi Eli:
> 
> More or less we have it, that's why I said that there was just some
> (small) missing pieces.
> 
> We have the `project-external-roots`, but project.el uses them only to
> find files and regexps. So it looks like they are intended to be source
> places somehow.
> 
> The compile or debug programs doesn't know that they are intended to
> execute there.
> 
> However, a `build` or `bin` are different because they are where the
> `compile` or `gud` are intended to run; generally not a place to search
> for sources (unless the source is generated like config.h).

Are you saying that "M-x compile" and "M-x gdb" are not (yet)
supported by project.el?



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-17 11:36   ` Augusto Stoffel
@ 2024-03-17 17:47     ` Ergus
  2024-03-19 18:36       ` Ergus
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-03-17 17:47 UTC (permalink / raw)
  To: Augusto Stoffel; +Cc: Dmitry Gutov, emacs-devel

On Sun, Mar 17, 2024 at 12:36:44PM +0100, Augusto Stoffel wrote:
>On Sat, 16 Mar 2024 at 14:12, Ergus wrote:
>
>> 5. Project local variables (a man can dream, a man can dream)
>>
>> There are some situations where we want to have variables shared among a
>> project. (i.e some output directory, logging option when executing,
>> flags, environment variables).
>>
>> At the moment these options work partially by using directory
>> variables. If we have the concept of a "project", maybe it is logical to
>> have some sort of project scope concept, specially for projects sharing
>> a common root.
>
>For Emacs variables, why you think dir-local variables only works
>“partially”?
>
>As to environment variables, they are generally assumed to be global in
>Emacs and it goes a bit against the grain to make them buffer-local /
>project dependent.  Nonetheless, there are packages for that (one is
>buffer-env, which I wrote) and they work fine (at least for me).
>
They work for one directory, but when the project includes multiple
roots (project-external-roots) the variables are somehow missing just
when jumping with xref to a definition.

However, we can ignore this as I mentioned before.

>> For example vs-code adds a subdirectory with project variables that the
>> user (but also any plugin) can refer to in the project's scope.
>
>Is this meant to store editor configuration or environment variables?  I
>guess turning VSCode configuration variables into Emacs alternatives
>would be tricky. 

No please. No interaction-importing from other editors will be
maintainable in the long path.

> But if this is about env vars and the directory
>organization is sensible, then one could configure buffer-env to
>interoperate.

This is it. The idea is to extend project.el in order to bring more
project-oriented features.

Lets say:

The user (or project.el backend) could specify the `build` directory and
the environment that needs to be set to execute the program from the
build or bin directory.

So with a simple modification of dir-locals.el the user may be capable
to run M-x compile (or project-compile), M-x gud-gdb... or being more
ambitious M-x project-generate (to call autogen.sh and ./configure or
cmake depending of the project-backend)

For sure we wont cover the 100% of the projects, but wuth automake and
autotools we will be very close (and in the future someone could add
other backends for it)







^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-17 17:38           ` Eli Zaretskii
@ 2024-03-17 17:58             ` Ergus
  0 siblings, 0 replies; 53+ messages in thread
From: Ergus @ 2024-03-17 17:58 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: dmitry, emacs-devel

On Sun, Mar 17, 2024 at 07:38:16PM +0200, Eli Zaretskii wrote:
>> Date: Sun, 17 Mar 2024 18:33:21 +0100
>> From: Ergus <spacibba@aol.com>
>> Cc: dmitry@gutov.dev, emacs-devel@gnu.org
>>
>> >Maybe I'm missing something, but isn't the build tree just one more
>> >tree that is part of the project?  If so, can't you use
>> >project-external-roots to add this tree to the project?  I thought
>> >this was the mechanism to add trees to a project as included in the
>> >original design of project.el and its support in Emacs?
>> >
>>
>> >> There is also some need for a 'bin' dir, that is, where the final executable will reside, useful to execute and debug with tools like gud and independent from 'build'...  For example in a python project this may be the project root OR where the file with __main__ resides, but a python project usually won't specify a build dir. But let's go for one thing at a time.
>> >
>> >Likewise here.
>> >
>> >Or what am I missing?
>>
>> Hi Eli:
>>
>> More or less we have it, that's why I said that there was just some
>> (small) missing pieces.
>>
>> We have the `project-external-roots`, but project.el uses them only to
>> find files and regexps. So it looks like they are intended to be source
>> places somehow.
>>
>> The compile or debug programs doesn't know that they are intended to
>> execute there.
>>
>> However, a `build` or `bin` are different because they are where the
>> `compile` or `gud` are intended to run; generally not a place to search
>> for sources (unless the source is generated like config.h).
>
>Are you saying that "M-x compile" and "M-x gdb" are not (yet)
>supported by project.el?
>

Kind of. There is project-compile, but executes in the root
directory. And M-x compile does not have any custom to control the
execution place, so it needs to do a let around it.

something similar happen to gdb, or project-eshell.

So, what is not actually supported is out of sources
compilation/execution. In the first email I explained that part. Even
Eglot requires extra work to function properly.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: Project out of sources compilation
  2024-03-17 17:47     ` Ergus
@ 2024-03-19 18:36       ` Ergus
  2024-03-27 16:38         ` [PATCH] " Ergus
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-03-19 18:36 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 2725 bytes --]

Hi Dmitry:

Here I attach a very simple approach to add support for out of sources
compilation in project.el

This is just a POC, but to have some starting point to discuss.

Best,
Ergus



On Sun, Mar 17, 2024 at 06:47:43PM +0100, Ergus wrote:
>On Sun, Mar 17, 2024 at 12:36:44PM +0100, Augusto Stoffel wrote:
>>On Sat, 16 Mar 2024 at 14:12, Ergus wrote:
>>
>>>5. Project local variables (a man can dream, a man can dream)
>>>
>>>There are some situations where we want to have variables shared among a
>>>project. (i.e some output directory, logging option when executing,
>>>flags, environment variables).
>>>
>>>At the moment these options work partially by using directory
>>>variables. If we have the concept of a "project", maybe it is logical to
>>>have some sort of project scope concept, specially for projects sharing
>>>a common root.
>>
>>For Emacs variables, why you think dir-local variables only works
>>“partially”?
>>
>>As to environment variables, they are generally assumed to be global in
>>Emacs and it goes a bit against the grain to make them buffer-local /
>>project dependent.  Nonetheless, there are packages for that (one is
>>buffer-env, which I wrote) and they work fine (at least for me).
>>
>They work for one directory, but when the project includes multiple
>roots (project-external-roots) the variables are somehow missing just
>when jumping with xref to a definition.
>
>However, we can ignore this as I mentioned before.
>
>>>For example vs-code adds a subdirectory with project variables that the
>>>user (but also any plugin) can refer to in the project's scope.
>>
>>Is this meant to store editor configuration or environment variables?  I
>>guess turning VSCode configuration variables into Emacs alternatives
>>would be tricky.
>
>No please. No interaction-importing from other editors will be
>maintainable in the long path.
>
>>But if this is about env vars and the directory
>>organization is sensible, then one could configure buffer-env to
>>interoperate.
>
>This is it. The idea is to extend project.el in order to bring more
>project-oriented features.
>
>Lets say:
>
>The user (or project.el backend) could specify the `build` directory and
>the environment that needs to be set to execute the program from the
>build or bin directory.
>
>So with a simple modification of dir-locals.el the user may be capable
>to run M-x compile (or project-compile), M-x gud-gdb... or being more
>ambitious M-x project-generate (to call autogen.sh and ./configure or
>cmake depending of the project-backend)
>
>For sure we wont cover the 100% of the projects, but wuth automake and
>autotools we will be very close (and in the future someone could add
>other backends for it)
>
>
>
>
>

[-- Attachment #2: project-build-dir.patch --]
[-- Type: text/plain, Size: 2875 bytes --]

commit cfa10dfeac01ae25a7acd6857fae0b377b625779
Author: Jimmy Aguilar Mena <kratsbinovish@gmail.com>
Date:   Tue Mar 19 17:57:25 2024 +0100

    Add basic project-build-dir implementation
    
    This is just a dummy prototype to discuss about alternatives.

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index ac18aceadcf..0ca4ca85566 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -51,6 +51,9 @@
 ;; files inside the root must not be considered a part of it).  It
 ;; should be consistent with `project-files'.
 ;;
+;; `project-build-dir' can be overridden if the project backend has some
+;; extra information about the project build directory.
+;;
 ;; This list can change in future versions.
 ;;
 ;; Transient project:
@@ -208,6 +211,12 @@ project-prompter
   :group 'project
   :version "30.1")
 
+(defcustom project-build-dir nil
+  "Build directory for current project."
+  :type 'directory
+  :safe t
+  :version "30.1")
+
 ;;;###autoload
 (defun project-current (&optional maybe-prompt directory)
   "Return the project instance in DIRECTORY, defaulting to `default-directory'.
@@ -289,8 +298,30 @@ project-external-roots
 headers search path, load path, class path, and so on."
   nil)
 
+(cl-defgeneric project-build-dir (_project)
+  "Return build directory of the current PROJECT.
+
+This function is intended to be defined by the backend when possible.
+Otherwise this returns nil and the `project-compile' command will be
+called in the project-root.
+If the user defines the directory-local variable `project-build-dir' it
+will have precedence over the result of this function."
+ nil)
+
+(defun project-get-build-dir (project)
+  "Return build directory of the current PROJECT.
+If the variable `project-build-dir' is defined, this function returns
+it, otherwise it returns the project root.  If the defined path is
+relative, this expands it relatively to the project's root"
+  (let ((dir (or project-build-dir
+                 (project-build-dir project)
+                 (project-root project)))) ;; I assume project-root is absolute
+    (if (file-name-absolute-p dir)
+        dir
+      (expand-file-name dir (project-root project)))))
+
 (cl-defgeneric project-name (project)
-  "A human-readable name for the project.
+  "A human-readable name for the PROJECT.
 Nominally unique, but not enforced."
   (file-name-nondirectory (directory-file-name (project-root project))))
 
@@ -1391,7 +1422,7 @@ project-compile
   "Run `compile' in the project root."
   (declare (interactive-only compile))
   (interactive)
-  (let ((default-directory (project-root (project-current t)))
+  (let ((default-directory (project-get-build-dir (project-current t)))
         (compilation-buffer-name-function
          (or project-compilation-buffer-name-function
              compilation-buffer-name-function)))

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH] Project out of sources compilation
  2024-03-19 18:36       ` Ergus
@ 2024-03-27 16:38         ` Ergus
  2024-03-31  2:41           ` Dmitry Gutov
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-03-27 16:38 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 6361 bytes --]

Hi:

Here I attach a very simple patch to allow project out of sources
compilation and project build detection.

So far I already implemented a very primitive backend [1] for project.el
using these features to detect and compile autotools and cmake projects;
and, at some point I will propose it for elpa.

But I need to work a bit more on it; and, of course, the functionalities
in the attached patch need to go to vanilla this way or some other
equivalent.

Best,
Ergus

[1] https://github.com/Ergus/project-multi-mode/blob/master/project-multi-mode.el



On Tue, Mar 19, 2024 at 07:36:35PM +0100, Ergus wrote:
>Hi Dmitry:
>
>Here I attach a very simple approach to add support for out of sources
>compilation in project.el
>
>This is just a POC, but to have some starting point to discuss.
>
>Best,
>Ergus
>
>
>
>On Sun, Mar 17, 2024 at 06:47:43PM +0100, Ergus wrote:
>>On Sun, Mar 17, 2024 at 12:36:44PM +0100, Augusto Stoffel wrote:
>>>On Sat, 16 Mar 2024 at 14:12, Ergus wrote:
>>>
>>>>5. Project local variables (a man can dream, a man can dream)
>>>>
>>>>There are some situations where we want to have variables shared among a
>>>>project. (i.e some output directory, logging option when executing,
>>>>flags, environment variables).
>>>>
>>>>At the moment these options work partially by using directory
>>>>variables. If we have the concept of a "project", maybe it is logical to
>>>>have some sort of project scope concept, specially for projects sharing
>>>>a common root.
>>>
>>>For Emacs variables, why you think dir-local variables only works
>>>“partially”?
>>>
>>>As to environment variables, they are generally assumed to be global in
>>>Emacs and it goes a bit against the grain to make them buffer-local /
>>>project dependent.  Nonetheless, there are packages for that (one is
>>>buffer-env, which I wrote) and they work fine (at least for me).
>>>
>>They work for one directory, but when the project includes multiple
>>roots (project-external-roots) the variables are somehow missing just
>>when jumping with xref to a definition.
>>
>>However, we can ignore this as I mentioned before.
>>
>>>>For example vs-code adds a subdirectory with project variables that the
>>>>user (but also any plugin) can refer to in the project's scope.
>>>
>>>Is this meant to store editor configuration or environment variables?  I
>>>guess turning VSCode configuration variables into Emacs alternatives
>>>would be tricky.
>>
>>No please. No interaction-importing from other editors will be
>>maintainable in the long path.
>>
>>>But if this is about env vars and the directory
>>>organization is sensible, then one could configure buffer-env to
>>>interoperate.
>>
>>This is it. The idea is to extend project.el in order to bring more
>>project-oriented features.
>>
>>Lets say:
>>
>>The user (or project.el backend) could specify the `build` directory and
>>the environment that needs to be set to execute the program from the
>>build or bin directory.
>>
>>So with a simple modification of dir-locals.el the user may be capable
>>to run M-x compile (or project-compile), M-x gud-gdb... or being more
>>ambitious M-x project-generate (to call autogen.sh and ./configure or
>>cmake depending of the project-backend)
>>
>>For sure we wont cover the 100% of the projects, but wuth automake and
>>autotools we will be very close (and in the future someone could add
>>other backends for it)
>>
>>
>>
>>
>>

>commit cfa10dfeac01ae25a7acd6857fae0b377b625779
>Author: Jimmy Aguilar Mena <kratsbinovish@gmail.com>
>Date:   Tue Mar 19 17:57:25 2024 +0100
>
>    Add basic project-build-dir implementation
>
>    This is just a dummy prototype to discuss about alternatives.
>
>diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
>index ac18aceadcf..0ca4ca85566 100644
>--- a/lisp/progmodes/project.el
>+++ b/lisp/progmodes/project.el
>@@ -51,6 +51,9 @@
> ;; files inside the root must not be considered a part of it).  It
> ;; should be consistent with `project-files'.
> ;;
>+;; `project-build-dir' can be overridden if the project backend has some
>+;; extra information about the project build directory.
>+;;
> ;; This list can change in future versions.
> ;;
> ;; Transient project:
>@@ -208,6 +211,12 @@ project-prompter
>   :group 'project
>   :version "30.1")
>
>+(defcustom project-build-dir nil
>+  "Build directory for current project."
>+  :type 'directory
>+  :safe t
>+  :version "30.1")
>+
> ;;;###autoload
> (defun project-current (&optional maybe-prompt directory)
>   "Return the project instance in DIRECTORY, defaulting to `default-directory'.
>@@ -289,8 +298,30 @@ project-external-roots
> headers search path, load path, class path, and so on."
>   nil)
>
>+(cl-defgeneric project-build-dir (_project)
>+  "Return build directory of the current PROJECT.
>+
>+This function is intended to be defined by the backend when possible.
>+Otherwise this returns nil and the `project-compile' command will be
>+called in the project-root.
>+If the user defines the directory-local variable `project-build-dir' it
>+will have precedence over the result of this function."
>+ nil)
>+
>+(defun project-get-build-dir (project)
>+  "Return build directory of the current PROJECT.
>+If the variable `project-build-dir' is defined, this function returns
>+it, otherwise it returns the project root.  If the defined path is
>+relative, this expands it relatively to the project's root"
>+  (let ((dir (or project-build-dir
>+                 (project-build-dir project)
>+                 (project-root project)))) ;; I assume project-root is absolute
>+    (if (file-name-absolute-p dir)
>+        dir
>+      (expand-file-name dir (project-root project)))))
>+
> (cl-defgeneric project-name (project)
>-  "A human-readable name for the project.
>+  "A human-readable name for the PROJECT.
> Nominally unique, but not enforced."
>   (file-name-nondirectory (directory-file-name (project-root project))))
>
>@@ -1391,7 +1422,7 @@ project-compile
>   "Run `compile' in the project root."
>   (declare (interactive-only compile))
>   (interactive)
>-  (let ((default-directory (project-root (project-current t)))
>+  (let ((default-directory (project-get-build-dir (project-current t)))
>         (compilation-buffer-name-function
>          (or project-compilation-buffer-name-function
>              compilation-buffer-name-function)))


[-- Attachment #2: project-compile.patch --]
[-- Type: text/plain, Size: 4160 bytes --]

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index ac18aceadcf..5a4274d6f68 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -51,6 +51,9 @@
 ;; files inside the root must not be considered a part of it).  It
 ;; should be consistent with `project-files'.
 ;;
+;; `project-build-dir' can be overridden if the project backend has some
+;; extra information about the project build directory.
+;;
 ;; This list can change in future versions.
 ;;
 ;; Transient project:
@@ -208,6 +211,22 @@ project-prompter
   :group 'project
   :version "30.1")
 
+(defcustom project-build-dir nil
+  "Build directory for current project.
+This is the custom that the user could specify in dir-locals to override
+the option specified by the project's backend."
+  :type 'directory
+  :safe t
+  :version "30.1")
+
+(defcustom project-compile-command nil
+  "Build command for current project.
+This is the custom that the user could specify in dir-locals to override
+the option specified by the project's backend."
+  :type 'directory
+  :safe t
+  :version "30.1")
+
 ;;;###autoload
 (defun project-current (&optional maybe-prompt directory)
   "Return the project instance in DIRECTORY, defaulting to `default-directory'.
@@ -289,8 +308,42 @@ project-external-roots
 headers search path, load path, class path, and so on."
   nil)
 
+(cl-defgeneric project-build-dir (_project)
+  "Return build directory of the current PROJECT.
+
+This function is intended to be defined by the backend when possible.
+Otherwise this returns nil and the `project-compile' command will be
+called in the project-root.
+If the user defines the directory-local variable `project-build-dir' it
+will have precedence over the result of this function."
+ nil)
+
+(defun project-get-build-dir (project)
+  "Return build directory of the current PROJECT.
+1. If the variable `project-build-dir' is defined, this function returns
+it.  2. Else if the function `project-build-dir' return non-nil.
+3. else it return the project-root.
+If the defined path is relative, this expands it relatively to the
+project's root."
+  (let ((dir (or project-build-dir           ;; variable for dir-locals or connection local vars
+                 (project-build-dir project) ;; backend function
+                 (project-root project))))   ;; I assume project-root is always absolute
+    (if (file-name-absolute-p dir)
+        dir
+      (expand-file-name dir (project-root project)))))
+
+(cl-defgeneric project-compile-command (_project)
+  "Return build command of the current PROJECT.
+
+This function is intended to be defined by the backend when possible.
+Otherwise this returns nil and the `project-compile' command will use
+the default `compile-command' value.  If the user defines the
+directory-local variable `project-build-command' it will have preference
+over this function and this will be never called."
+ nil)
+
 (cl-defgeneric project-name (project)
-  "A human-readable name for the project.
+  "A human-readable name for the PROJECT.
 Nominally unique, but not enforced."
   (file-name-nondirectory (directory-file-name (project-root project))))
 
@@ -1391,10 +1444,16 @@ project-compile
   "Run `compile' in the project root."
   (declare (interactive-only compile))
   (interactive)
-  (let ((default-directory (project-root (project-current t)))
-        (compilation-buffer-name-function
-         (or project-compilation-buffer-name-function
-             compilation-buffer-name-function)))
+  ;; I am wondering whenever we need to expand connection local
+  ;; variables at this point... maybe before or inside the let.
+  (let* ((project (project-current t))
+         (default-directory (project-get-build-dir project))
+         (compile-command (or project-compile-command
+                              (project-compile-command project)
+                              compile-command))
+         (compilation-buffer-name-function
+          (or project-compilation-buffer-name-function
+              compilation-buffer-name-function)))
     (call-interactively #'compile)))
 
 (defun project-recompile (&optional edit-command)

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-03-27 16:38         ` [PATCH] " Ergus
@ 2024-03-31  2:41           ` Dmitry Gutov
  2024-03-31 21:07             ` Ergus
  0 siblings, 1 reply; 53+ messages in thread
From: Dmitry Gutov @ 2024-03-31  2:41 UTC (permalink / raw)
  To: Ergus; +Cc: emacs-devel

Hi Ergus,

On 27/03/2024 18:38, Ergus wrote:

> Here I attach a very simple patch to allow project out of sources
> compilation and project build detection.
> 
> So far I already implemented a very primitive backend [1] for project.el
> using these features to detect and compile autotools and cmake projects;
> and, at some point I will propose it for elpa.
> 
> But I need to work a bit more on it; and, of course, the functionalities
> in the attached patch need to go to vanilla this way or some other
> equivalent.

I originally suggested a variable that stores a relative file name. But 
I see that you prefer the fetching of this file name to be more dynamic.

Why don't we just make it a function variable, at least at first?

Call it 'project-compile-dir`, which would be settable to a string 
(constant value, to be customized through dir-locals), or a function 
that either accepts the current project value, or is simply called 
"inside project root" (with default-directory bound to that value).

Alternatively, if you see a need to add additional methods 
('project-compile-command'?), it could become a new hook and a new 
facility (e.g. project-build-functions) which could be structures 
similarly to project.el, i.e. the hook returns a "project builder" 
value, and the value has some methods that it can defined/override. Then 
different build tool backends could be mixed with different project 
backends more easily.

So far I only see one additional method, though, and the constructed 
compile-command value is relatively simple. Curious to see whether this 
will get more complex with additional targets.

As far as the redefinition of 'project-name' goes, perhaps we'll allow 
project-vc-name to be a function as well? Or decide on some other way to 
assign a build tool the name to a project.

> Best,
> Ergus
> 
> [1] 
> https://github.com/Ergus/project-multi-mode/blob/master/project-multi-mode.el
> 
> 
> 
> On Tue, Mar 19, 2024 at 07:36:35PM +0100, Ergus wrote:
>> Hi Dmitry:
>>
>> Here I attach a very simple approach to add support for out of sources
>> compilation in project.el
>>
>> This is just a POC, but to have some starting point to discuss.
>>
>> Best,
>> Ergus
>>
>>
>>
>> On Sun, Mar 17, 2024 at 06:47:43PM +0100, Ergus wrote:
>>> On Sun, Mar 17, 2024 at 12:36:44PM +0100, Augusto Stoffel wrote:
>>>> On Sat, 16 Mar 2024 at 14:12, Ergus wrote:
>>>>
>>>>> 5. Project local variables (a man can dream, a man can dream)
>>>>>
>>>>> There are some situations where we want to have variables shared 
>>>>> among a
>>>>> project. (i.e some output directory, logging option when executing,
>>>>> flags, environment variables).
>>>>>
>>>>> At the moment these options work partially by using directory
>>>>> variables. If we have the concept of a "project", maybe it is 
>>>>> logical to
>>>>> have some sort of project scope concept, specially for projects 
>>>>> sharing
>>>>> a common root.
>>>>
>>>> For Emacs variables, why you think dir-local variables only works
>>>> “partially”?
>>>>
>>>> As to environment variables, they are generally assumed to be global in
>>>> Emacs and it goes a bit against the grain to make them buffer-local /
>>>> project dependent.  Nonetheless, there are packages for that (one is
>>>> buffer-env, which I wrote) and they work fine (at least for me).
>>>>
>>> They work for one directory, but when the project includes multiple
>>> roots (project-external-roots) the variables are somehow missing just
>>> when jumping with xref to a definition.
>>>
>>> However, we can ignore this as I mentioned before.
>>>
>>>>> For example vs-code adds a subdirectory with project variables that 
>>>>> the
>>>>> user (but also any plugin) can refer to in the project's scope.
>>>>
>>>> Is this meant to store editor configuration or environment 
>>>> variables?  I
>>>> guess turning VSCode configuration variables into Emacs alternatives
>>>> would be tricky.
>>>
>>> No please. No interaction-importing from other editors will be
>>> maintainable in the long path.
>>>
>>>> But if this is about env vars and the directory
>>>> organization is sensible, then one could configure buffer-env to
>>>> interoperate.
>>>
>>> This is it. The idea is to extend project.el in order to bring more
>>> project-oriented features.
>>>
>>> Lets say:
>>>
>>> The user (or project.el backend) could specify the `build` directory and
>>> the environment that needs to be set to execute the program from the
>>> build or bin directory.
>>>
>>> So with a simple modification of dir-locals.el the user may be capable
>>> to run M-x compile (or project-compile), M-x gud-gdb... or being more
>>> ambitious M-x project-generate (to call autogen.sh and ./configure or
>>> cmake depending of the project-backend)
>>>
>>> For sure we wont cover the 100% of the projects, but wuth automake and
>>> autotools we will be very close (and in the future someone could add
>>> other backends for it)
>>>
>>>
>>>
>>>
>>>
> 
>> commit cfa10dfeac01ae25a7acd6857fae0b377b625779
>> Author: Jimmy Aguilar Mena <kratsbinovish@gmail.com>
>> Date:   Tue Mar 19 17:57:25 2024 +0100
>>
>>    Add basic project-build-dir implementation
>>
>>    This is just a dummy prototype to discuss about alternatives.
>>
>> diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
>> index ac18aceadcf..0ca4ca85566 100644
>> --- a/lisp/progmodes/project.el
>> +++ b/lisp/progmodes/project.el
>> @@ -51,6 +51,9 @@
>> ;; files inside the root must not be considered a part of it).  It
>> ;; should be consistent with `project-files'.
>> ;;
>> +;; `project-build-dir' can be overridden if the project backend has some
>> +;; extra information about the project build directory.
>> +;;
>> ;; This list can change in future versions.
>> ;;
>> ;; Transient project:
>> @@ -208,6 +211,12 @@ project-prompter
>>   :group 'project
>>   :version "30.1")
>>
>> +(defcustom project-build-dir nil
>> +  "Build directory for current project."
>> +  :type 'directory
>> +  :safe t
>> +  :version "30.1")
>> +
>> ;;;###autoload
>> (defun project-current (&optional maybe-prompt directory)
>>   "Return the project instance in DIRECTORY, defaulting to 
>> `default-directory'.
>> @@ -289,8 +298,30 @@ project-external-roots
>> headers search path, load path, class path, and so on."
>>   nil)
>>
>> +(cl-defgeneric project-build-dir (_project)
>> +  "Return build directory of the current PROJECT.
>> +
>> +This function is intended to be defined by the backend when possible.
>> +Otherwise this returns nil and the `project-compile' command will be
>> +called in the project-root.
>> +If the user defines the directory-local variable `project-build-dir' it
>> +will have precedence over the result of this function."
>> + nil)
>> +
>> +(defun project-get-build-dir (project)
>> +  "Return build directory of the current PROJECT.
>> +If the variable `project-build-dir' is defined, this function returns
>> +it, otherwise it returns the project root.  If the defined path is
>> +relative, this expands it relatively to the project's root"
>> +  (let ((dir (or project-build-dir
>> +                 (project-build-dir project)
>> +                 (project-root project)))) ;; I assume project-root 
>> is absolute
>> +    (if (file-name-absolute-p dir)
>> +        dir
>> +      (expand-file-name dir (project-root project)))))
>> +
>> (cl-defgeneric project-name (project)
>> -  "A human-readable name for the project.
>> +  "A human-readable name for the PROJECT.
>> Nominally unique, but not enforced."
>>   (file-name-nondirectory (directory-file-name (project-root project))))
>>
>> @@ -1391,7 +1422,7 @@ project-compile
>>   "Run `compile' in the project root."
>>   (declare (interactive-only compile))
>>   (interactive)
>> -  (let ((default-directory (project-root (project-current t)))
>> +  (let ((default-directory (project-get-build-dir (project-current t)))
>>         (compilation-buffer-name-function
>>          (or project-compilation-buffer-name-function
>>              compilation-buffer-name-function)))
> 




^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-03-31  2:41           ` Dmitry Gutov
@ 2024-03-31 21:07             ` Ergus
  2024-04-01  7:49               ` Dirk-Jan C. Binnema
  2024-04-02 21:39               ` Richard Stallman
  0 siblings, 2 replies; 53+ messages in thread
From: Ergus @ 2024-03-31 21:07 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

Hi Dmitry:

On Sun, Mar 31, 2024 at 05:41:24AM +0300, Dmitry Gutov wrote:
>Hi Ergus,
>
>On 27/03/2024 18:38, Ergus wrote:
>
>>Here I attach a very simple patch to allow project out of sources
>>compilation and project build detection.
>>
>>So far I already implemented a very primitive backend [1] for project.el
>>using these features to detect and compile autotools and cmake projects;
>>and, at some point I will propose it for elpa.
>>
>>But I need to work a bit more on it; and, of course, the functionalities
>>in the attached patch need to go to vanilla this way or some other
>>equivalent.
>
>I originally suggested a variable that stores a relative file name. 
>But I see that you prefer the fetching of this file name to be more 
>dynamic.
>
Yes, that's actually the point. In out of sources compilation there is a
possibility that there will be more than one subdirectory to build, in
that case the user may be asked at least once which one to choose. This
is also valid to detect the `compile-commands.json' which may have
different version in each of the build-dir candidates.

https://github.com/Ergus/project-multi-mode/blob/c679d73a26b162e4158f6451027ca94e33faca0b/project-multi-mode.el#L109

>Why don't we just make it a function variable, at least at first?
>
Because I wanted somehow to follow the project.el schema of using a
generic that the backend could optionally define transparently.

And make that definition independent from the user config variable, so,
the user defined variable takes preference over the generic search
performed by the backend which if not defined automatically goes to
project root. Somehow adding a custom variable I think interferes with
the idea that the backend provides the functionality... But I may be
wrong.

>Call it 'project-compile-dir`, which would be settable to a string 
>(constant value, to be customized through dir-locals), or a function 
>that either accepts the current project value, or is simply called 
>"inside project root" (with default-directory bound to that value).
>
I will try this again, but I had some issues before.

>Alternatively, if you see a need to add additional methods 
>('project-compile-command'?), it could become a new hook and a new 
>facility (e.g. project-build-functions) which could be structures 
>similarly to project.el, i.e. the hook returns a "project builder" 
>value, and the value has some methods that it can defined/override. 
>Then different build tool backends could be mixed with different 
>project backends more easily.
>
I would prefer avoid more abstraction layers in the api. We actually
need some `project-compile-command' like function, but structuring it
with generics and more layer is IMHO adding too much complexity for a
public api... Isn't there a simpler alternative?

>So far I only see one additional method, though, and the constructed 
>compile-command value is relatively simple. Curious to see whether 
>this will get more complex with additional targets.
>
I added one method and with that I already support autotools and cmake.

>As far as the redefinition of 'project-name' goes, perhaps we'll allow 
>project-vc-name to be a function as well? Or decide on some other way 
>to assign a build tool the name to a project.
>
In project-name I actually only changed the docstring, I didn't change
anything else.

>>Best,
>>Ergus
>>
>>[1] https://github.com/Ergus/project-multi-mode/blob/master/project-multi-mode.el
>>
>>
>>
>>On Tue, Mar 19, 2024 at 07:36:35PM +0100, Ergus wrote:
>>>Hi Dmitry:
>>>
>>>Here I attach a very simple approach to add support for out of sources
>>>compilation in project.el
>>>
>>>This is just a POC, but to have some starting point to discuss.
>>>
>>>Best,
>>>Ergus
>>>
>>>
>>>
>>>On Sun, Mar 17, 2024 at 06:47:43PM +0100, Ergus wrote:
>>>>On Sun, Mar 17, 2024 at 12:36:44PM +0100, Augusto Stoffel wrote:
>>>>>On Sat, 16 Mar 2024 at 14:12, Ergus wrote:
>>>>>
>>>>>>5. Project local variables (a man can dream, a man can dream)
>>>>>>
>>>>>>There are some situations where we want to have variables 
>>>>>>shared among a
>>>>>>project. (i.e some output directory, logging option when executing,
>>>>>>flags, environment variables).
>>>>>>
>>>>>>At the moment these options work partially by using directory
>>>>>>variables. If we have the concept of a "project", maybe it 
>>>>>>is logical to
>>>>>>have some sort of project scope concept, specially for 
>>>>>>projects sharing
>>>>>>a common root.
>>>>>
>>>>>For Emacs variables, why you think dir-local variables only works
>>>>>“partially”?
>>>>>
>>>>>As to environment variables, they are generally assumed to be global in
>>>>>Emacs and it goes a bit against the grain to make them buffer-local /
>>>>>project dependent.  Nonetheless, there are packages for that (one is
>>>>>buffer-env, which I wrote) and they work fine (at least for me).
>>>>>
>>>>They work for one directory, but when the project includes multiple
>>>>roots (project-external-roots) the variables are somehow missing just
>>>>when jumping with xref to a definition.
>>>>
>>>>However, we can ignore this as I mentioned before.
>>>>
>>>>>>For example vs-code adds a subdirectory with project 
>>>>>>variables that the
>>>>>>user (but also any plugin) can refer to in the project's scope.
>>>>>
>>>>>Is this meant to store editor configuration or environment 
>>>>>variables?  I
>>>>>guess turning VSCode configuration variables into Emacs alternatives
>>>>>would be tricky.
>>>>
>>>>No please. No interaction-importing from other editors will be
>>>>maintainable in the long path.
>>>>
>>>>>But if this is about env vars and the directory
>>>>>organization is sensible, then one could configure buffer-env to
>>>>>interoperate.
>>>>
>>>>This is it. The idea is to extend project.el in order to bring more
>>>>project-oriented features.
>>>>
>>>>Lets say:
>>>>
>>>>The user (or project.el backend) could specify the `build` directory and
>>>>the environment that needs to be set to execute the program from the
>>>>build or bin directory.
>>>>
>>>>So with a simple modification of dir-locals.el the user may be capable
>>>>to run M-x compile (or project-compile), M-x gud-gdb... or being more
>>>>ambitious M-x project-generate (to call autogen.sh and ./configure or
>>>>cmake depending of the project-backend)
>>>>
>>>>For sure we wont cover the 100% of the projects, but wuth automake and
>>>>autotools we will be very close (and in the future someone could add
>>>>other backends for it)
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>>>commit cfa10dfeac01ae25a7acd6857fae0b377b625779
>>>Author: Jimmy Aguilar Mena <kratsbinovish@gmail.com>
>>>Date:   Tue Mar 19 17:57:25 2024 +0100
>>>
>>>   Add basic project-build-dir implementation
>>>
>>>   This is just a dummy prototype to discuss about alternatives.
>>>
>>>diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
>>>index ac18aceadcf..0ca4ca85566 100644
>>>--- a/lisp/progmodes/project.el
>>>+++ b/lisp/progmodes/project.el
>>>@@ -51,6 +51,9 @@
>>>;; files inside the root must not be considered a part of it).  It
>>>;; should be consistent with `project-files'.
>>>;;
>>>+;; `project-build-dir' can be overridden if the project backend has some
>>>+;; extra information about the project build directory.
>>>+;;
>>>;; This list can change in future versions.
>>>;;
>>>;; Transient project:
>>>@@ -208,6 +211,12 @@ project-prompter
>>>  :group 'project
>>>  :version "30.1")
>>>
>>>+(defcustom project-build-dir nil
>>>+  "Build directory for current project."
>>>+  :type 'directory
>>>+  :safe t
>>>+  :version "30.1")
>>>+
>>>;;;###autoload
>>>(defun project-current (&optional maybe-prompt directory)
>>>  "Return the project instance in DIRECTORY, defaulting to 
>>>`default-directory'.
>>>@@ -289,8 +298,30 @@ project-external-roots
>>>headers search path, load path, class path, and so on."
>>>  nil)
>>>
>>>+(cl-defgeneric project-build-dir (_project)
>>>+  "Return build directory of the current PROJECT.
>>>+
>>>+This function is intended to be defined by the backend when possible.
>>>+Otherwise this returns nil and the `project-compile' command will be
>>>+called in the project-root.
>>>+If the user defines the directory-local variable `project-build-dir' it
>>>+will have precedence over the result of this function."
>>>+ nil)
>>>+
>>>+(defun project-get-build-dir (project)
>>>+  "Return build directory of the current PROJECT.
>>>+If the variable `project-build-dir' is defined, this function returns
>>>+it, otherwise it returns the project root.  If the defined path is
>>>+relative, this expands it relatively to the project's root"
>>>+  (let ((dir (or project-build-dir
>>>+                 (project-build-dir project)
>>>+                 (project-root project)))) ;; I assume 
>>>project-root is absolute
>>>+    (if (file-name-absolute-p dir)
>>>+        dir
>>>+      (expand-file-name dir (project-root project)))))
>>>+
>>>(cl-defgeneric project-name (project)
>>>-  "A human-readable name for the project.
>>>+  "A human-readable name for the PROJECT.
>>>Nominally unique, but not enforced."
>>>  (file-name-nondirectory (directory-file-name (project-root project))))
>>>
>>>@@ -1391,7 +1422,7 @@ project-compile
>>>  "Run `compile' in the project root."
>>>  (declare (interactive-only compile))
>>>  (interactive)
>>>-  (let ((default-directory (project-root (project-current t)))
>>>+  (let ((default-directory (project-get-build-dir (project-current t)))
>>>        (compilation-buffer-name-function
>>>         (or project-compilation-buffer-name-function
>>>             compilation-buffer-name-function)))
>>
>
>



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-03-31 21:07             ` Ergus
@ 2024-04-01  7:49               ` Dirk-Jan C. Binnema
  2024-04-01 13:52                 ` Ergus
  2024-04-02 21:39               ` Richard Stallman
  1 sibling, 1 reply; 53+ messages in thread
From: Dirk-Jan C. Binnema @ 2024-04-01  7:49 UTC (permalink / raw)
  To: emacs-devel; +Cc: Dmitry Gutov

On Sunday Mar 31 2024, Ergus wrote:

> Hi Dmitry:
>
> On Sun, Mar 31, 2024 at 05:41:24AM +0300, Dmitry Gutov wrote:
>>Hi Ergus,
>>
>>On 27/03/2024 18:38, Ergus wrote:
>>

>> I originally suggested a variable that stores a relative file name. But I see
>> that you prefer the fetching of this file name to be more dynamic.
>>
> Yes, that's actually the point. In out of sources compilation there is a
> possibility that there will be more than one subdirectory to build, in
> that case the user may be asked at least once which one to choose. This
> is also valid to detect the `compile-commands.json' which may have
> different version in each of the build-dir candidates.
>
> https://github.com/Ergus/project-multi-mode/blob/c679d73a26b162e4158f6451027ca94e33faca0b/project-multi-mode.el#L109

My 0.02: Having separate build-directories seems to be increasingly
common (in my bubble I hardly see anything else anymore)

Allowing for _multiple_ build-directories is also quit common/useful.
E.g., for doing debug/release builds, or for building for different
target hw (embedded dev). For me, a daily need.

So having direct support for this in project.el would be quite welcome,
so I can use that instead of my private hacks.

>>Why don't we just make it a function variable, at least at first?
>>
> Because I wanted somehow to follow the project.el schema of using a
> generic that the backend could optionally define transparently.
>
> And make that definition independent from the user config variable, so,
> the user defined variable takes preference over the generic search
> performed by the backend which if not defined automatically goes to
> project root. Somehow adding a custom variable I think interferes with
> the idea that the backend provides the functionality... But I may be
> wrong.
>
>> Call it 'project-compile-dir`, which would be settable to a string (constant
>> value, to be customized through dir-locals), or a function that either accepts
>> the current project value, or is simply called "inside project root" (with
>> default-directory bound to that value).
>>
> I will try this again, but I had some issues before.
>
>> Alternatively, if you see a need to add additional methods
>> ('project-compile-command'?), it could become a new hook and a new facility
>> (e.g. project-build-functions) which could be structures similarly to
>> project.el, i.e. the hook returns a "project builder" value, and the value has
>> some methods that it can defined/override. Then different build tool backends
>> could be mixed with different project backends more easily.
>>
> I would prefer avoid more abstraction layers in the api. We actually
> need some `project-compile-command' like function, but structuring it
> with generics and more layer is IMHO adding too much complexity for a
> public api... Isn't there a simpler alternative?
>
>> So far I only see one additional method, though, and the constructed
>> compile-command value is relatively simple. Curious to see whether this will
>> get more complex with additional targets.
>>
> I added one method and with that I already support autotools and cmake.

(would be nice to add "meson" too!)

No immediate opinions on how to implement this, but things I commonly
want in projects:

- build
- run
- test
- install
- flash
- debug
- clean

It's possible of course to "multiplex" with the compile-command and put
in a "transient" or something like that; but it'd be nice to handle this
directly across projects; also for adding menu / toolbar commands.

Guess this is a bit of an open-ended list, so having `project-*-command`
may not to be the best.

Kind regards,
Dirk.

-- 
Dirk-Jan C. Binnema                  Helsinki, Finland
e:djcb@djcbsoftware.nl           w:www.djcbsoftware.nl
gpg: 6987 9CED 1745 9375 0F14 DA98 11DD FEA9 DCC4 A036



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-01  7:49               ` Dirk-Jan C. Binnema
@ 2024-04-01 13:52                 ` Ergus
  2024-04-01 15:09                   ` Dirk-Jan C. Binnema
  2024-04-02 23:23                   ` Dmitry Gutov
  0 siblings, 2 replies; 53+ messages in thread
From: Ergus @ 2024-04-01 13:52 UTC (permalink / raw)
  To: Dirk-Jan C. Binnema; +Cc: emacs-devel, Dmitry Gutov

On Mon, Apr 01, 2024 at 10:49:25AM +0300, Dirk-Jan C. Binnema wrote:
>On Sunday Mar 31 2024, Ergus wrote:
>
>> Hi Dmitry:
>>
>> On Sun, Mar 31, 2024 at 05:41:24AM +0300, Dmitry Gutov wrote:
>>>Hi Ergus,
>>>
>>>On 27/03/2024 18:38, Ergus wrote:
>>>
>
>>> I originally suggested a variable that stores a relative file name. But I see
>>> that you prefer the fetching of this file name to be more dynamic.
>>>
>> Yes, that's actually the point. In out of sources compilation there is a
>> possibility that there will be more than one subdirectory to build, in
>> that case the user may be asked at least once which one to choose. This
>> is also valid to detect the `compile-commands.json' which may have
>> different version in each of the build-dir candidates.
>>
>> https://github.com/Ergus/project-multi-mode/blob/c679d73a26b162e4158f6451027ca94e33faca0b/project-multi-mode.el#L109
>
>My 0.02: Having separate build-directories seems to be increasingly
>common (in my bubble I hardly see anything else anymore)
>
Same for me

>Allowing for _multiple_ build-directories is also quit common/useful.
>E.g., for doing debug/release builds, or for building for different
>target hw (embedded dev). For me, a daily need.
>
This is actually part of the main goal

>So having direct support for this in project.el would be quite welcome,
>so I can use that instead of my private hacks.
>
Projectile already have support for many backends and the implementation
is pretty clean. I am doing this because I thing that something so basic
must be part of vanilla.

>>>Why don't we just make it a function variable, at least at first?
>>>
>> Because I wanted somehow to follow the project.el schema of using a
>> generic that the backend could optionally define transparently.
>>
>> And make that definition independent from the user config variable, so,
>> the user defined variable takes preference over the generic search
>> performed by the backend which if not defined automatically goes to
>> project root. Somehow adding a custom variable I think interferes with
>> the idea that the backend provides the functionality... But I may be
>> wrong.
>>
>>> Call it 'project-compile-dir`, which would be settable to a string (constant
>>> value, to be customized through dir-locals), or a function that either accepts
>>> the current project value, or is simply called "inside project root" (with
>>> default-directory bound to that value).
>>>
>> I will try this again, but I had some issues before.
>>
>>> Alternatively, if you see a need to add additional methods
>>> ('project-compile-command'?), it could become a new hook and a new facility
>>> (e.g. project-build-functions) which could be structures similarly to
>>> project.el, i.e. the hook returns a "project builder" value, and the value has
>>> some methods that it can defined/override. Then different build tool backends
>>> could be mixed with different project backends more easily.
>>>
>> I would prefer avoid more abstraction layers in the api. We actually
>> need some `project-compile-command' like function, but structuring it
>> with generics and more layer is IMHO adding too much complexity for a
>> public api... Isn't there a simpler alternative?
>>
>>> So far I only see one additional method, though, and the constructed
>>> compile-command value is relatively simple. Curious to see whether this will
>>> get more complex with additional targets.
>>>
>> I added one method and with that I already support autotools and cmake.
>
>(would be nice to add "meson" too!)
>
No problem once the feature will be more advanced I can add it to the
project-multi backend in my github...

>No immediate opinions on how to implement this, but things I commonly
>want in projects:
>
>- build
>- run
>- test
>- install
>- flash
>- debug
>- clean
>
Lets start with the one common to all: `build'

Some others are difficult to support i.e I have no idea what is
`flash'. And makefile based projects may not have a clean target defined.

The install may be also controversial as it may refer to install in the
system or create an install directory.

And run is complicated because most projects create multiple outputs, so
we need to find some kind of standard method to list them in order to at
least ask the user in the first call.

As Dmitry said, the multiple targets may be also difficult to handle
with this abstraction either to `build only one' (may require completion
in the command manually) or to debug (in order to know which
target/executable to debug). I am not saying it is impossible, just that
I don't have enough knowledge about all the tools and their support for
such commands.

Also remember that this only applies to specific backends because vc
backends know nothing about this, so the commands need to be defined
somehow conditionally in M-x according to the backend... That's very far
from my elisp capabilities...

>It's possible of course to "multiplex" with the compile-command and put
>in a "transient" or something like that; but it'd be nice to handle this
>directly across projects; also for adding menu / toolbar commands.
>
>Guess this is a bit of an open-ended list, so having `project-*-command`
>may not to be the best.
>
Agree here, but I am not a good lisper... so probably Dmitry can bring
some advanced elisp feature I am not be aware of that could solve this issue.

>Kind regards,
>Dirk.
>
>-- 
>Dirk-Jan C. Binnema                  Helsinki, Finland
>e:djcb@djcbsoftware.nl           w:www.djcbsoftware.nl
>gpg: 6987 9CED 1745 9375 0F14 DA98 11DD FEA9 DCC4 A036
>



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-01 13:52                 ` Ergus
@ 2024-04-01 15:09                   ` Dirk-Jan C. Binnema
  2024-04-01 17:18                     ` Ergus
  2024-04-02 23:23                   ` Dmitry Gutov
  1 sibling, 1 reply; 53+ messages in thread
From: Dirk-Jan C. Binnema @ 2024-04-01 15:09 UTC (permalink / raw)
  To: emacs-devel

On Monday Apr 01 2024, Ergus wrote:

> On Mon, Apr 01, 2024 at 10:49:25AM +0300, Dirk-Jan C. Binnema wrote:

>>> I added one method and with that I already support autotools and cmake.
>>
>>(would be nice to add "meson" too!)
>>
> No problem once the feature will be more advanced I can add it to the
> project-multi backend in my github...

Great!

For meson support, I found I needed somewhat of a variation on
`locate-dominating-file' to find the remotest parent directory that has
a meson.build. Guess it's similar to what's needed for cmake. 

>>No immediate opinions on how to implement this, but things I commonly
>>want in projects:
>>
>>- build
>>- run
>>- test
>>- install
>>- flash
>>- debug
>>- clean
>>
> Lets start with the one common to all: `build'
>
> Some others are difficult to support i.e I have no idea what is
> `flash'. And makefile based projects may not have a clean target defined.

Yes, these are only useful for _some_ projects; currently I have some
custom solution to determine the flash command for the current
project (if any). But I think it'd be nice to somehow attach the
functionality to the current project. 

Note - it'd indeed be hard to automatically guess/determine this for
some project; in many cases, it would probably require the user to
configure some project-specific variable to set the commands.

> The install may be also controversial as it may refer to install in the
> system or create an install directory.
>
> And run is complicated because most projects create multiple outputs, so
> we need to find some kind of standard method to list them in order to at
> least ask the user in the first call.

As above, some of this might require some project-specific user
variables.

For comparison e.g. VSCode allows for doing such actions for many
projects; Emacs can do that as well, but currently it requires quite a
bit of custom tweaking; I'm just saying it'd be nice to integrate that
with project.el.

Some .project-locals.el could be useful for this.

> As Dmitry said, the multiple targets may be also difficult to handle
> with this abstraction either to `build only one' (may require
> completion in the command manually) or to debug (in order to know
> which target/executable to debug). I am not saying it is impossible,
> just that I don't have enough knowledge about all the tools and their
> support for such commands.
>
> Also remember that this only applies to specific backends because vc
> backends know nothing about this, so the commands need to be defined
> somehow conditionally in M-x according to the backend... That's very far
> from my elisp capabilities...
>
>>It's possible of course to "multiplex" with the compile-command and put
>>in a "transient" or something like that; but it'd be nice to handle this
>>directly across projects; also for adding menu / toolbar commands.
>>
>>Guess this is a bit of an open-ended list, so having `project-*-command`
>>may not to be the best.
>>
> Agree here, but I am not a good lisper... so probably Dmitry can bring
> some advanced elisp feature I am not be aware of that could solve this
> issue.

Sorry for hi-jacking your useful proposal to propose some more :-)

Kind regards,
Dirk.

-- 
Dirk-Jan C. Binnema                  Helsinki, Finland
e:djcb@djcbsoftware.nl           w:www.djcbsoftware.nl
gpg: 6987 9CED 1745 9375 0F14 DA98 11DD FEA9 DCC4 A036



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-01 15:09                   ` Dirk-Jan C. Binnema
@ 2024-04-01 17:18                     ` Ergus
  0 siblings, 0 replies; 53+ messages in thread
From: Ergus @ 2024-04-01 17:18 UTC (permalink / raw)
  To: Dirk-Jan C. Binnema; +Cc: emacs-devel

On Mon, Apr 01, 2024 at 06:09:48PM +0300, Dirk-Jan C. Binnema wrote:
>On Monday Apr 01 2024, Ergus wrote:
>
>> On Mon, Apr 01, 2024 at 10:49:25AM +0300, Dirk-Jan C. Binnema wrote:
>
>>>> I added one method and with that I already support autotools and cmake.
>>>
>>>(would be nice to add "meson" too!)
>>>
>> No problem once the feature will be more advanced I can add it to the
>> project-multi backend in my github...
>
>Great!
>
>For meson support, I found I needed somewhat of a variation on
>`locate-dominating-file' to find the remotest parent directory that has
>a meson.build. Guess it's similar to what's needed for cmake.�
>

Yes in project-multi I only need to add a simple entry like this:

(:type cmake                      // will be meson in your case
  :program "cmake"                 // name of program to build
  :root-hint "CMakeLists.txt"      // the filename to search in the root dir (gets the top most if nested)
  :build-hint "CMakeCache.txt"     // The file to search in the build dir (searches in root and all root's subdirs)
  :project-regex "project[[:blank:]]*([[:blank:]]*\\([[:alnum:]]+\\).+)" // Regex to search the project name in :root-hint
  :compile-command ":program --build ."  // A command pattern to build from build-dir; `:program' will be substituted
)

:project-regex can also be a function.

sadly I don't really have time now to implement another version of the
code needed in project.el... Because there is a part in Dmitry's
comments I don't really know how to manage to support the workflow I
have now...


>>>No immediate opinions on how to implement this, but things I commonly
>>>want in projects:
>>>
>>>- build
>>>- run
>>>- test
>>>- install
>>>- flash
>>>- debug
>>>- clean
>>>
>> Lets start with the one common to all: `build'
>>
>> Some others are difficult to support i.e I have no idea what is
>> `flash'. And makefile based projects may not have a clean target defined.
>
>Yes, these are only useful for _some_ projects; currently I have some
>custom solution to determine the flash command for the current
>project (if any). But I think it'd be nice to somehow attach the
>functionality to the current project.
>
>Note - it'd indeed be hard to automatically guess/determine this for
>some project; in many cases, it would probably require the user to
>configure some project-specific variable to set the commands.
>
>> The install may be also controversial as it may refer to install in the
>> system or create an install directory.
>>
>> And run is complicated because most projects create multiple outputs, so
>> we need to find some kind of standard method to list them in order to at
>> least ask the user in the first call.
>
>As above, some of this might require some project-specific user
>variables.
>
>For comparison e.g. VSCode allows for doing such actions for many
>projects; Emacs can do that as well, but currently it requires quite a
>bit of custom tweaking; I'm just saying it'd be nice to integrate that
>with project.el.
>
>Some .project-locals.el could be useful for this.
>
>> As Dmitry said, the multiple targets may be also difficult to handle
>> with this abstraction either to `build only one' (may require
>> completion in the command manually) or to debug (in order to know
>> which target/executable to debug). I am not saying it is impossible,
>> just that I don't have enough knowledge about all the tools and their
>> support for such commands.
>>
>> Also remember that this only applies to specific backends because vc
>> backends know nothing about this, so the commands need to be defined
>> somehow conditionally in M-x according to the backend... That's very far
>> from my elisp capabilities...
>>
>>>It's possible of course to "multiplex" with the compile-command and put
>>>in a "transient" or something like that; but it'd be nice to handle this
>>>directly across projects; also for adding menu / toolbar commands.
>>>
>>>Guess this is a bit of an open-ended list, so having `project-*-command`
>>>may not to be the best.
>>>
>> Agree here, but I am not a good lisper... so probably Dmitry can bring
>> some advanced elisp feature I am not be aware of that could solve this
>> issue.
>
>Sorry for hi-jacking your useful proposal to propose some more :-)
>
Not need to sorry... is actually good that someone else also cares about
this... ;)

>Kind regards,
>Dirk.
>
>-- 
>Dirk-Jan C. Binnema                  Helsinki, Finland
>e:djcb@djcbsoftware.nl           w:www.djcbsoftware.nl
>gpg: 6987 9CED 1745 9375 0F14 DA98 11DD FEA9 DCC4 A036
>



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-03-31 21:07             ` Ergus
  2024-04-01  7:49               ` Dirk-Jan C. Binnema
@ 2024-04-02 21:39               ` Richard Stallman
  2024-04-02 22:43                 ` Dr. Arne Babenhauserheide
  2024-04-03 10:40                 ` Konstantin Kharlamov
  1 sibling, 2 replies; 53+ messages in thread
From: Richard Stallman @ 2024-04-02 21:39 UTC (permalink / raw)
  To: Ergus; +Cc: dmitry, emacs-devel

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

What does "Project out of sources compilation" mean?
I am not sure how to parse that, let alone determine what it signfies.
-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)





^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-02 21:39               ` Richard Stallman
@ 2024-04-02 22:43                 ` Dr. Arne Babenhauserheide
  2024-04-05 21:40                   ` Richard Stallman
  2024-04-03 10:40                 ` Konstantin Kharlamov
  1 sibling, 1 reply; 53+ messages in thread
From: Dr. Arne Babenhauserheide @ 2024-04-02 22:43 UTC (permalink / raw)
  To: rms; +Cc: Ergus, dmitry, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 529 bytes --]


Richard Stallman <rms@gnu.org> writes:

> What does "Project out of sources compilation" mean?
> I am not sure how to parse that, let alone determine what it signfies.

That usually means that the compilation happens in a directory which is
not the source directory (similar to how make distcheck does it).

In this case that the build command is issued in a dedicated build
directory which is not the sources root.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-01 13:52                 ` Ergus
  2024-04-01 15:09                   ` Dirk-Jan C. Binnema
@ 2024-04-02 23:23                   ` Dmitry Gutov
  2024-04-03 19:47                     ` Ergus
  2024-04-06  2:05                     ` Ergus
  1 sibling, 2 replies; 53+ messages in thread
From: Dmitry Gutov @ 2024-04-02 23:23 UTC (permalink / raw)
  To: Ergus, Dirk-Jan C. Binnema; +Cc: emacs-devel

On 01/04/2024 16:52, Ergus wrote:
>> So having direct support for this in project.el would be quite welcome,
>> so I can use that instead of my private hacks.
>>
> Projectile already have support for many backends and the implementation
> is pretty clean. I am doing this because I thing that something so basic
> must be part of vanilla.

Speaking of Projectile, it has this block of definitions of "project 
types". Which could maybe better called "build tool types" in the 
context of this discussion.

It would be nice if we could port them over to a design which would be 
useable by both Projectile and project.el (and together with other 
project.el backends), rather than having every project backend 
re-implement these settings, or re-enumerate the supported types.

That's why the idea of orthogonal APIs seems appealing to me (the 
current one for file listing and root finding; the new one for build 
tools, and compiling, and running, I guess).

Until we've drawn up some design for the latter, though, maybe a 
customizable function variable will suffice.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-02 21:39               ` Richard Stallman
  2024-04-02 22:43                 ` Dr. Arne Babenhauserheide
@ 2024-04-03 10:40                 ` Konstantin Kharlamov
  2024-04-03 11:45                   ` Eli Zaretskii
  1 sibling, 1 reply; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-04-03 10:40 UTC (permalink / raw)
  To: rms, Ergus; +Cc: dmitry, emacs-devel

On Tue, 2024-04-02 at 17:39 -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider   
> ]]]
> [[[ whether defending the US Constitution against all enemies,    
> ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example.
> ]]]
> 
> What does "Project out of sources compilation" mean?
> I am not sure how to parse that, let alone determine what it
> signfies.

In Emacs terms, if you enter Emacs repo and execute

	mkdir build && cd build && ../configure && make

it will be an "out of sources compilation", because all artifacts¹ will
be inside this `build` dir rather than being scattered all over the
repo.

1: Emacs does not fully support it though, because while object files
and configs do end up inside `build`, however `*.elc` files are still
scattered all over the repo. But I hope you get the idea.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 10:40                 ` Konstantin Kharlamov
@ 2024-04-03 11:45                   ` Eli Zaretskii
  2024-04-03 13:31                     ` Konstantin Kharlamov
  0 siblings, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-03 11:45 UTC (permalink / raw)
  To: Konstantin Kharlamov; +Cc: rms, spacibba, dmitry, emacs-devel

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Cc: dmitry@gutov.dev, emacs-devel@gnu.org
> Date: Wed, 03 Apr 2024 13:40:59 +0300
> 
> 1: Emacs does not fully support it though, because while object files
> and configs do end up inside `build`, however `*.elc` files are still
> scattered all over the repo. But I hope you get the idea.

The *.elc files are considered part of the source tree (they are in
the release tarball, and are portable, so no need to rebuild them when
changing some configuration options).  Thus, Emacs does "fully support
it".



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 11:45                   ` Eli Zaretskii
@ 2024-04-03 13:31                     ` Konstantin Kharlamov
  2024-04-03 14:11                       ` Eli Zaretskii
  0 siblings, 1 reply; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-04-03 13:31 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: rms, spacibba, dmitry, emacs-devel

On Wed, 2024-04-03 at 14:45 +0300, Eli Zaretskii wrote:
> > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > Cc: dmitry@gutov.dev, emacs-devel@gnu.org
> > Date: Wed, 03 Apr 2024 13:40:59 +0300
> > 
> > 1: Emacs does not fully support it though, because while object
> > files
> > and configs do end up inside `build`, however `*.elc` files are
> > still
> > scattered all over the repo. But I hope you get the idea.
> 
> The *.elc files are considered part of the source tree (they are in
> the release tarball, and are portable, so no need to rebuild them
> when
> changing some configuration options).  

They are generated during compilation and are not commited into git. 
That makes them build artifacts.  The fact they're distributed in
release tarballs does not make it less true.  Distributing them in
tarballs is not strictly necessary, exactly because they can be
generated by anyone.

Having .elc files in the source tree results in problems when a user
created a dir `build-feature1` for one feature branch, and a dir
`build-feature2` for another; and then `build-feature1` somehow ends up
having "feature2".  Which is because it's using build artifacts that
are outside `build-feature1`.

In fact, I myself has experienced some problems due to that while
researching `61960` bug and then later making a separate build dir for
something else.  I don't remember details, but it has been something
with the fact that elc files are outside build dir, and then for some
reason Emacs hasn't been regenerating them or something like that (Idk
exactly what was up with them, I didn't dig into it).

Thus, Emacs certainly does not fully support "out of tree" builds.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 13:31                     ` Konstantin Kharlamov
@ 2024-04-03 14:11                       ` Eli Zaretskii
  2024-04-03 15:00                         ` Konstantin Kharlamov
  0 siblings, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-03 14:11 UTC (permalink / raw)
  To: Konstantin Kharlamov; +Cc: rms, spacibba, dmitry, emacs-devel

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev, emacs-devel@gnu.org
> Date: Wed, 03 Apr 2024 16:31:30 +0300
> 
> On Wed, 2024-04-03 at 14:45 +0300, Eli Zaretskii wrote:
> > > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > > Cc: dmitry@gutov.dev, emacs-devel@gnu.org
> > > Date: Wed, 03 Apr 2024 13:40:59 +0300
> > > 
> > > 1: Emacs does not fully support it though, because while object
> > > files
> > > and configs do end up inside `build`, however `*.elc` files are
> > > still
> > > scattered all over the repo. But I hope you get the idea.
> > 
> > The *.elc files are considered part of the source tree (they are in
> > the release tarball, and are portable, so no need to rebuild them
> > when
> > changing some configuration options).  
> 
> They are generated during compilation and are not commited into git. 
> That makes them build artifacts.

How is this relevant to whether or not Emacs supports out of tree
builds?  Building from Git is a special kind of activity; it has its
own INSTALL file and _must_ build some of the files in the source
tree.  Are you aware of many projects whose build from Git can be done
without first populating the source tree with some files?  Heck, you
don't even have the configure script when you clone the repository,
and out-of-tree build works by invoking the configure script from a
directory _other_ than the one where the script lives.  Right?

> The fact they're distributed in release tarballs does not make it
> less true.  Distributing them in tarballs is not strictly necessary,
> exactly because they can be generated by anyone.

Not true.

> Having .elc files in the source tree results in problems when a user
> created a dir `build-feature1` for one feature branch, and a dir
> `build-feature2` for another; and then `build-feature1` somehow ends up
> having "feature2".  Which is because it's using build artifacts that
> are outside `build-feature1`.

Again, not relevant to the aspect on which I commented.

> Thus, Emacs certainly does not fully support "out of tree" builds.

It does.  You are just talking about a very different kind of "out of
tree build".



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 14:11                       ` Eli Zaretskii
@ 2024-04-03 15:00                         ` Konstantin Kharlamov
  2024-04-03 15:47                           ` Eli Zaretskii
  0 siblings, 1 reply; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-04-03 15:00 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: rms, spacibba, dmitry, emacs-devel

On Wed, 2024-04-03 at 17:11 +0300, Eli Zaretskii wrote:
> > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev,
> > emacs-devel@gnu.org
> > Date: Wed, 03 Apr 2024 16:31:30 +0300
> > 
> > On Wed, 2024-04-03 at 14:45 +0300, Eli Zaretskii wrote:
> > > > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > > > Cc: dmitry@gutov.dev, emacs-devel@gnu.org
> > > > Date: Wed, 03 Apr 2024 13:40:59 +0300
> > > > 
> > > > 1: Emacs does not fully support it though, because while object
> > > > files
> > > > and configs do end up inside `build`, however `*.elc` files are
> > > > still
> > > > scattered all over the repo. But I hope you get the idea.
> > > 
> > > The *.elc files are considered part of the source tree (they are
> > > in
> > > the release tarball, and are portable, so no need to rebuild them
> > > when
> > > changing some configuration options).  
> > 
> > They are generated during compilation and are not commited into
> > git. 
> > That makes them build artifacts.
> 
> How is this relevant to whether or not Emacs supports out of tree
> builds? 

Directly: the point of "out of tree build" is encapsulating build
artifacts in the build dir.  I'm pointing out that Emacs has build
artifacts being put outside build dir, hence the "out of tree build" 
workflow is not working as expected.

>  Building from Git is a special kind of activity; it has its
> own INSTALL file and _must_ build some of the files in the source
> tree.  Are you aware of many projects whose build from Git can be
> done
> without first populating the source tree with some files?

Most existing active projects 😊  Most that were using autotools/make
has migrated to Meson.  That includes active freedesktop projects
(Mesa, Xorg, libinput, Pulseaudio, etc), Gnome projects, i3wm, Compton
after it became Picom, all active Enlightenment projects…  The list
goes on and on.  Meson drastically reduces maintainance burden but it's
a separate story.  What matters is that Meson by design works with out
of tree builds.  It is possible to hack in generating files in the
tree, but it's frowned upon for many reasons.

Then you have CMake projects, which I think has always supported out of
tree build without creating any config files whatsoever outside?
Offhand that includes all KDE projects.

Lastly, you have autotools projects where `configure` file is not
generated 😊 `fio` is one example (although admittedly I didn't test
out of tree build in there, but clearly if you have `configure` as part
of the codebase, then you don't have to generate it in the tree).

>   Heck, you
> don't even have the configure script when you clone the repository,
> and out-of-tree build works by invoking the configure script from a
> directory _other_ than the one where the script lives.  Right?

Good point.  That's yet another reason "out of tree build" workflow
isn't fully supported.  I just didn't think about it because didn't
have problems with that, but you are correct here.

> > The fact they're distributed in release tarballs does not make it
> > less true.  Distributing them in tarballs is not strictly
> > necessary,
> > exactly because they can be generated by anyone.
> 
> Not true.

Well, admittedly I didn't try building from release tarball.  But if it
lacks some code to re-create `.elc` files, that's a bug.  I can explain
why if you're interested.

> > Having .elc files in the source tree results in problems when a
> > user
> > created a dir `build-feature1` for one feature branch, and a dir
> > `build-feature2` for another; and then `build-feature1` somehow
> > ends up
> > having "feature2".  Which is because it's using build artifacts
> > that
> > are outside `build-feature1`.
> 
> Again, not relevant to the aspect on which I commented.

Fair enough.  I was just pointing out why "out of tree builds" should
work the way they don't in Emacs right now.  But yeah, if you already
understand that, I guess my explanation on that is unnecessary.

> > Thus, Emacs certainly does not fully support "out of tree" builds.
> 
> It does.  You are just talking about a very different kind of "out of
> tree build".

Well… There does not exist strict definition for "out of tree build"
that I can point you to, so that risks becoming a philosophical debate.

To understand what that means you need to understand what it solves. 
Which you seem to understand already per the previous paragraph, so I
can only add that "partial out of tree build" still has problems that
"full out of tree build" solves.

So there is only one kind of "out of tree build", the "full" one. The
"partial" one should be explicitly specified as being only partial,
whereas saying "out of tree build" without any prefix implies it is a
"full" one.

I hope now you understand why I'm saying Emacs doesn't fully support it
😊



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 15:00                         ` Konstantin Kharlamov
@ 2024-04-03 15:47                           ` Eli Zaretskii
  2024-04-03 17:27                             ` Konstantin Kharlamov
  0 siblings, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-03 15:47 UTC (permalink / raw)
  To: Konstantin Kharlamov; +Cc: rms, spacibba, dmitry, emacs-devel

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev, emacs-devel@gnu.org
> Date: Wed, 03 Apr 2024 18:00:28 +0300
> 
> I hope now you understand why I'm saying Emacs doesn't fully support it
> 😊

I don't.  I think you are plainly mistaken.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 15:47                           ` Eli Zaretskii
@ 2024-04-03 17:27                             ` Konstantin Kharlamov
  2024-04-03 18:22                               ` Eli Zaretskii
       [not found]                               ` <87jzlefgi9.fsf@dick>
  0 siblings, 2 replies; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-04-03 17:27 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: rms, spacibba, dmitry, emacs-devel

On Wed, 2024-04-03 at 18:47 +0300, Eli Zaretskii wrote:
> > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev,
> > emacs-devel@gnu.org
> > Date: Wed, 03 Apr 2024 18:00:28 +0300
> >
> > I hope now you understand why I'm saying Emacs doesn't fully
> > support it
> > 😊
>
> I don't.  I think you are plainly mistaken.

Sure, that's a fair point, because if I consider something simple, that doesn't
necessarily mean other people have the knowledge to think similarly.  So to explain
why I was saying that here's some points from my previous email that clarify that:

The point of "out of tree build" is encapsulating build artifacts in the build dir.
Emacs has some build artifacts outside build dir, hence the "out of tree build"
workflow is not working as expected.

To understand what "out of tree build" means you need to understand what it solves.
Which you seem to understand already per prior discussion (but I may be mistaken
here, just tell me if that's so), so let me just add that "partial out of tree build"
still has problems that "full out of tree build" solves.

So there is only one kind of "out of tree build", the "full" one.  The "partial" one
should be explicitly specified as such, whereas saying "out of tree build" without
any prefix implies it is a "full" one.

I hope now you understand why I'm saying Emacs doesn't fully support it 😊



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 17:27                             ` Konstantin Kharlamov
@ 2024-04-03 18:22                               ` Eli Zaretskii
  2024-04-03 19:08                                 ` Konstantin Kharlamov
       [not found]                               ` <87jzlefgi9.fsf@dick>
  1 sibling, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-03 18:22 UTC (permalink / raw)
  To: Konstantin Kharlamov; +Cc: rms, spacibba, dmitry, emacs-devel

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev, emacs-devel@gnu.org
> Date: Wed, 03 Apr 2024 20:27:09 +0300
> 
> I hope now you understand why I'm saying Emacs doesn't fully support it 😊

I still think you are mistaken.  What you call "out of tree build" is
not what is generally meant by that.  Bootstrapping a repository
invariably causes some "build artifacts" to be deposed in the source
tree, and the more the project makes a point of not having generated
files in the repository, the more artifacts are needed in the source
tree before you can start building out of tree.  And that is
perfectly okay, nothing wrong about that.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
       [not found]                               ` <87jzlefgi9.fsf@dick>
@ 2024-04-03 18:44                                 ` Konstantin Kharlamov
  0 siblings, 0 replies; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-04-03 18:44 UTC (permalink / raw)
  To: dick; +Cc: Eli Zaretskii, rms, spacibba, dmitry, emacs-devel

Wow. Please, offense is unwelcome.  Eli is a great man who maintains
the project and writes code. Other people come and go, Eli stays.  I'm
very thankful to him for doing that job.  Disagreement happens, that's
okay.  No need to call names.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 18:22                               ` Eli Zaretskii
@ 2024-04-03 19:08                                 ` Konstantin Kharlamov
  2024-04-03 20:12                                   ` Ergus
  2024-04-04  5:07                                   ` Eli Zaretskii
  0 siblings, 2 replies; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-04-03 19:08 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: rms, spacibba, dmitry, emacs-devel

On Wed, 2024-04-03 at 21:22 +0300, Eli Zaretskii wrote:
> > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev,
> > emacs-devel@gnu.org
> > Date: Wed, 03 Apr 2024 20:27:09 +0300
> > 
> > I hope now you understand why I'm saying Emacs doesn't fully
> > support it 😊
> 
> I still think you are mistaken.  What you call "out of tree build" is
> not what is generally meant by that.

Okay, what do you consider meant by that?

> Bootstrapping a repository
> invariably causes some "build artifacts" to be deposed in the source
> tree

Sure, that's unfortunately how autotools work.  On the upside though,
it is rare for these files to cause problems.  Which can't be said
about .elc ones.

> , and the more the project makes a point of not having generated
> files in the repository, the more artifacts are needed in the source
> tree before you can start building out of tree.  

Citation needed 😄  You asked me previously for examples of projects
that do not generate any files in the tree, I provided you with that 😊
I can certainly assure you that you don't necessarily have to generate
files "in tree".  I worked with many different projects and distinct
build systems, so I have a lot of experience in that regard.  Do you
have in mind some specific case that you see a problem with?

> And that is
> perfectly okay, nothing wrong about that.

There are problems with generating code in tree, some of which I did
show to you previously.  Unless I misunderstand what you mean by that.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-02 23:23                   ` Dmitry Gutov
@ 2024-04-03 19:47                     ` Ergus
  2024-04-06  2:05                     ` Ergus
  1 sibling, 0 replies; 53+ messages in thread
From: Ergus @ 2024-04-03 19:47 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: Dirk-Jan C. Binnema, emacs-devel

Hi Dmitry:

So far I Think I understood part of your las email about functionality.

On Wed, Apr 03, 2024 at 02:23:03AM +0300, Dmitry Gutov wrote:
>On 01/04/2024 16:52, Ergus wrote:
>>>So having direct support for this in project.el would be quite welcome,
>>>so I can use that instead of my private hacks.
>>>
>>Projectile already have support for many backends and the implementation
>>is pretty clean. I am doing this because I thing that something so basic
>>must be part of vanilla.
>
>Speaking of Projectile, it has this block of definitions of "project 
>types". Which could maybe better called "build tool types" in the 
>context of this discussion.
>
>It would be nice if we could port them over to a design which would be 
>useable by both Projectile and project.el (and together with other 
>project.el backends), rather than having every project backend 
>re-implement these settings, or re-enumerate the supported types.
>
Their design is actually fine, but I would prefer to keep things simpler
in the api side because project is intended to be more general.

Their use case adds too many functions in order to register, call, test
and generate the different backends I thing it is actually too much for
vanilla. If we want that complexity maybe is better to retake the last
year discussion you mentioned before:

https://lists.gnu.org/archive/html/emacs-devel/2023-05/msg00509.html

But adding that will require also parallel work and code in order to
integrate with project.el.

>That's why the idea of orthogonal APIs seems appealing to me (the 
>current one for file listing and root finding; the new one for build 
>tools, and compiling, and running, I guess).
>
I would probably insist in a simpler basic feature still in the current
project.el. I can't see the features as a separated thing.

What I would probably propose here is (maybe) to move the vc-backend to
it's own file as a separated package in order to keep the general api
separated from the "plugin". So if a developer wants to create a backend
it is clear whats needed and what the standard implementation example...

>Until we've drawn up some design for the latter, though, maybe a 
>customizable function variable will suffice.
>
What I have in mind as now is to add a variable and a method that can be
both optionally defined.

The method defined by backends and the variable by the user (maybe in
dir-locals).

This way the backends won't be forced to define any of these if
compilation is not supported (i.e vs-backend) or if the project is for a
non-compilable languaje (i.e an hypothetic python project)

project-compile-info accepts a second variable that in this case expects
'dir or 'compile-cmd in any case when not implemented the function gives
null and the caller (compile in this case) falls back to the default
values.

This can accept any other command like 'generate-cmd 'test-cmd and so
on.  In the future we can define any other random command relying on
these

So far the only we need for compilation is more or less this:

```
diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index a10e24f3e28..9d77d5452f3 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -290,8 +290,29 @@ project-external-roots
  headers search path, load path, class path, and so on."
    nil)
  
+(cl-defgeneric project-compile-info (_project _info)
+  "Return build information of the current PROJECT.
+
+This function is intended to be defined by the backend when possible.
+Otherwise this returns nil and the `project-compile' command will use
+the default values.
+The current valid values for INFO are `dir' and `compile-cmd'"
+  nil)
+
+(defun project-get-build-dir (project)
+  "Return absolute build directory for the current PROJECT.
+1. Tries the generic function `project-compile-info' with info='dir.
+2. else it return the project-root.
+If the defined path is relative, this expands it relatively to the
+project's root."
+  (let ((dir (or (project-compile-info project 'dir)   ;; backend function
+                 (project-root project))))        ;; I assume project-root is always absolute
+    (if (file-name-absolute-p dir)
+        dir
+      (expand-file-name dir (project-root project)))))


@@ -1392,10 +1413,15 @@ project-compile
    "Run `compile' in the project root."
    (declare (interactive-only compile))
    (interactive)
-  (let ((default-directory (project-root (project-current t)))
-        (compilation-buffer-name-function
-         (or project-compilation-buffer-name-function
-             compilation-buffer-name-function)))
+  ;; I am wondering whenever we need to expand connection local
+  ;; variables at this point... maybe before or inside the let.
+  (let* ((project (project-current t))
+         (default-directory (project-get-build-dir project))
+         (compile-command (or (project-compile-info project 'compile-cmd)
+                              compile-command))
+         (compilation-buffer-name-function
+          (or project-compilation-buffer-name-function
+              compilation-buffer-name-function)))
      (call-interactively #'compile)))

```

Finally the only missing detail is user side configuration... I was
thinking in a plist defconst (similar to the approach in
eglot-workspace-configuration) so the user can set it completelly or
partially and we only need to do:

(or (plist-get project-info-plist 'compile-cmd)
     (project-compile-info project 'compile-cmd)
     compile-command)

In this way the user only has one variable to set in the config with all
the parameters optional as well which stays easy, simple in
implementation and with a predictable behavior.

WDYT??
Best,
Ergus



^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 19:08                                 ` Konstantin Kharlamov
@ 2024-04-03 20:12                                   ` Ergus
  2024-04-04  5:26                                     ` Eli Zaretskii
  2024-04-04  5:07                                   ` Eli Zaretskii
  1 sibling, 1 reply; 53+ messages in thread
From: Ergus @ 2024-04-03 20:12 UTC (permalink / raw)
  To: Konstantin Kharlamov; +Cc: Eli Zaretskii, rms, dmitry, emacs-devel

Hi Eli:

I won't come into this discussion but:

My basic concept of OOSC (probably mistaken) is that the sources of the
project are those required to reproduce a program, because otherwise it
is not possible or are too difficult/expensive to reproduce.

If I change project.el, then project.elc is not valid anymore and needs
to be regenerated... so, project.el is the source and the elc is a
target (not necessarily a final one)

In the same line, when we do out of sources compilation it may be
possible to remove the `build` directory and have the project sources
exactly as downloaded from sources. Without any other clean
needed...

Also in same line, for example, when using GIT_NAMESPACE the project.el
will change with the namespace, but the elc won't (as not tracked by
git), so GIT_NAMESPACE becomes inconsistent and requires a regeneration
of .elc, so, not a source, but a target

Finally when executing from build1 it may be possible to access the
version of project.elc used when the project was compiled in that
directory... independently of if I changed project.el latter and
compiled in build2. Latter actions in source + compilation in build2
should not affect the consistency of build1.



On Wed, Apr 03, 2024 at 10:08:16PM +0300, Konstantin Kharlamov wrote:
>On Wed, 2024-04-03 at 21:22 +0300, Eli Zaretskii wrote:
>> > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
>> > Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev,
>> > emacs-devel@gnu.org
>> > Date: Wed, 03 Apr 2024 20:27:09 +0300
>> >
>> > I hope now you understand why I'm saying Emacs doesn't fully
>> > support it 😊
>>
>> I still think you are mistaken.  What you call "out of tree build" is
>> not what is generally meant by that.
>
>Okay, what do you consider meant by that?
>
>> Bootstrapping a repository
>> invariably causes some "build artifacts" to be deposed in the source
>> tree
>
>Sure, that's unfortunately how autotools work.  On the upside though,
>it is rare for these files to cause problems.  Which can't be said
>about .elc ones.
>
>> , and the more the project makes a point of not having generated
>> files in the repository, the more artifacts are needed in the source
>> tree before you can start building out of tree.  
>
>Citation needed 😄  You asked me previously for examples of projects
>that do not generate any files in the tree, I provided you with that 😊
>I can certainly assure you that you don't necessarily have to generate
>files "in tree".  I worked with many different projects and distinct
>build systems, so I have a lot of experience in that regard.  Do you
>have in mind some specific case that you see a problem with?
>
>> And that is
>> perfectly okay, nothing wrong about that.
>
>There are problems with generating code in tree, some of which I did
>show to you previously.  Unless I misunderstand what you mean by that.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 19:08                                 ` Konstantin Kharlamov
  2024-04-03 20:12                                   ` Ergus
@ 2024-04-04  5:07                                   ` Eli Zaretskii
  1 sibling, 0 replies; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-04  5:07 UTC (permalink / raw)
  To: Konstantin Kharlamov; +Cc: rms, spacibba, dmitry, emacs-devel

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev, emacs-devel@gnu.org
> Date: Wed, 03 Apr 2024 22:08:16 +0300
> 
> On Wed, 2024-04-03 at 21:22 +0300, Eli Zaretskii wrote:
> > > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > > Cc: rms@gnu.org, spacibba@aol.com, dmitry@gutov.dev,
> > > emacs-devel@gnu.org
> > > Date: Wed, 03 Apr 2024 20:27:09 +0300
> > > 
> > > I hope now you understand why I'm saying Emacs doesn't fully
> > > support it 😊
> > 
> > I still think you are mistaken.  What you call "out of tree build" is
> > not what is generally meant by that.
> 
> Okay, what do you consider meant by that?

In general, a build outside of the tree where a release tarball has
been unpacked.  At the very least, a build _after_ autogen.sh (or a
similar "bootstrap" script) was run.

Note that the rules which produce *.elc files do deposit them in the
build tree, not in the source tree, btw.  AFAIR, the only Lisp files
produced by a build from Git that are deposited in the source tree are
those built by admin/grammars/Makefile.  If someone wants to modify
that Makefile to produce the *.el files in the build tree, I won't
object, but rejecting the Emacs out-of-tree build procedure because we
currently put them in the source tree is absurd, IMNSHO, since *.el
files are certainly portable and not configuration- or
platform-dependent.  Putting platform-independent generated files in
the source tree has the advantage of making the subsequent builds
faster, sometimes significantly faster.

> > Bootstrapping a repository
> > invariably causes some "build artifacts" to be deposed in the source
> > tree
> 
> Sure, that's unfortunately how autotools work.

See above: not just autotools.  And anyway, as long as the Emacs build
uses autotools, we can only support out-of-tree builds in the style
supported by autotools.

> On the upside though, it is rare for these files to cause problems.
> Which can't be said about .elc ones.

See above: the *.elc files are not relevant here.

> > , and the more the project makes a point of not having generated
> > files in the repository, the more artifacts are needed in the source
> > tree before you can start building out of tree.  
> 
> Citation needed 😄  You asked me previously for examples of projects
> that do not generate any files in the tree, I provided you with that 😊
> I can certainly assure you that you don't necessarily have to generate
> files "in tree".  I worked with many different projects and distinct
> build systems, so I have a lot of experience in that regard.  Do you
> have in mind some specific case that you see a problem with?

Yes, look at Gawk, for example: they have many generated files in Git.
Even Info files are in Git.

Another example is GDB: they store the configure script in Git.

> > And that is
> > perfectly okay, nothing wrong about that.
> 
> There are problems with generating code in tree, some of which I did
> show to you previously.

There are problems everywhere, but those are quite easy to deal with.
Otherwise you wouldn't be seeing projects doing that, and doing that
on purpose.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-03 20:12                                   ` Ergus
@ 2024-04-04  5:26                                     ` Eli Zaretskii
  2024-04-04  9:59                                       ` Ergus
  0 siblings, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-04  5:26 UTC (permalink / raw)
  To: Ergus; +Cc: Hi-Angel, rms, dmitry, emacs-devel

> Date: Wed, 3 Apr 2024 22:12:44 +0200
> From: Ergus <spacibba@aol.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, rms@gnu.org, dmitry@gutov.dev, 
> 	emacs-devel@gnu.org
> 
> My basic concept of OOSC (probably mistaken) is that the sources of the
> project are those required to reproduce a program, because otherwise it
> is not possible or are too difficult/expensive to reproduce.
> 
> If I change project.el, then project.elc is not valid anymore and needs
> to be regenerated... so, project.el is the source and the elc is a
> target (not necessarily a final one)

The same is true for a change in configure.ac: the configure script is
no longer valid after that, and needs to be rebuilt.

> In the same line, when we do out of sources compilation it may be
> possible to remove the `build` directory and have the project sources
> exactly as downloaded from sources. Without any other clean
> needed...

That's not how the GNU build infrastructure works.  You have the
distclean target in Makefile to clean up the tree for the new build.

> Also in same line, for example, when using GIT_NAMESPACE the project.el
> will change with the namespace, but the elc won't (as not tracked by
> git), so GIT_NAMESPACE becomes inconsistent and requires a regeneration
> of .elc, so, not a source, but a target

That is a problem with project.el, I think, not with how Emacs is
built.  In every project, when you bootstrap it, i.e. build from the
VCS repository, there are some generated files that are part of the
source tree.

> Finally when executing from build1 it may be possible to access the
> version of project.elc used when the project was compiled in that
> directory... independently of if I changed project.el latter and
> compiled in build2. Latter actions in source + compilation in build2
> should not affect the consistency of build1.

You are talking about modifying project.el while it is being used by a
running Emacs session?  In that case, all bets are off, I think.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04  5:26                                     ` Eli Zaretskii
@ 2024-04-04  9:59                                       ` Ergus
  2024-04-04 11:59                                         ` Eli Zaretskii
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-04-04  9:59 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Hi-Angel, rms, dmitry, emacs-devel

Hi Eli

On April 4, 2024 7:26:35 AM GMT+02:00, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Wed, 3 Apr 2024 22:12:44 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>, rms@gnu.org, dmitry@gutov.dev, 
>> 	emacs-devel@gnu.org
>> 
>> My basic concept of OOSC (probably mistaken) is that the sources of the
>> project are those required to reproduce a program, because otherwise it
>> is not possible or are too difficult/expensive to reproduce.
>> 
>> If I change project.el, then project.elc is not valid anymore and needs
>> to be regenerated... so, project.el is the source and the elc is a
>> target (not necessarily a final one)
>
>The same is true for a change in configure.ac: the configure script is
>no longer valid after that, and needs to be rebuilt.
>
And that's why the newer build systems generate the equivalent to the configure also out of sources. Or use the equivalent to configure.ac directly as input, generating everything out of sources.


>> In the same line, when we do out of sources compilation it may be
>> possible to remove the `build` directory and have the project sources
>> exactly as downloaded from sources. Without any other clean
>> needed...
>
>That's not how the GNU build infrastructure works.  You have the
>distclean target in Makefile to clean up the tree for the new build.
>
Makefile does not support oosc at all. Actually CMake generates the makefiles outside sources.


>> Also in same line, for example, when using GIT_NAMESPACE the project.el
>> will change with the namespace, but the elc won't (as not tracked by
>> git), so GIT_NAMESPACE becomes inconsistent and requires a regeneration
>> of .elc, so, not a source, but a target
>
>That is a problem with project.el, I think, not with how Emacs is
>built.  In every project, when you bootstrap it, i.e. build from the
>VCS repository, there are some generated files that are part of the
>source tree.
>
Indeed, CMake and actually most of newer autotools projects prefer to generate those files out of sources too.

>> Finally when executing from build1 it may be possible to access the
>> version of project.elc used when the project was compiled in that
>> directory... independently of if I changed project.el latter and
>> compiled in build2. Latter actions in source + compilation in build2
>> should not affect the consistency of build1.
>
>You are talking about modifying project.el while it is being used by a
>running Emacs session?  In that case, all bets are off, I think.

No I mean having two emacs builds in different subdirectories. For example one compiled before  certain change and the other after 




-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04  9:59                                       ` Ergus
@ 2024-04-04 11:59                                         ` Eli Zaretskii
  2024-04-04 12:34                                           ` Ergus
  0 siblings, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-04 11:59 UTC (permalink / raw)
  To: Ergus; +Cc: Hi-Angel, rms, dmitry, emacs-devel

> Date: Thu, 04 Apr 2024 11:59:29 +0200
> From: Ergus <spacibba@aol.com>
> CC: Hi-Angel@yandex.ru, rms@gnu.org, dmitry@gutov.dev, emacs-devel@gnu.org
> 
> >> Finally when executing from build1 it may be possible to access the
> >> version of project.elc used when the project was compiled in that
> >> directory... independently of if I changed project.el latter and
> >> compiled in build2. Latter actions in source + compilation in build2
> >> should not affect the consistency of build1.
> >
> >You are talking about modifying project.el while it is being used by a
> >running Emacs session?  In that case, all bets are off, I think.
> 
> No I mean having two emacs builds in different subdirectories. For example one compiled before  certain change and the other after 

You are saying that Emacs can load project.elc from a wrong directory?
Any reproducible recipe for that?



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04 11:59                                         ` Eli Zaretskii
@ 2024-04-04 12:34                                           ` Ergus
  2024-04-04 13:02                                             ` Eli Zaretskii
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-04-04 12:34 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Hi-Angel, rms, dmitry, emacs-devel

On Thu, Apr 04, 2024 at 02:59:27PM +0300, Eli Zaretskii wrote:
>> Date: Thu, 04 Apr 2024 11:59:29 +0200
>> From: Ergus <spacibba@aol.com>
>> CC: Hi-Angel@yandex.ru, rms@gnu.org, dmitry@gutov.dev, emacs-devel@gnu.org
>>
>> >> Finally when executing from build1 it may be possible to access the
>> >> version of project.elc used when the project was compiled in that
>> >> directory... independently of if I changed project.el latter and
>> >> compiled in build2. Latter actions in source + compilation in build2
>> >> should not affect the consistency of build1.
>> >
>> >You are talking about modifying project.el while it is being used by a
>> >running Emacs session?  In that case, all bets are off, I think.
>>
>> No I mean having two emacs builds in different subdirectories. For example one compiled before  certain change and the other after
>
>You are saying that Emacs can load project.elc from a wrong directory?
>Any reproducible recipe for that?
>
No Eli:

That's not what I mean.

Ex:

mkdir build1 && cd build1 && ../configure ... && make
cd ..

edit project.el
mkdir build2 && cd build2 && ../configure ... && make
cd ..

build1/src/emacs -Q

This will load the new elc, not the one used when it was compiled in
spite of we have not recompiled in that build dir.






^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04 12:34                                           ` Ergus
@ 2024-04-04 13:02                                             ` Eli Zaretskii
  2024-04-04 14:27                                               ` Ergus
  0 siblings, 1 reply; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-04 13:02 UTC (permalink / raw)
  To: Ergus; +Cc: Hi-Angel, rms, dmitry, emacs-devel

> Date: Thu, 4 Apr 2024 14:34:16 +0200
> From: Ergus <spacibba@aol.com>
> Cc: Hi-Angel@yandex.ru, rms@gnu.org, dmitry@gutov.dev, emacs-devel@gnu.org
> 
> On Thu, Apr 04, 2024 at 02:59:27PM +0300, Eli Zaretskii wrote:
> >
> >You are saying that Emacs can load project.elc from a wrong directory?
> >Any reproducible recipe for that?
> >
> No Eli:
> 
> That's not what I mean.

Actually, yes, that's what you meant.

> mkdir build1 && cd build1 && ../configure ... && make
> cd ..
> 
> edit project.el
> mkdir build2 && cd build2 && ../configure ... && make
> cd ..
> 
> build1/src/emacs -Q
> 
> This will load the new elc, not the one used when it was compiled in
> spite of we have not recompiled in that build dir.

AFAIU, it doesn't load the new project.elc from build2, it loads
project.el (i.e. uncompiled Lisp) from the source tree, where you
modified project.el.

The reason is that Emacs's support for "running uininstalled" assumes
the Lisp files are in the source tree.  IOW, running uninstalled from
a build directory outside of the source tree does not take the *.elc
files from that build directory, as I think you can verify if you look
at the value of load-path.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04 13:02                                             ` Eli Zaretskii
@ 2024-04-04 14:27                                               ` Ergus
  2024-04-04 14:41                                                 ` Eli Zaretskii
  0 siblings, 1 reply; 53+ messages in thread
From: Ergus @ 2024-04-04 14:27 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Hi-Angel, rms, dmitry, emacs-devel

On Thu, Apr 04, 2024 at 04:02:50PM +0300, Eli Zaretskii wrote:
>> Date: Thu, 4 Apr 2024 14:34:16 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: Hi-Angel@yandex.ru, rms@gnu.org, dmitry@gutov.dev, emacs-devel@gnu.org
>>
>> On Thu, Apr 04, 2024 at 02:59:27PM +0300, Eli Zaretskii wrote:
>> >
>> >You are saying that Emacs can load project.elc from a wrong directory?
>> >Any reproducible recipe for that?
>> >
>> No Eli:
>>
>> That's not what I mean.
>
>Actually, yes, that's what you meant.
>
>> mkdir build1 && cd build1 && ../configure ... && make
>> cd ..
>>
>> edit project.el
>> mkdir build2 && cd build2 && ../configure ... && make
>> cd ..
>>
>> build1/src/emacs -Q
>>
>> This will load the new elc, not the one used when it was compiled in
>> spite of we have not recompiled in that build dir.
>
>AFAIU, it doesn't load the new project.elc from build2, it loads
>project.el (i.e. uncompiled Lisp) from the source tree, where you
>modified project.el.
>
>The reason is that Emacs's support for "running uininstalled" assumes
>the Lisp files are in the source tree.  IOW, running uninstalled from
>a build directory outside of the source tree does not take the *.elc
>files from that build directory, as I think you can verify if you look
>at the value of load-path.

Exactly, so this is not OOSC and does not behave as expected in
OOSC. Which is what Konstantin pointed out since the beginning.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04 14:27                                               ` Ergus
@ 2024-04-04 14:41                                                 ` Eli Zaretskii
  2024-04-04 18:15                                                   ` Ergus
  2024-04-04 20:16                                                   ` Konstantin Kharlamov
  0 siblings, 2 replies; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-04 14:41 UTC (permalink / raw)
  To: Ergus; +Cc: Hi-Angel, rms, dmitry, emacs-devel

> Date: Thu, 4 Apr 2024 16:27:34 +0200
> From: Ergus <spacibba@aol.com>
> Cc: Hi-Angel@yandex.ru, rms@gnu.org, dmitry@gutov.dev, emacs-devel@gnu.org
> 
> >The reason is that Emacs's support for "running uininstalled" assumes
> >the Lisp files are in the source tree.  IOW, running uninstalled from
> >a build directory outside of the source tree does not take the *.elc
> >files from that build directory, as I think you can verify if you look
> >at the value of load-path.
> 
> Exactly, so this is not OOSC and does not behave as expected in
> OOSC.

No, it isn't.  Not many programs at all support running uninstalled.
Emacs does, but only if you build inside the source.  This is
unrelated to the possibility of out-of-source build, where you are
supposed to say "make install" after you build.

> Which is what Konstantin pointed out since the beginning.

No, he didn't.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04 14:41                                                 ` Eli Zaretskii
@ 2024-04-04 18:15                                                   ` Ergus
  2024-04-04 18:56                                                     ` Eli Zaretskii
  2024-04-04 20:16                                                   ` Konstantin Kharlamov
  1 sibling, 1 reply; 53+ messages in thread
From: Ergus @ 2024-04-04 18:15 UTC (permalink / raw)
  To: emacs-devel, Eli Zaretskii; +Cc: Hi-Angel, rms, dmitry



On April 4, 2024 4:41:58 PM GMT+02:00, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Thu, 4 Apr 2024 16:27:34 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: Hi-Angel@yandex.ru, rms@gnu.org, dmitry@gutov.dev, emacs-devel@gnu.org
>> 
>> >The reason is that Emacs's support for "running uininstalled" assumes
>> >the Lisp files are in the source tree.  IOW, running uninstalled from
>> >a build directory outside of the source tree does not take the *.elc
>> >files from that build directory, as I think you can verify if you look
>> >at the value of load-path.
>> 
>> Exactly, so this is not OOSC and does not behave as expected in
>> OOSC.
>
>No, it isn't.  Not many programs at all support running uninstalled.

All CMake and Meson programs do it by default. Mainly because they also provide test features  where the executables are supposed to be tested before calling install or pack.

>Emacs does, but only if you build inside the source.  This is
>unrelated to the possibility of out-of-source build, where you are
>supposed to say "make install" after you build.
>
>> Which is what Konstantin pointed out since the beginning.
>
>No, he didn't.
>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04 18:15                                                   ` Ergus
@ 2024-04-04 18:56                                                     ` Eli Zaretskii
  0 siblings, 0 replies; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-04 18:56 UTC (permalink / raw)
  To: Ergus; +Cc: emacs-devel, Hi-Angel, rms, dmitry

> Date: Thu, 04 Apr 2024 20:15:56 +0200
> From: Ergus <spacibba@aol.com>
> CC: Hi-Angel@yandex.ru, rms@gnu.org, dmitry@gutov.dev
> 
> >> Exactly, so this is not OOSC and does not behave as expected in
> >> OOSC.
> >
> >No, it isn't.  Not many programs at all support running uninstalled.
> 
> All CMake and Meson programs do it by default. Mainly because they also provide test features  where the executables are supposed to be tested before calling install or pack.

I don't see how CMake or Meson can make a program run uninstalled,
unless the program already provides the necessary features for doing
so.  Imagine a program that needs to read some data files, which it
expects to find in a directory under /usr/share whose absolute file
name is hard-coded into it, as determined from --prefix at configure
time.  Such a program cannot run uninstalled, because the files in
/usr/share are not yet installed.

Emacs needs to know where to find the Lisp files it loads on demand.
If those files are in the place configured via --prefix, they can be
found easily, but if Emacs runs uninstalled, the Lisp files are not
there, and Emacs needs to find them somehow.  Unless someone writes
code to look for them in the right places, there's no way CMake or
Meson or any other general-purpose tool could fix that.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04 14:41                                                 ` Eli Zaretskii
  2024-04-04 18:15                                                   ` Ergus
@ 2024-04-04 20:16                                                   ` Konstantin Kharlamov
  2024-04-05  5:11                                                     ` Eli Zaretskii
  1 sibling, 1 reply; 53+ messages in thread
From: Konstantin Kharlamov @ 2024-04-04 20:16 UTC (permalink / raw)
  To: Eli Zaretskii, Ergus; +Cc: rms, dmitry, emacs-devel

On Thu, 2024-04-04 at 17:41 +0300, Eli Zaretskii wrote:
> > Date: Thu, 4 Apr 2024 16:27:34 +0200
> > From: Ergus <spacibba@aol.com>
> > Cc: Hi-Angel@yandex.ru, rms@gnu.org, dmitry@gutov.dev,
> > emacs-devel@gnu.org
> > 
> > > The reason is that Emacs's support for "running uininstalled"
> > > assumes
> > > the Lisp files are in the source tree.  IOW, running uninstalled
> > > from
> > > a build directory outside of the source tree does not take the
> > > *.elc
> > > files from that build directory, as I think you can verify if you
> > > look
> > > at the value of load-path.
> > 
> > Exactly, so this is not OOSC and does not behave as expected in
> > OOSC.
> 
> No, it isn't.  Not many programs at all support running uninstalled.
> Emacs does, but only if you build inside the source.  This is
> unrelated to the possibility of out-of-source build, where you are
> supposed to say "make install" after you build.

Indeed running uninstalled is kind of orthogonal to OOSC and may or may
not be supported by a program.  I think this discussion might be
confusing two orthogonal things: α) `.elc` files being put in-tree, and
β) Emacs requiring for them to be put in-tree to run "uninstalled".

For β: I'm sure it's possible to modify Emacs to load `.elc` from the
build dir rather than from the tree.

For α: the lone fact that build artifacts are being put in-tree means
OOSC is not working as expected.

I see what you mean by saying that `configure` and similar files can't
be put into the build dir because that's how autotools work.  Okay. 
But I think this whole discussion about the terminology is becoming not
very useful.  Can we please agree that generating .elc files in the
build dir rather than in tree allows to avoid a number of problems? 
Please note that this has nothing to do with `.elc` files being put in
tarballs, because for that to work you could just build "in-tree" as
usual.  Putting `.elc` files into build dir (which may be in-tree or
out) and allowing Emacs to take those into account is just
simplification in how things work.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-04 20:16                                                   ` Konstantin Kharlamov
@ 2024-04-05  5:11                                                     ` Eli Zaretskii
  0 siblings, 0 replies; 53+ messages in thread
From: Eli Zaretskii @ 2024-04-05  5:11 UTC (permalink / raw)
  To: Konstantin Kharlamov; +Cc: spacibba, rms, dmitry, emacs-devel

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Cc: rms@gnu.org, dmitry@gutov.dev, emacs-devel@gnu.org
> Date: Thu, 04 Apr 2024 23:16:29 +0300
> 
> Can we please agree that generating .elc files in the
> build dir rather than in tree allows to avoid a number of problems? 
> Please note that this has nothing to do with `.elc` files being put in
> tarballs, because for that to work you could just build "in-tree" as
> usual.  Putting `.elc` files into build dir (which may be in-tree or
> out) and allowing Emacs to take those into account is just
> simplification in how things work.

As I wrote up-thread, I have no objections for adding this to Emacs,
but it is not as trivial as it seems, because Emacs needs more files
than just *.elc to run, and some of those other files are in the
source tree when Emacs runs uninstalled (example: the images in
etc/images).  So adding this mode of running Emacs would need some
changes to the C startup code, not just to Makefile's.  Also, if you
edit *.el files in the source directory, the *.eln files produced from
the *.el files before the edits will no longer be loaded by Emacs
(because they don't match the corresponding *.el files); if we want to
fix that as well, we'd need much more serious changes.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-02 22:43                 ` Dr. Arne Babenhauserheide
@ 2024-04-05 21:40                   ` Richard Stallman
  0 siblings, 0 replies; 53+ messages in thread
From: Richard Stallman @ 2024-04-05 21:40 UTC (permalink / raw)
  To: Dr. Arne Babenhauserheide; +Cc: spacibba, dmitry, emacs-devel

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > What does "Project out of sources compilation" mean?
  > > I am not sure how to parse that, let alone determine what it signfies.

  > That usually means that the compilation happens in a directory which is
  > not the source directory (similar to how make distcheck does it).

Thanks.  I've been building Emacs that way for half my life,
but I could not tell that the phrase should be parsed that way.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)





^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-02 23:23                   ` Dmitry Gutov
  2024-04-03 19:47                     ` Ergus
@ 2024-04-06  2:05                     ` Ergus
  2024-04-14  1:44                       ` Dmitry Gutov
  1 sibling, 1 reply; 53+ messages in thread
From: Ergus @ 2024-04-06  2:05 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 791 bytes --]

Hi Dmitry:

Here I attach a new patch with what I think is the best approach to
handle the extra info for projects. This implementation is pretty simple
for both sides (user, project.el and backends) and uses your main idea
of the generics + a custom.

However I decided not to use only the custom as you suggested, because
considering the project initialization it will require a complex and
error prone approach to initalize the backend variables correctly (for
example, when some buffers are already open).

With this implementation the backends only need to define the optional
generic specializations and the policy is pretty simple to explain (see
the doc strings).

I also modifies the backend project on github to match this new api, so,
you can check it as well.

WDYT?
Best,
Ergus

[-- Attachment #2: project-extra-info.patch --]
[-- Type: text/plain, Size: 5481 bytes --]

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index da211566a3b..1986c14e861 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -51,6 +51,16 @@
 ;; files inside the root must not be considered a part of it).  It
 ;; should be consistent with `project-files'.
 ;;
+;; `project-extra-info' is a user custom variable and a method to
+;; specify extra project information like build command or equivalent.
+;; The variable is a plist were the values can be strings or functions
+;; that receive the project-current as argument.  The
+;; `project-extra-info' expect project current as first parameter and an
+;; extra parameter equivalent to the plist key.  The user defined plist
+;; takes precedence over the backend defined methods, but all the
+;; specializations are optional and the functions calling them may
+;; provide conditions in case both are undefined.
+;;
 ;; This list can change in future versions.
 ;;
 ;; Transient project:
@@ -229,8 +239,8 @@ project-current
 of the project instance object."
   (unless directory (setq directory (or project-current-directory-override
                                         default-directory)))
-  (let* ((non-essential (not maybe-prompt))
-         (pr (project--find-in-directory directory)))
+  (let ((pr (project--find-in-directory directory))
+        (non-essential (not maybe-prompt)))
     (cond
      (pr)
      ((unless project-current-directory-override
@@ -290,8 +300,54 @@ project-external-roots
 headers search path, load path, class path, and so on."
   nil)
 
+(defcustom project-extra-info nil
+  "Project extra info defined in user space.
+
+This is intended to be set by the user.  This is expected to be a plist
+with key entries for compile.  At the moment the implemented keys are
+`:compile-command' and `:compile-dir'.  The entries may be either string
+constants, paths or functions.  This custom has a symmetric generic
+method with the same name that are intended to be implemented by the
+project backends.  When this variable is defined it takes precedence
+over the backend methods."
+  :safe t
+  :version "30.1"
+  :type '(plist :key-type (choice (const :compile-dir)
+                                  (const :compile-command))
+                :value-type (choice string
+                                    directory
+                                    function)))
+
+(cl-defgeneric project-extra-info (_project _info)
+  "Return extra INFO for the current PROJECT.
+
+This function is intended to be defined by the backend when needed.
+Otherwise this returns nil and the `project-compile' command will use
+some default values.  The current valid values for INFO are the same key
+types in project-compile-info: `:compile-dir' and `:compile-command'
+This method is independent from the custom variable with same name
+because project.el initializes itself lazily and variable propagation
+within directories and buffers already open will require too much work
+in the user side potentially more error prone."
+  nil)
+
+(defun project--get-extra-info (project info)
+  "Steps to get PROJECT's INFO internally.
+1. Parse the user defined variable `project-compile-info'.  If the key
+exists:
+   a. Check if it is a function and call it passing project as the first
+parameter.
+   b. If the key is a string return it as is.
+   c. Otherwise return nil.
+2. Else call the backend defined method `project-compile-info'."
+  (if-let ((value (plist-get project-extra-info info)))
+      (cond ((functionp value) (funcall value project))
+            ((stringp info) info)
+            (t nil))
+    (project-extra-info project info)))
+
 (cl-defgeneric project-name (project)
-  "A human-readable name for the project.
+  "A human-readable name for the PROJECT.
 Nominally unique, but not enforced."
   (file-name-nondirectory (directory-file-name (project-root project))))
 
@@ -1389,13 +1445,26 @@ project-compilation-buffer-name-function
 
 ;;;###autoload
 (defun project-compile ()
-  "Run `compile' in the project root."
+  "Run `compile' in the project build directory.
+When the variable `project-extra-info' contains the entries
+`:compile-dir' or `:compile-command' or the project backend specializes
+the method `project-extra-info' for those values; then this command uses
+that instead of the default: `project-root' and `compile-command'."
   (declare (interactive-only compile))
   (interactive)
-  (let ((default-directory (project-root (project-current t)))
-        (compilation-buffer-name-function
-         (or project-compilation-buffer-name-function
-             compilation-buffer-name-function)))
+  ;; I am wondering whenever we need to expand connection local
+  ;; variables at this point... maybe before or inside the let.
+  (let* ((project (project-current t))
+         (dir (or (project--get-extra-info project :compile-dir)
+                  (project-root project)))
+         (default-directory (if (file-name-absolute-p dir)
+                                dir
+                              (expand-file-name dir (project-root project))))
+         (compile-command (or (project--get-extra-info project :compile-command)
+                              compile-command))
+         (compilation-buffer-name-function
+          (or project-compilation-buffer-name-function
+              compilation-buffer-name-function)))
     (call-interactively #'compile)))
 
 (defun project-recompile (&optional edit-command)

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-06  2:05                     ` Ergus
@ 2024-04-14  1:44                       ` Dmitry Gutov
  2024-04-16 14:56                         ` Ergus
  2024-04-22 17:05                         ` Ergus
  0 siblings, 2 replies; 53+ messages in thread
From: Dmitry Gutov @ 2024-04-14  1:44 UTC (permalink / raw)
  To: Ergus; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1020 bytes --]

Hi Ergus!

On 06/04/2024 05:05, Ergus wrote:

> Here I attach a new patch with what I think is the best approach to
> handle the extra info for projects. This implementation is pretty simple
> for both sides (user, project.el and backends) and uses your main idea
> of the generics + a custom.
> 
> However I decided not to use only the custom as you suggested, because
> considering the project initialization it will require a complex and
> error prone approach to initalize the backend variables correctly (for
> example, when some buffers are already open).
> 
> With this implementation the backends only need to define the optional
> generic specializations and the policy is pretty simple to explain (see
> the doc strings).
> 
> I also modifies the backend project on github to match this new api, so,
> you can check it as well.

How about the idea with the orthogonal hook, then, as in the attached 
patch? Something I described before.

I think your github code would be easy enough to adapt to use it.

WDYT?

[-- Attachment #2: project-build-functions.diff --]
[-- Type: text/x-patch, Size: 1996 bytes --]

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 000a05804a8..bd1264b5525 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -1387,15 +1387,33 @@ project-compilation-buffer-name-function
                         project-prefixed-buffer-name)
                  (function :tag "Custom function")))
 
+(defvar project-build-functions nil
+  "Special hook to find the project build configuration.
+Each functions on this hook is called in turn with one
+argument (project), and should return either nil to mean that it is not
+applicable, or a plist with build configuration.
+
+The configuration can contain keys such as `:compile-dir',
+`:compile-command', or others.")
+
 ;;;###autoload
 (defun project-compile ()
-  "Run `compile' in the project root."
+  "Run `compile' in the project build directory.
+Uses the `project-build-functions' hook to determine the compile command
+and the directory to run it in."
   (declare (interactive-only compile))
   (interactive)
-  (let ((default-directory (project-root (project-current t)))
-        (compilation-buffer-name-function
-         (or project-compilation-buffer-name-function
-             compilation-buffer-name-function)))
+  (let* ((project (project-current t))
+         (build-config (run-hook-with-args 'project-build-functions project))
+         (default-directory (if (plist-get build-config :compile-dir)
+                                (expand-file-name (plist-get build-config :compile-dir)
+                                                  (project-root project))
+                              default-directory))
+         (compile-command (or (plist-get build-config :compile-command)
+                              compile-command))
+         (compilation-buffer-name-function
+          (or project-compilation-buffer-name-function
+              compilation-buffer-name-function)))
     (call-interactively #'compile)))
 
 (defun project-recompile (&optional edit-command)

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-14  1:44                       ` Dmitry Gutov
@ 2024-04-16 14:56                         ` Ergus
  2024-04-22 17:05                         ` Ergus
  1 sibling, 0 replies; 53+ messages in thread
From: Ergus @ 2024-04-16 14:56 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

Hi Dmitry:

Sorry for th delay with this

On Sun, Apr 14, 2024 at 04:44:36AM +0300, Dmitry Gutov wrote:
>Hi Ergus!
>
>On 06/04/2024 05:05, Ergus wrote:
>
>>Here I attach a new patch with what I think is the best approach to
>>handle the extra info for projects. This implementation is pretty simple
>>for both sides (user, project.el and backends) and uses your main idea
>>of the generics + a custom.
>>
>>However I decided not to use only the custom as you suggested, because
>>considering the project initialization it will require a complex and
>>error prone approach to initalize the backend variables correctly (for
>>example, when some buffers are already open).
>>
>>With this implementation the backends only need to define the optional
>>generic specializations and the policy is pretty simple to explain (see
>>the doc strings).
>>
>>I also modifies the backend project on github to match this new api, so,
>>you can check it as well.
>
>How about the idea with the orthogonal hook, then, as in the attached 
>patch? Something I described before.
>
>I think your github code would be easy enough to adapt to use it.
>
>WDYT?

The main idea is somehow the same, but IMHO I think that this approach
complicates the user config and backend implementation being also less
efficient and scalable (we will need a new hook for every new similar
feature).

There is no need to separate the backend from the build, and after some
tests, I find the cl-defgeneric behavior more predictable and simpler to
debug than using a function hook.

Your initial idea implemented like 

(cl-defgeneric project-extra-info (_project _info)
   ...)

In my opinion is simpler and more scalable (while could be recycled to
provide also other existing functionalities like
project-compilation-buffer-name-function. All the generic
implementations become then optional which is very desired for the
general purpose.



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-14  1:44                       ` Dmitry Gutov
  2024-04-16 14:56                         ` Ergus
@ 2024-04-22 17:05                         ` Ergus
  2024-04-22 18:48                           ` Ergus
  1 sibling, 1 reply; 53+ messages in thread
From: Ergus @ 2024-04-22 17:05 UTC (permalink / raw)
  To: Dmitry Gutov; +Cc: emacs-devel

Hi Juri:

Give a look to this:

https://github.com/mohkale/projection

It seems similar to what we may consider to support in vanilla. OR at
least provide the interactive interface wrapper functions in order to
make project.el a bit more standard.

On Sun, Apr 14, 2024 at 04:44:36AM GMT, Dmitry Gutov wrote:
>Hi Ergus!
>
>On 06/04/2024 05:05, Ergus wrote:
>
>>Here I attach a new patch with what I think is the best approach to
>>handle the extra info for projects. This implementation is pretty simple
>>for both sides (user, project.el and backends) and uses your main idea
>>of the generics + a custom.
>>
>>However I decided not to use only the custom as you suggested, because
>>considering the project initialization it will require a complex and
>>error prone approach to initalize the backend variables correctly (for
>>example, when some buffers are already open).
>>
>>With this implementation the backends only need to define the optional
>>generic specializations and the policy is pretty simple to explain (see
>>the doc strings).
>>
>>I also modifies the backend project on github to match this new api, so,
>>you can check it as well.
>
>How about the idea with the orthogonal hook, then, as in the attached 
>patch? Something I described before.
>
>I think your github code would be easy enough to adapt to use it.
>
>WDYT?





^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH] Project out of sources compilation
  2024-04-22 17:05                         ` Ergus
@ 2024-04-22 18:48                           ` Ergus
  0 siblings, 0 replies; 53+ messages in thread
From: Ergus @ 2024-04-22 18:48 UTC (permalink / raw)
  To: mohkale; +Cc: emacs-devel

Adding Mohsin Kaleem to the discussion.


On Mon, Apr 22, 2024 at 07:05:20PM GMT, Ergus wrote:
>Hi Juri:
>
>Give a look to this:
>
>https://github.com/mohkale/projection
>
>It seems similar to what we may consider to support in vanilla. OR at
>least provide the interactive interface wrapper functions in order to
>make project.el a bit more standard.
>
>On Sun, Apr 14, 2024 at 04:44:36AM GMT, Dmitry Gutov wrote:
>>Hi Ergus!
>>
>>On 06/04/2024 05:05, Ergus wrote:
>>
>>>Here I attach a new patch with what I think is the best approach to
>>>handle the extra info for projects. This implementation is pretty simple
>>>for both sides (user, project.el and backends) and uses your main idea
>>>of the generics + a custom.
>>>
>>>However I decided not to use only the custom as you suggested, because
>>>considering the project initialization it will require a complex and
>>>error prone approach to initalize the backend variables correctly (for
>>>example, when some buffers are already open).
>>>
>>>With this implementation the backends only need to define the optional
>>>generic specializations and the policy is pretty simple to explain (see
>>>the doc strings).
>>>
>>>I also modifies the backend project on github to match this new api, so,
>>>you can check it as well.
>>
>>How about the idea with the orthogonal hook, then, as in the 
>>attached patch? Something I described before.
>>
>>I think your github code would be easy enough to adapt to use it.
>>
>>WDYT?
>
>
>



^ permalink raw reply	[flat|nested] 53+ messages in thread

end of thread, other threads:[~2024-04-22 18:48 UTC | newest]

Thread overview: 53+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <4wwljrdnra3bsloehioa46y24ozxajajmvf2elvskxxq3mhtg2.ref@pyv2z5snot6h>
2024-03-16 13:12 ` Project out of sources compilation Ergus
2024-03-16 16:50   ` Konstantin Kharlamov
2024-03-16 19:00     ` Ergus
2024-03-16 20:56       ` Konstantin Kharlamov
2024-03-17  2:53   ` Dmitry Gutov
2024-03-17  7:22     ` Ergus
2024-03-17  8:45       ` Eli Zaretskii
2024-03-17 17:33         ` Ergus
2024-03-17 17:38           ` Eli Zaretskii
2024-03-17 17:58             ` Ergus
2024-03-17 11:36   ` Augusto Stoffel
2024-03-17 17:47     ` Ergus
2024-03-19 18:36       ` Ergus
2024-03-27 16:38         ` [PATCH] " Ergus
2024-03-31  2:41           ` Dmitry Gutov
2024-03-31 21:07             ` Ergus
2024-04-01  7:49               ` Dirk-Jan C. Binnema
2024-04-01 13:52                 ` Ergus
2024-04-01 15:09                   ` Dirk-Jan C. Binnema
2024-04-01 17:18                     ` Ergus
2024-04-02 23:23                   ` Dmitry Gutov
2024-04-03 19:47                     ` Ergus
2024-04-06  2:05                     ` Ergus
2024-04-14  1:44                       ` Dmitry Gutov
2024-04-16 14:56                         ` Ergus
2024-04-22 17:05                         ` Ergus
2024-04-22 18:48                           ` Ergus
2024-04-02 21:39               ` Richard Stallman
2024-04-02 22:43                 ` Dr. Arne Babenhauserheide
2024-04-05 21:40                   ` Richard Stallman
2024-04-03 10:40                 ` Konstantin Kharlamov
2024-04-03 11:45                   ` Eli Zaretskii
2024-04-03 13:31                     ` Konstantin Kharlamov
2024-04-03 14:11                       ` Eli Zaretskii
2024-04-03 15:00                         ` Konstantin Kharlamov
2024-04-03 15:47                           ` Eli Zaretskii
2024-04-03 17:27                             ` Konstantin Kharlamov
2024-04-03 18:22                               ` Eli Zaretskii
2024-04-03 19:08                                 ` Konstantin Kharlamov
2024-04-03 20:12                                   ` Ergus
2024-04-04  5:26                                     ` Eli Zaretskii
2024-04-04  9:59                                       ` Ergus
2024-04-04 11:59                                         ` Eli Zaretskii
2024-04-04 12:34                                           ` Ergus
2024-04-04 13:02                                             ` Eli Zaretskii
2024-04-04 14:27                                               ` Ergus
2024-04-04 14:41                                                 ` Eli Zaretskii
2024-04-04 18:15                                                   ` Ergus
2024-04-04 18:56                                                     ` Eli Zaretskii
2024-04-04 20:16                                                   ` Konstantin Kharlamov
2024-04-05  5:11                                                     ` Eli Zaretskii
2024-04-04  5:07                                   ` Eli Zaretskii
     [not found]                               ` <87jzlefgi9.fsf@dick>
2024-04-03 18:44                                 ` Konstantin Kharlamov

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.