Table of Contents ───────────────── 1. The problem it tries to solve 2. How it works .. 1. Build system detection .. 2. Configure and compile .. 3. Project only or not? .. 4. Command formatting .. 5. `function-modification' .. 6. Configuration 3. Open questions .. 1. Local configuration .. 2. Build directory name .. 3. Build system detection .. 4. Tramp support .. 5. Flexibility .. 6. Adding a build system .. 7. Other 4. Conclusion Hello, I would like to propose a new package to integrate in Emacs, "Builder". It provides integration with build systems, to ease the configuration and the compilation process, by automatically detecting the build system and proposing relevant commands. Big disclaimer: I do not know many build systems, I do not have a lot of experience in programming, I do not know (E)lisp very well, and English is not my first language. Now that you are definitely sold on it, let's have a deeper explanation! As far as I know, Emacs does not provide a convenient way to use build systems, like CMake, Meson, Autotools, Cargo, Dune, and so on. I know your can configure the compilation command with local variables, but: • You cannot have a choice, like `cmake .. -DCMAKE_BUILD_TYPE=Release' or `cmake .. -DCMAKE_BUILD_TYPE=Debug' • It requires to set it up manually • There is no (convenient?) way to have a configuration phase and a compilation phase. (like `cmake ..' followed by `make') With Builder, you just run `builder-configure' which set up the build system, and `builder-compile' to compile the project. `builder-configure' allow you to easily switch between release and debug. You can try it with the `builder-demo/' directory found inside this git repository: There are more explanations in the README.org. 1 The problem it tries to solve ═══════════════════════════════ Builder tries to solve two issues at the same time: being convenient and being easy to configure. Independently they are not *that* difficult to solve, but together it gets really complex. For example, some build systems need to run their command /outside/ the build directory, while others need to run it /inside/. Also, some commands need to be modified according to the state of the project: Meson needs the flag `--reconfigure' if it was already configured. 2 How it works ══════════════ 2.1 Build system detection ────────────────────────── The first step is the detection of the build system. It simply checks in a list of files which one exists and it returns a list of build system identifiers. For instance if both CMake and Meson are available, it returns the list `("cmake" "meson")'. Notice that the detection is completely separated from build systems addition. This is a deliberate design choice to make it more versatile, for example to filter results or have a cache system (both not implemented). 2.2 Configure and compile ───────────────────────── The configuration is a first step for some build systems (Autotools, CMake, Meson)… Indeed they are not actually build systems, but build system /generator/ (not that it really matters). `builder-configure' and `builder-compile' respectively provide, well, configuration and compilation. They both do what you would expect by proposing relevant commands and running the selected one at the right place. 2.3 Project only or not? ──────────────────────── You will quickly notice that many functions have the argument `variable'. I am not really sure about this one, but it would allow to use a build system not located at the root of a project. I guess it is useful for monorepo projects? I did minimal testing on it and it seems to works. 2.4 Command formatting ────────────────────── Commands pass though the function `builder-format-command' before being run. It uses the function `format-spec` with predefined specifiers, such as `%n` for the number of cores, `%b` for the build directory name… For example "mycommand -proc %n –dir %b" becomes "mycommand -proc 4 –dir build". 2.5 `function-modification' ─────────────────────────── A pretty neat feature is `function-modification': before running, the command is modified by a function. Its power is showcased with the Dune build system and CMake Presets, because you can chose a specific target from a list, and the command is modified as needed! Some details need to be investigated though: should the command pass through `builder-format-command' before being send to the function? After? Or let the function take care of it? 2.6 Configuration ───────────────── The configuration of a build system is hierarchical: at toplevel, it affects every instruction from the build system, but you can configure each instruction independently. For example, you don't need to specify manually that each CMake command need to run inside the directory, but you can still customize a specific command. In the following example, both "debug" and "fast" commands are executed inside the build directory, and "release" is executed at the root of the project. ┌──── │ (builder-add-build-system │ :build-system-id "gcc" │ :compile '(:inside-directory t ;; Affects all instructions │ :instructions │ ((:name "debug" │ :command "gcc main.c -g") │ (:name "release" │ :command "gcc main.c -O2" │ ;; Overrides the higher level :inside-directory │ :inside-directory nil) │ (:name "fast" │ :command "gcc main.c -Ofast")))) └──── 3 Open questions ════════════════ This is a list of problems I don't have a definitive answer to. Suggestions are most welcome. 3.1 Local configuration ─────────────────────── It's necessary to have a way to locally configure a project to add commands, specify the build directory… But I don't really know how to do that. Should it affect the whole project? Only a specific directory (e.g. only `project-root/subdir')? I know that local variables are a thing in Emacs, but they are not a good solution for this since they are actually *buffer* variables. So changing the local configuration would require to reload each local variable even unrelated one that could have been modified on a per buffer basis. Also, the commands cannot be considered as safe, so opening a file trigger a warning each time (unless you mark the values as safe). It would be a good idea to discuss with the Projectile package devs about it, to agree on a common solution if they wish. I have implemented an alternative, dir-val.el. The variables are stored in a tree, each node corresponding to a directory. The variables are "hierarchical". If you have a directory `/dirA/dirB/dirC' and you set a variable to `dirB', then `dirC' will also access it. But you can also change the value only for `dirC'. I'm wondering if this is actually useful. Would project variables be enough? 3.2 Build directory name ──────────────────────── This is harder to solve than it seems. I would like to have a mechanism to change it easily, so you can have `build-release' and `build-debug' directories, allowing you to conveniently switch between release and debug without having to recompile the whole project each time you switch. BUT: some tools have specific expectations about this name. `clangd' expects it to be named `build', and I'm pretty sure that the OCaml ecosystem expect it to be named `_build'. Also, how to know which directory to use to execute a command, as "configure" and "compile" may have different names? With a cache? With a regex to select candidates? Local configuration only? I really would like to have this feature implemented. Currently you can chose to modify the build directory name according to the current git branch (which is pretty reliable), but it would be really useful to have `build-release' and `build-debug' directories. 3.3 Build system detection ────────────────────────── Currently there are few build systems added to the list. But there could be hundreds. How well would it scale? Especially on slow file systems? 3.4 Tramp support ───────────────── I don't use Tramp and have never used it. I don't think it would be too hard to support it, but the detection could be a problem if there are too many build systems. 3.5 Flexibility ─────────────── Is Builder good enough to support every build system? To accommodate most use cases? As I said I don't have a lot of experience in build systems, so suggestions are welcome. 3.6 Adding a build system ───────────────────────── A build system is added with `builder-add-build-system'. Is it good enough? I fear it is a bit too error prone because it's easy to mess with parenthesis and keywords location. How to solve that? With a macro? EIEIO? 3.7 Other ───────── Some other are listed in the README and there are probably many other issues I did not think about. 4 Conclusion ════════════ I hope you will find this package useful. Please tell me if this is unclear, needs more documentation, or if you have issues. Currently only a few build systems are supported. You can already use it to build Emacs with the build system `autotools'. `builder-configure' and `builder-compile' are respectively bind to `C-x p C' and `C-x p c'. `builder-configure' will create a directory named "build" and run `../configure' from it, and `builder-compile' will run `make -jN' with N the number of cores on your processor (the result of `num-processor'). I think this package is far from ready, but I need to know if the design is sound enough and future proof. Obviously I will need to sign the copyright assignment.