From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ergus Newsgroups: gmane.emacs.devel Subject: Re: [PATCH] Project out of sources compilation Date: Tue, 23 Apr 2024 17:17:51 +0200 Message-ID: <4e5bg4tmzu3pl334wiw5fei5s5uxbqnnoio3avqou5sde3dodk@3bewafrmdzus> References: <1fd527fc-9643-49d2-8fae-d7e7fd043fe1@gutov.dev> <87le5x34l6.fsf@gmail.com> <27rton4k4r6sacysluk7iikj57ai2tyiak4ldd5nzpts7thmhg@nriej75catir> <09a8189d-07e3-4bc5-a4f4-127dcdcec2d1@gutov.dev> <2bpiuffyzhqvirscyn5prs4rb6m5ito7xnwqvc53obm32isp3g@w73yut5uku4y> <87il09hz6n.fsf@kisara.moe> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="25637"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Mohsin Kaleem Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Apr 23 17:19:15 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rzHvO-0006RX-TX for ged-emacs-devel@m.gmane-mx.org; Tue, 23 Apr 2024 17:19:15 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rzHuQ-0000PO-Qx; Tue, 23 Apr 2024 11:18:14 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rzHuK-0000DH-9I for emacs-devel@gnu.org; Tue, 23 Apr 2024 11:18:10 -0400 Original-Received: from sonic314-13.consmr.mail.bf2.yahoo.com ([74.6.132.123]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rzHuE-0001kO-0L for emacs-devel@gnu.org; Tue, 23 Apr 2024 11:18:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aol.com; s=a2048; t=1713885477; bh=m/D+hpKyM6+JN4ObCh+TJS7j53g4qsBUhVZ7kuzHQoc=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From:Subject:Reply-To; b=ebUmeLhWu6ZNJkjKoeErzNB/3DWwLTLkQ/N+ug/rAE/HvXwuv5ZHpNHautzczkX/msrbXJBqid8KrJ65VOvauGtRJmtk7nfZ4aw2eASCM4DCxOeaXkvpIYWNlQ6LvEqcTHTxWKJmG00dOCBDB16mDznKantKhx8Fz4aVeG31HDM5xFEGdtmi/ltTce1XLPfwXepGEERuOr9hHi6rOyH9NDA7qLQrvFobocLPZbReMeCTdJfjikxKyglWnx0gQ9I7GvBM/JsmVJNC3vZOPhbD18EkvK6zkc/thxVCg/KfXrRGzuZCEb31gq7t95VY0bBlHB9l8O2Q0n+S+UAbXbDyhA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1713885477; bh=xP/X1j35JH3+xjq/EN1REMlvcqQrlvVMiRHSf2XVCYX=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=kGuxNk/qyK38eC099QdIetDIooHqzl46PTh/AtMo6+cDUISHmBnXo53RtwyMJxEUKlOsRuCg5ftSYH0d/Sbk64WbBIUssHWSZvGcSqM1PRm4l8H1qv14lMZhcWyjKrq1MOlfzSnsIzmRih0rT3bVdk1UmwfYOT7WNMKUjQCH4l/q0fJGPaXoEEMbKMfzH0YdDZ6WtfXlygU2YUjZ5nUzdyAhOB6wNOmdU0Wkeq7ODcQzq4FHvKWWI6Lqn77ydM4rcFDkMnvIALaaRdHfXZ3CYy7RX/PUHSM/sX1Tpy56dfjobcfc5k6mkgxB317HSJkSZXoEzEMizEmM03+Be2zbhg== X-YMail-OSG: o00jDa4VM1lg0lotGGHEForOUrjyifS22ZhgActjc.VfYbeGQqRj0tDM0aTJ0kG Obw1HiLlMcAFk5Vb4w90SbSBwP7bhnxkD84xtheFiGoQBLZDl29GfoMSmTA0uk22tTR.Tg_L7_Kw Sbg6199fy6PGqe_CmPph2g88Fhze7pQuWOvbxhEvxSxDPWFurUyHvrCO4pi7W3gIXPUaTe6rBoUF 3WugcXDPEkGTMZgZG4Wn0SmYse0YuqBb88MUXCyhpE8i9xBEGMLISfX0E8i5h4oTQlQkj2lT47nm izJkcqRwItLmart0QqOKpDpgx_W4z5yUtGp5LDK1fCdtkqZgTljXEc65YIkgGwqIPB28vtVdqIBl _.Qx0BLA0_9qEg6rbKgc58Wed4gDa3auzrTnwHuIfqDUrfzTocQldP_u9e2gu1GKfkgL43bANa_q PIo0cHPp6h0ZY8Zphgkl5LQ4zVCtaK1T28xMKbywELme2OpQfq5WN0GYgOHRJ6ofiAlHLh9OjH2Q ndon23SOhJM.kRZwClg3JKHg.OEjbUwIZ1cViXPG640.EtU7IEMdEZJ5vSKRxRuDyUwMz7hZhsD_ kTt_YHURgkIGra_yceUlgqiU6UxJospUqNtwrHFeY34jGol0KuOip5pBoea1UUMl37uGVtEpvML. iurcAXbJ1uJBOs34hjNfax97IT6b.cHSROFsb.FvtGXbxTKwM5W2ndCayvP0H8o3eJRLb4Lg.WOZ tgo4GwLL0rJgrnfeU7uM3i7Y8P_Yy2Iad3cMuktQnuTo8X997KnbU72HzexJ68f_yB0SbECW_tF8 eS.SukjkDzj7tgiTcvoW7M_Z3rzm1DhRX4M9Aof9sc X-Sonic-MF: X-Sonic-ID: 9b2e0db4-fa81-45d7-b569-3ff59395a6ea Original-Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.bf2.yahoo.com with HTTP; Tue, 23 Apr 2024 15:17:57 +0000 Original-Received: by hermes--production-ir2-7b99fc9bb6-dfpgm (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 3cfbe21333157547258aa4e0f99775f5; Tue, 23 Apr 2024 15:17:52 +0000 (UTC) Content-Disposition: inline In-Reply-To: <87il09hz6n.fsf@kisara.moe> X-Mailer: WebService/1.1.22256 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.aol Received-SPF: pass client-ip=74.6.132.123; envelope-from=spacibba@aol.com; helo=sonic314-13.consmr.mail.bf2.yahoo.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:318006 Archived-At: Hi: On Mon, Apr 22, 2024 at 10:20:32PM GMT, Mohsin Kaleem wrote: >Ergus writes: > >Howdie howdie, > >I'm the maintainer of projection. Looks like it's a similar external >package to what we're trying to formalize in Emacs core project.el . > >I'm just going to highlight 2 things which I think would help you Ergus. >Ninja multi-config generators [1] which basically lets you configure a >project for Debug, Release, ASAN, UBSAN, etc. simultaneously. And CMake >presets [2] which is a standard for loading common configuration in CMake >like build directories but also compiler flags, environment variables, >etc. > >[1]: https://cmake.org/cmake/help/latest/generator/Ninja%20Multi-Config.html >[2]: https://cmake.org/cmake/help/latest/manual/cmake-presets.7.html > I'll give it a look. I am aware of cmake presets, but we don't even support cmake or out of sources compilation yet, so I tink that supporting the most basic cmake setup will be already a step forward. >I'll add my 2-cents on some of the discussed points. > >Firstly, I disagree with the goal of making the build directory option >customizable in project.el itself. Most build tools I've encountered >(including all supported by projection) take an extra flag to point to >the build directory and I find this a nicer UX because it's easily >inspectable and editable with `C-u M-x recompile`. I noticed projectile >supported this feature of "run somewhere else" as well but it was only >for one project type "meson" and even that has this option to move >directories so I don't really see the value of complicating the >interface to support it. Another pain point with a generalised >"build-directory" option is that it's not generic for all possible >commands. Configuration often runs before the build directory exists >(because it creates it) so then you need an exemption for one command >type but not another. I think the project-root as a standard location >for commands to run is a good common default and changing it unless >absolutely necessary is un-needed :-). > Yes, that's actually the current approach in my POC code. after some fight with rust projects. However, the difference on moving or not is so simple that it doesn't even represent a difficulty. >Secondly I agree with Dmitry about defining an alternate orthoganal API >to project.el for this. It probably belongs more in a project-types.el >package. The motivation being that project.el is for more bare-bones >mechanics, things like "am I in a project", "where is the root of the >project", "show me all the files in this project". Project types is more >"how do I build this project", "how do I test it", "what artifacts does it >produce". The former is a set of functionalities which is relatively >common and static across all projects, the latter is very specific to >what kind of a project we're dealing with and how you interface with it. >For example python projects probably don't have a configure step or >build directory, node projects may treat "npm install" as a configure >step but omit building, CMake projects support all steps through to >packaging and installing. Things don't generalise well here. > This implies removing many of the features project.el already has (project-compile, project-eshell, project-compile-buffer-name-function) + somehow create another level of abstraction over project.el... The initial basic commands (config, build, test/run) seems to be common, even in python projects it usually needs to get dependencies, create virtualenvs and so on. However if a project does not support a build command a simple message can solve it easily. Initially we are not even supporting the generate/configure, and the functionality is limited to: recognize compile and test already generated projects; but using some very primitive heuristics. More complex setups I thing must be left to more specialized external projects. These seems limiting, but it is much more than what we already have now. >I think Ergus you've actually touched on a fun corner case. You've got a >project with sub-projects and define a regex in a marker file to detect >this. I'll admit I haven't touched on this before, I know you can have a >CMake project with different sub-directories being considered their own >projects and buildable independently, but I've never seen this practice >in my day-to-day work. I'd be inclined to treat this as a project.el >feature, something akin to git sub-modules. My only concern with it as a >feature is its not trivial to check like git submodules. Basically every >directory in a CMake project can have a CMakeLists.txt file and >recursively searching upwards for one that might coincidentally also be >a project root marker isn't ideal :-(. > FWIK every directory may have a CMakeLists.txt, but they need to have a project() entry to be considered a project's CMake. The search is actually unavoidable and the regex check actually solves the race conditions. I know that the search is not efficient, but with a couple of optimizations if works very well even with tramp. Nested projects is something very common these days and supporting them in project.el is part of another parallel discussion going on already (very slowly btw). For now it is save enough to find the top-most one as it is what the default backend does. It will probably fit a high percent of all projects around. >While we're discussing interfaces for this project-type abstraction I'll >mention what I did for projection. In projection a project-type is an >eioio class. It has some slots like :predicate to check if the current >project matches that type and :build, :test, etc. for various project >specific commands that can be run. The list of all actively checked-for >project types is just a flat hook. I opted for this for 2 reasons: > >1. It's easy to modify. Every project-type can be edited directly you >can deregistered one from the monitored set of types by just removing it >from the collection. >2. It enforces a general ordering concept which is important for >determining a primary project type. I think moving the properties into a >generic function though would just decouple them and make it harder to >reason about and modify them. > More or less the same idea; I use with plist. I think they a bit simpler, but the same underground. I tend to avoid eioio because I have heard some performance complains (don't know if they are justified) but also because plists are somehow more flexible in some aspects. >In regards to interactivity, I outsourced a lot to another package I >maintain called compile-multi [3]. At its core its just a menu for >predicates and compile commands. When predicate X, you have the option >to choose command Y. I've plugged it into projection so that through it >you can see project type specific options. For example in a CMake >project this menu shows you all compilation targets and all CTest >targets. Running a selected target will run the compile command that >builds that target. If you want to permanently override one that runs >with the standard projection-commands-build-project or >projection-commands-test-project interactive commands (this is the same >as setting what you're wanting to build going forward) I recommend using >of embark [4]. What I normally do is run `M-x projection-multi-compile` >narrow to the build or test target I care about and then run `M-x >embark-act p c` (or p t for a test) and next time I run >projection-commands-build-project it'll use that selected command. If I >want to reset this to the project default I can use `M-x >projection-cache-clear` which shows all cached project options and lets >me select one or more to prune. Next time I run any command it'll be >the default for the current project. > >[3]: https://github.com/mohkale/compile-multi >[4]: https://github.com/oantolin/embark > Actually I am not trying to do anything so fancy. We cannot go from 0 to 100 in a single pass. If we start 1. recognizing projects backends and giving a better compile-command for them in the right place and to the right build-dir is a good step forward compared to what we have. Then we talk about test and finally we can consider the generate one. >I think generalising this latter feature to a common interface will be >tricky. To begin with just knowing a project has build target X doesn't >really tell you how to build X, just that you can. You need to feed that >selection back into the project-type or export it directly as is. I >recently added a feature to list build artifacts (for debugger >integration) and that shows just how much variety there is here. > >Lastly, just a general point (sorry for writing so much), I think any >feature for this shouldn't assume only a single project type is valid. >There should be a concept of a primary type just to make commands like >configure or build make sense relative to each other, but nothing stops >a project from matching two types at once and at least IME that's the >more natural state of things. I commonly work on C++ projects but we >also have a package.json in those projects file so we can depend on >prettier and other non-C++ specific linters. The way projection gets >around this is by always matching all defined project types. Users can >add to the list of matched types for the current project and can cycle >the primary project type. Certain interfaces that aren't single project >specific like the aforementioned projection-multi-compile will source >compilation targets from all applicable project types. This provides an >extremely rich interface for interacting with projects (at least IMO). > At the moment project.el only handles one backend at the time. I made a simple workaround to extend it a bit, but that is a bit tricky and at this point simplicity matters. Otherwise we end up with an inefficient and complex interface easy to break and hard to make it work. >Let me know if anyone has any thoughts or questions about projection. >I'm not averse to re-aligning across something in Emacs mainline. The >only minor concern is how specific it is to the kinds of projects I work >on and that may not generalise well to others. > >-- >Mohsin Kaleem > Ergus