From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tim Cross Newsgroups: gmane.emacs.devel Subject: Re: Subprojects in project.el (Was: Eglot, project.el, and python virtual environments) Date: Mon, 28 Nov 2022 15:10:55 +1100 Message-ID: <86cz97jzpi.fsf@gmail.com> References: <87zgcq68zp.fsf@ericabrahamsen.net> <84781346-5b88-2be5-38bb-02696fcf1364@yandex.ru> <87o7t2vj19.fsf@dfreeman.email> <877czqtyfy.fsf@dfreeman.email> <87zgcml7g7.fsf@gmail.com> <2ba04533-097a-a1da-ff3f-2c9506fd488e@yandex.ru> <875yf9bbzb.fsf@gmail.com> <87wn7oa0aw.fsf@gmail.com> <7a5b76fd-fb15-8c1e-ea29-bf11f7e0d2ae@yandex.ru> <87bkoya815.fsf@gmail.com> <0024a67d-b8e5-b35c-1b22-82541a170eb3@yandex.ru> <871qptai4d.fsf_-_@gmail.com> <86bkowdjx5.fsf@gmail.com> <43aa2f10-d947-dfcd-82b0-f6f1be3aaaec@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35905"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.9.3; emacs 29.0.50 Cc: =?utf-8?B?Sm/Do28gVMOhdm9yYQ==?= , Stefan Monnier , Danny Freeman , Eric Abrahamsen , emacs-devel@gnu.org To: Dmitry Gutov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Nov 28 08:39:27 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ozYjd-00096k-Q1 for ged-emacs-devel@m.gmane-mx.org; Mon, 28 Nov 2022 08:39:25 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ozYj0-0002FR-2o; Mon, 28 Nov 2022 02:38:46 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ozYix-0002F5-OQ for emacs-devel@gnu.org; Mon, 28 Nov 2022 02:38:43 -0500 Original-Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ozYiv-00044V-G3 for emacs-devel@gnu.org; Mon, 28 Nov 2022 02:38:43 -0500 Original-Received: by mail-pg1-x52c.google.com with SMTP id w37so4065258pga.5 for ; Sun, 27 Nov 2022 23:38:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:in-reply-to:date :subject:cc:to:from:user-agent:references:from:to:cc:subject:date :message-id:reply-to; bh=V9kFOW56JjkFOEe78HwzMuSdzY90I674qFju31xJ2A4=; b=F1euY1An1juB3lvDxN+vb5MLNcTQi1hHfvDt0mplm/8Q4MyiwRiCXu2Yx2Sobe/bKv Kot3p8yr/9LcndrK9CfIMZlqOtbTuCW1mxlCgda702pXd2kR8/Jum/+NbPRRpRKP0ApI iIKD2CvCNlrAny+eROrYj2GSS1m/kad1tdLVaUMTVWrfnOdpev3Mue6pKlwy7420XMEE b5jDDwW03ospU3ygq+x3JFq+CLIjVMeVocpJGe8KJD4EWlbEdxste4lJg6Smm0bVvRaI /Wt3eB8/Se/uRy5YJM4cLF/omWoWMFQFH3zfmFv0adn7ZGx98ILGeRNpr5Z+6vkM5SSA OB4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:in-reply-to:date :subject:cc:to:from:user-agent:references:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=V9kFOW56JjkFOEe78HwzMuSdzY90I674qFju31xJ2A4=; b=IhfILXKTagB2ffAKvoPomgio7LQ3T/0e5toT5CmlKMFYzprCOh6cdxRmWtWjHRPKjC SdlYatP2gJ6fMb7f0SvHc7GRkT+mFFF1BSIt9jb+ziInCia3vLmyVm4KajxVlmgTeAB9 h23IXAhfZ4JLzS8JczVnnB4WPpvZDRkc+iULYIboSgPk3zBRiY6MxqkupMzmOEv6QCjR pOQkIgnv0715Q2MOwphtlEm6aujzucxeKIa8EIGHWRoLqTAeDIIqyQ88s7tHfYOfVonu J4FNzVVgI6uyvMjbTw+dQDxvoQn/42Y2i3gb0FD0ZpjXN/c618AR3yncxHxlWAdQKYxE RpQg== X-Gm-Message-State: ANoB5pmGnOEbrxzMIzA5f5pjFHMhpH0mUkgXwsjwgWajeYDe3VcG2FJJ Ah0N6CUQaN8tx+PxH+o6pDA26WiBOrg= X-Google-Smtp-Source: AA0mqf7RXMqhAQ6PIkZ5zg4EyU9aQMaHuDap3Fp0tbZIxW+m2qfKdzTe3WsGI3O9VjH1GkR9U1kxBQ== X-Received: by 2002:a63:ec45:0:b0:470:4320:ef39 with SMTP id r5-20020a63ec45000000b004704320ef39mr26838916pgj.381.1669621118794; Sun, 27 Nov 2022 23:38:38 -0800 (PST) Original-Received: from dingbat (203-173-24-107.dyn.iinet.net.au. [203.173.24.107]) by smtp.gmail.com with ESMTPSA id g8-20020a1709026b4800b001868981a18esm8089356plt.6.2022.11.27.23.38.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Nov 2022 23:38:38 -0800 (PST) In-reply-to: <43aa2f10-d947-dfcd-82b0-f6f1be3aaaec@yandex.ru> Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=theophilusx@gmail.com; helo=mail-pg1-x52c.google.com X-Spam_score_int: -4 X-Spam_score: -0.5 X-Spam_bar: / X-Spam_report: (-0.5 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_03_06=1.592, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:300663 Archived-At: Dmitry Gutov writes: > On 25/11/22 00:46, Tim Cross wrote: >> Jo=C3=A3o T=C3=A1vora writes: >>=20 >>> On Thu, Nov 24, 2022 at 3:01 AM Dmitry Gutov wrote: >>> >>>=20=20=20 >>>> I'm imagining that traversing a directory tree with an arbitrary >>>> predicate is going to be slow. If the predicate is limited somehow (= e.g. >>>> to a list of "markers" as base file name, or at least wildcards), 'g= it >>>> ls-files' can probably handle this, with certain but bounded cost. >> I've seen references to superior performance benefits of git ls-file a >> couple of times in this thread, which has me a little confused. >> There has been lots in other threads regarding the importance of not >> relying on and not basing development on an underlying assumption >> regarding the VCS being used. For example, I would expect project.el to >> be completely neutral with respect to the VCS used in a project. > > That's the situation where we can optimize this case: when a project is G= it/Hg. > >> So how is git ls-file at all relevant when discussing performance >> characteristics when identifying files in a project? > > Not files, though. Subprojects. Meaning, listing all (direct and indirect= ) subdirectories > which satisfy a particular predicate. If the predicate is simple (has a p= articular project > marker: file name or wildcard), it can be fetched in one shell command, l= ike: > > git ls-files -co -- "Makefile" "package.json" > > (which will traverse the directory tree for you, but will also use Git's = cache). > > If the predicate is arbitrary (i.e. implemented in Lisp), the story would= become harder. > >> I also wonder if some of the performance concerns may be premature. I've >> seen references to poor performance in projects with 400k or even 100k >> files. What is the expected/acceptable performance for projects of that >> size? How common are projects of that size? When considering >> performance, are we not better off focusing on the common case rather >> than extreme cases, leaving the extremes for once we have a known >> problem we can then focus in on? > > OT1H, large projects are relatively rare. OT2H, having a need for subproj= ects seems to be > correlated with working on large projects. > > What is the common case, in your experience, and how is it better solved?= Globally > customizing a list of "markers", or customizing a list of subprojects for= every "parent" > project? In my personal experience, sub-projects have been more about project structure and not size. I would agree they are more prevalent in large projects, but can exist in medium and even smaller projects. I don't think I have a preference for customizing a list of markers or a list of sub project definitions per project. I suspect different approaches will work better in different scenarios and neither is a clear 'winner'. However, as pointed out by Stephan, terminology confusion/meaning may well be contributing to the confusion here. Not only am I unsure everyone is thinking the same thing when talking about sub-projects, I'm not sure everyone is even talking about the same thing when referencing 'project'. I wrote a lot about how I use projects and sub-projects in my work flow and then realised it probably isn't helping that much. It struck me that perhaps the issue is that the notion of sub-projects isn't really that useful in itself and may actually be more detrimental than useful. When you think about it, a sub-project is really just a more narrow project focus. A project is really just a collection of files and environment settings which can be considered, for some purpose, as a 'unit' in itself. It might define the set of files used when considering find and replace for a symbol, when looking for symbol completion candidates, or file/buffer switching, opening, linting, cross referencing etc. It may correspond to a VCS repository, but it may not. It could cut across repositories, or it could be made up of multiple repositories or it could simply be some bespoke virtual project concept specific to a particular use case. I guess what I want is the ability to define arbitrary collections of files and environment settings as a project, have a way to select/target a project and an API which various tools can use to get the files or environment settings to then operate on. Whether one project can be considered a sub-project of another project is less relevant compared to the ability to select/identify the target project. Automatic definition of projects based on VCS repositories is great and a real time saver, but the ability to define what makes up a project manually is also important. The ability of the system to automatically determine which project is 'active' (for example, based on the location of the file being opened) is good and having the system prompt you when it isn't clear or when there are multiple options is useful, but just being able to run a command to set the current project would also be sufficient. However, how one project relates to another project i.e. sub project, main project, etc, seem of limited use compared to just having the ability to select a sub-set of the files and environment settings of a project, whether we call these sub project or nested projects or whatever, seems of limited benefit.