From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: Subprojects in project.el (Was: Eglot, project.el, and python virtual environments) Date: Fri, 25 Nov 2022 01:38:08 +0200 Message-ID: <43aa2f10-d947-dfcd-82b0-f6f1be3aaaec@yandex.ru> References: <87zgcq68zp.fsf@ericabrahamsen.net> <4c5f4b07-3df6-d700-83f8-9a9d1b684afc@yandex.ru> <84781346-5b88-2be5-38bb-02696fcf1364@yandex.ru> <87o7t2vj19.fsf@dfreeman.email> <877czqtyfy.fsf@dfreeman.email> <87zgcml7g7.fsf@gmail.com> <2ba04533-097a-a1da-ff3f-2c9506fd488e@yandex.ru> <875yf9bbzb.fsf@gmail.com> <87wn7oa0aw.fsf@gmail.com> <7a5b76fd-fb15-8c1e-ea29-bf11f7e0d2ae@yandex.ru> <87bkoya815.fsf@gmail.com> <0024a67d-b8e5-b35c-1b22-82541a170eb3@yandex.ru> <871qptai4d.fsf_-_@gmail.com> <86bkowdjx5.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28834"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Cc: Stefan Monnier , Danny Freeman , Eric Abrahamsen , emacs-devel@gnu.org To: Tim Cross , =?UTF-8?B?Sm/Do28gVMOhdm9yYQ==?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Nov 25 00:39:02 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oyLo5-0007J4-LY for ged-emacs-devel@m.gmane-mx.org; Fri, 25 Nov 2022 00:39:01 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oyLnM-0000Az-1M; Thu, 24 Nov 2022 18:38:16 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oyLnL-0000An-0E for emacs-devel@gnu.org; Thu, 24 Nov 2022 18:38:15 -0500 Original-Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oyLnI-0002mh-Pe for emacs-devel@gnu.org; Thu, 24 Nov 2022 18:38:14 -0500 Original-Received: by mail-wr1-x436.google.com with SMTP id b12so4536055wrn.2 for ; Thu, 24 Nov 2022 15:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=yV5fp7yFnF2tiHyfmB0VqoB6C10o091YKzNe66waG1A=; b=RNphdbRjOtgRDUEJVH6xMuef+bMDBnrq5axx2Tl4b7fM1/wtC8WEts6xv7ToCAvyBH I3uEpKo4DWy7Trlq5aKf11dah0m95zE4cNobwaMXwiRvjIqTx7177suuc5DRgWO3Ry+h Yc4CugqI5sLCaV38lc2VRqIzmgoaCONE0VA3FUote4sebFvtOTdRd0Dg6az9TgEkVJBm bX7eAMzU4RjPq5OrZbh+a2nyU9pW8aydd0YcVQIuGSSJZbwG5YhKFj4/QFHSGXjU4Rlf VLiZVF//YkH909AK+FCZ/yMh+JO9KJGqmvQLFHln8O/AdSmSAI56JnOMchC57IsKzMhp CoSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yV5fp7yFnF2tiHyfmB0VqoB6C10o091YKzNe66waG1A=; b=qGdTcDc5FP3+S8vETkRCEZbJ1z0HF0gPr4NGDZfcL12XdXgp3DZCnt4JCjiQTKHhIY cPQmQt1a54k1Ho8LAEaJ7Bf0SJDmRyC59rF7IglJN+E199optolUpACmcngZjfv4tm6U QHSRoVTbBmwIg0Ki7sBp6zrmbd2tbyWA+VzYZY+84v5addsbsGnNo9vD9eQki3z/xl+T 3qO3OdtS2JrvKxwwbRWfMj6GSMFAwP35aX/Jx5qn1ANvnsT+kngUqLVeleb5TJgfuCLN fzLNEJ3tTla6t09LgH0+ZU3Dvw57NjsjF3l0wTvgQjTkX15ZkvL3OHbxLVIUNdsdpkli xXYg== X-Gm-Message-State: ANoB5pklN93kfR1bh9q7FFrHHX0SX2WQof8qTpWI0VDdLpYAv25oJs0i qn0VlqWYMkb2qAZDwcPjN7I= X-Google-Smtp-Source: AA0mqf5lftRjocfH/7Yw7VvzOIla6kxWDwwYwvCA1a0n1enkfsiUJhcDbv5HIGL+KmFwGAh8UYPVgw== X-Received: by 2002:adf:eb92:0:b0:236:80a8:485e with SMTP id t18-20020adfeb92000000b0023680a8485emr14549776wrn.362.1669333090794; Thu, 24 Nov 2022 15:38:10 -0800 (PST) Original-Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id t2-20020a5d42c2000000b0023662d97130sm2396688wrr.20.2022.11.24.15.38.09 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 24 Nov 2022 15:38:10 -0800 (PST) Content-Language: en-US In-Reply-To: <86bkowdjx5.fsf@gmail.com> Received-SPF: pass client-ip=2a00:1450:4864:20::436; envelope-from=raaahh@gmail.com; helo=mail-wr1-x436.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.209, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:300453 Archived-At: On 25/11/22 00:46, Tim Cross wrote: > > João Távora writes: > >> On Thu, Nov 24, 2022 at 3:01 AM Dmitry Gutov wrote: >> >> >>> I'm imagining that traversing a directory tree with an arbitrary >>> predicate is going to be slow. If the predicate is limited somehow (e.g. >>> to a list of "markers" as base file name, or at least wildcards), 'git >>> ls-files' can probably handle this, with certain but bounded cost. > > I've seen references to superior performance benefits of git ls-file a > couple of times in this thread, which has me a little confused. > > There has been lots in other threads regarding the importance of not > relying on and not basing development on an underlying assumption > regarding the VCS being used. For example, I would expect project.el to > be completely neutral with respect to the VCS used in a project. That's the situation where we can optimize this case: when a project is Git/Hg. > So how is git ls-file at all relevant when discussing performance > characteristics when identifying files in a project? Not files, though. Subprojects. Meaning, listing all (direct and indirect) subdirectories which satisfy a particular predicate. If the predicate is simple (has a particular project marker: file name or wildcard), it can be fetched in one shell command, like: git ls-files -co -- "Makefile" "package.json" (which will traverse the directory tree for you, but will also use Git's cache). If the predicate is arbitrary (i.e. implemented in Lisp), the story would become harder. > I also wonder if some of the performance concerns may be premature. I've > seen references to poor performance in projects with 400k or even 100k > files. What is the expected/acceptable performance for projects of that > size? How common are projects of that size? When considering > performance, are we not better off focusing on the common case rather > than extreme cases, leaving the extremes for once we have a known > problem we can then focus in on? OT1H, large projects are relatively rare. OT2H, having a need for subprojects seems to be correlated with working on large projects. What is the common case, in your experience, and how is it better solved? Globally customizing a list of "markers", or customizing a list of subprojects for every "parent" project?