From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Bj=C3=B6rn?= Bidar Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Sun, 29 Dec 2024 21:06:01 +0200 Message-ID: <49542.0174951263$1735514206@news.gmane.org> References: <1ed88fca-788a-fe9f-b6c8-edb2f49751c9@mavit.org.uk> <67428b3d.c80a0220.2f3036.adbdSMTPIN_ADDED_BROKEN@mx.google.com> <86ldwdm7xg.fsf@gnu.org> <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com> <00554790-CACA-4233-8846-9E091CF1F7AA@gmail.com> <86msgl2red.fsf@gnu.org> <87o710sr7y.fsf@debian-hx90.lan> <8734i9tmze.fsf@posteo.net> <86plldwb7w.fsf@gnu.org> <86h66pw4sd.fsf@gnu.org> <947A7DB0-43F7-4288-8FBF-0984FCFFAA93@dancol.org> <663726A2-141B-4B98-80FB-BD93E99AC122@dancol.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="37328"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , casouri@gmail.com, emacs-devel@gnu.org To: Daniel Colascione Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Dec 30 00:16:38 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tS2WT-0009XC-JQ for ged-emacs-devel@m.gmane-mx.org; Mon, 30 Dec 2024 00:16:37 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tS2VU-0000KO-Ba; Sun, 29 Dec 2024 18:15:36 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tS2VR-0000K8-IP for emacs-devel@gnu.org; Sun, 29 Dec 2024 18:15:33 -0500 Original-Received: from thaodan.de ([2a03:4000:4f:f15::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tS2VG-0007ry-Ee; Sun, 29 Dec 2024 18:15:25 -0500 Original-Received: from odin (dsl-trebng12-50dc7b-49.dhcp.inet.fi [80.220.123.49]) by thaodan.de (Postfix) with ESMTPSA id 76BFED00030; Mon, 30 Dec 2024 01:15:10 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=thaodan.de; s=mail; t=1735514110; bh=b9wZEg0ao46arHI16TrKNqxawmmtIz9hRAEwpKBTrEc=; h=From:To:Cc:Subject:In-Reply-To:Date:References; b=Nw57bZxVpl/PIxLqIillQJ4hYZM0sPScEkOaDS/RhfV7lHm62/DrxwfmIsCtmSaX3 6fYIDhrRRbh34ftqDXXB+XQKfLuaIRBJb/LZRR30hsVsL5rHFEHUpEXL0owROf1e25 VHv8NlEONajYl9BLi3knkojxMxUHRyqZkD9LNKhWkaRTu8IrBg2gVr08oAZDA3jiS/ FNrMdGQ18GDwh3C9UPNvbtoTLNeDfDhJbBJXmuVNRutkf4kzf8S2n+UafkuBFhhhsu e9BOZ4jJkNUlqvk+oBUaZYWuxGAfG7d5z262s6LlC2v0ciZibHVdEc4RoJt2Icx6X8 EQX7a5W1+ufKJSaPy44U9cex4ax6GYuFokTYzbAcdRDVsllZiZynOLj/F9scdalzBn tIz0LoxZqi4Zeah/xVaM+J+JPvCZTT2vdjoz0Z7brU0xYmO68ggOTPTXr455jadDIF alxyAiWowJqz9OG+7rn30WybojBcxNd8n9jZ54iwSTmPEjf6jfU3vYWxGaT+J76jci vfUoiX2r0SevPx0HLD2SD8YWeb/8VTjtVc9X805a6Mj8X6QgxPy8CLB1Vv1NOnZjH3 JPkrBDjW/5qj3qZacZo7AdJ3EOyRPoerUTwWWcvyoLm4iD//W9FEoSqHlsjXx9zvT8 6VP6RkBJtmjICUbu2kCk3Dxo= In-Reply-To: <663726A2-141B-4B98-80FB-BD93E99AC122@dancol.org> (Daniel Colascione's message of "Sun, 29 Dec 2024 11:13:03 -0500") Autocrypt: addr=bjorn.bidar@thaodan.de; prefer-encrypt=nopreference; keydata= mDMEZNfpPhYJKwYBBAHaRw8BAQdACBEmr+0xwIIHZfIDlZmm7sa+lHHSb0g9FZrN6qE6ru60JUJq w7ZybiBCaWRhciA8Ympvcm4uYmlkYXJAdGhhb2Rhbi5kZT6IlgQTFgoAPgIbAwULCQgHAgIiAgYV CgkICwIEFgIDAQIeBwIXgBYhBFHxdut1RzAepymoq1wbdKFlHF9oBQJk1/YmAhkBAAoJEFwbdKFl HF9oB9cBAJoIIGQKXm4cpap+Flxc/EGnYl0123lcEyzuduqvlDT0AQC3OlFKm/OiqJ8IMTrzJRZ8 phFssTkSrrFXnM2jm5PYDoiTBBMWCgA7FiEEUfF263VHMB6nKairXBt0oWUcX2gFAmTX6T4CGwMF CwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQXBt0oWUcX2hbCQEAtru7kvM8hi8zo6z9ux2h K+B5xViKuo7Z8K3IXuK5ugwA+wUfKzomzdBPhfxDsqLcEziGRxoyx0Q3ld9aermBUccHtBxCasO2 cm4gQmlkYXIgPG1lQHRoYW9kYW4uZGU+iJMEExYKADsCGwMFCwkIBwICIgIGFQoJCAsCBBYCAwEC HgcCF4AWIQRR8XbrdUcwHqcpqKtcG3ShZRxfaAUCZNf2FQAKCRBcG3ShZRxfaCzSAP4hZ7cSp0YN XYpcjHdsySh2MuBhhoPeLGXs+2kSiqBiOwD/TP8AgPEg/R+SI9GI9on7fBJJ0mp2IT8kZ2rhDOjg gA6IkwQTFgoAOxYhBFHxdut1RzAepymoq1wbdKFlH Received-SPF: pass client-ip=2a03:4000:4f:f15::1; envelope-from=bjorn.bidar@thaodan.de; helo=thaodan.de X-Spam_score_int: 1 X-Spam_score: 0.1 X-Spam_bar: / X-Spam_report: (0.1 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_03_06=1.592, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, INVALID_MSGID=0.568, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327375 Archived-At: Daniel Colascione writes: > On December 29, 2024 11:02:47 AM EST, "Bj=C3=B6rn Bidar" wrote: >>Daniel Colascione writes: >> >>> On December 29, 2024 10:05:26 AM EST, "Bj=C3=B6rn Bidar" >>> wrote: >>>>Daniel Colascione writes: >>>> >>>>> On December 27, 2024 9:59:14 AM EST, Eli Zaretskii wro= te: >>>>>>> Date: Fri, 27 Dec 2024 08:46:06 -0500 >>>>>>> From: Daniel Colascione >>>>>>> CC: rms@gnu.org, manphiz@gmail.com >>>>>>>=20 >>>>>>> >> It might take a while for that to happen, which is why I still >>>>>>> >> believe >>>>>>> >> it would be better if tree-sitter major modes would populate >>>>>>> >> `treesit-language-source-alist' on their own, and point to the >>>>>>> >> specific >>>>>>> >> checkouts that the major mode developer tested their implementat= ion >>>>>>> >> against. >>>>>>> > >>>>>>> >We could have done that, but there's no way we could keep the valu= e of >>>>>>> >treesit-language-source-alist up-to-date, because the grammar >>>>>>> >libraries put out new versions much more frequently than Emacs >>>>>>> >releases, especially if you consider libraries that have no offici= al >>>>>>> >versions at all (in which case we can only point to some revision = in >>>>>>> >their repository). >>>>>>> > >>>>>>> >The question that bothers me is how useful is it to have >>>>>>> >treesit-language-source-alist that is outdated? What do we expect= the >>>>>>> >users to do with such an outdated value? >>>>>>> > >>>>>>>=20 >>>>>>> Why not just vendor all the grammars with the Emacs modes that >>>>>>> use them? >>>>>> >>>>>>We'd need to ask their developers to agree to this.=20 >>>>> >>>>> Why? They're free software. For copyright assignment? Seems like an >>>>> exception would make sense here. >>>>> >>>>>> Other than that, >>>>>>I don't see how is that different from pointing to a specific version >>>>>>of each grammar: both will be outdated a short time after we point to >>>>>>the version or release Emacs with that version. >>>>>> >>>>>>So why do you think this is better? >>>>> >>>>> Vendoring enables building a full featured Emacs without a network >>>>> connection and guarantees build reproducibility in perpetuity. >>>> >>>>Did you think of the long term consequences? >>>> >>>>The embedded dependencies would have to be maintained first by Emacs and >>>>later by packagers. >>>> >>>>All the infrastructure around syncing of grammars is time spend that >>>>could spend on more long term efforts such as stabilizing the >>>>tree-sitter based modes to not break as easy on grammar changes or to >>>>improve tree-sitter it self. >>> >>> I've vendored plenty of things. Works fine in practice. Big programs >>> like Firefox vendor the world too, and they work fine. It's really not >>> that much work. It eliminates entire classes of problem. It's going to >>> take more time to deal with the problems of taking a dependency and >>> the headaches of not having a stable interface than it would to set up >>> a few git subtrees or submodules and invoke their build system from >>> that of Emacs. >> >>Big programs like Firefox vendor the world only for packagers to have >>to revert those.=20 > > These packagers are wrong, FWIW. Unbundling is needless and often > introduces bugs. It introduces bugs in software with generally unstable APIs/ABIs or software using unfinished/unreleased versions of software. Partially the problem we are facing here right now. The rat tail of issues this can entrail can be long. FWIW You called over half of the Unix community wrong where bundled dependencies are frowned or forbidden upon. > In the mobile world, popular OSes seldom provide > libraries. Apps are expected to bundle their dependencies. The sky > doesn't fall. In fact, the mobile app ecosystem is healthier and more > secure than the desktop one precisely because it isn't burdened by > ideas that no longer make sense in a modern technical context, > e.g. that apps should casually share libraries. I work on mobile operating systems, what you describe is double edged sword. The applications size's increase and the party to blame for security issues moves from the os to the application developer. The mobile OS would had plenty of issues from this practice notably for example the log4you debacle. Most mobile operating systems provide their own set of available libraries, apps are not expected to bundle dependencies unless they are not available for that OS. Part of the issue is that library dependencies moving faster than many operating systems can or with stable APIs. The end result of such lets bundled everything approach is that you have use the exact chain of dependencies to build a software which is awesome if you like the fire and forget approach of software development. To me what you write reads like mobile operating systems =3D JavaScript/AI = developers. >> Vendoring only works long enough until the dependencies >>you have vendored are not out of date.=20 > > It doesn't matter whether the dependency is out of date so long as > it's in sync with code that interacts with it. It's even worse when > the dependency doesn't make any compatibility guarantees. IMHO, the > only reasonable way to consume a dependency with an unstable interface > is to bundle or hash-lock or outright vendor it. If you have software which has short life cycles this can work but I don't think this works for Emacs. Further bundled dependencies require to patch the software bundling the dependency to fix bugs and security issue. Bugfixes are not really an issue with grammars but with libtree-sitter which Emacs depend on. Putting bundled grammars and libtree-sitter in this equation makes it harder to maintain the Emacs package since I have to watch to not break the embedded grammars when updating tree-sitter or it's dependencies. >> It is something which only works >>in projects who control most of their dependency chain and/or have a >>fire and forget approach of software development. >> >>> It's not even the precise mechanics: pulling down a grammar by hash is >>> tantamount to just checking in the grammar, but with more moving >>> parts. You still pair one to one the grammar and the Lisp code meant >>> to use it so you don't end up chasing down weird compatibility >>> issues. IMHO, since they're tightly coupled anyway, we might as well >>> distribute them together. >>> >>> As for changing TS grammars not to break: why do you think that would >>> be feasible? So far TS grammar authors haven't felt particularly >>> obligated to maintain compatibility. >> >>I don't know exactly to be honst but I don't think we are alone with >>this issue. If we are we should check out it is handled in other >>editors. > > You're right that this is a problem everyone should be hitting. Maybe there's a way around it. The only way is to reach out to other projects using the library or upstream.